Patent Application
20070220492
Some software programming code errors can be difficult to identify with normal application testing. Generally described, an application verification tool is a software program that provides verification of the execution of programming code. In a typical application verification tool, the tool is configured to conduct a series of tests to detect and/or help to debug various common programming mistakes (errors) related to memory corruptions, hangs, handle usage, and other critical vulnerabilities. Based on the outcome of the series of tests, the errors can be identified to a user.
In a typical embodiment, an application verification tool includes an engine for monitoring the execution of a tested software application. The engine monitors the application's behavior by intercepting Application Programming Interface (API) calls. An application verification tool further includes an IAT replacement engine for providing processing the intercepted API calls. More specifically, in one embodiment, the IAT replacement engine enables specially written software modules located in dynamic link libraries (DLLs), generally referred to as API shims, to be inserted between an application and the operating system. The API shim DLLs intercept standard API calls to the modules of the operating system and facilitate the execution of the series of tests. The current approach to using a separate IAT replacement engine with extra configuration requirement can limit the flexibility of the application verification tool. In one aspect, the shim DLLs are typically statically created and managed. Additionally, each API shim DLL is tightly coupled with a corresponding verification procedure. This static nature of the IAT replacement engine prevents the application verification tool from being easily integrated into various computing environments other than the environment in which the API shims were created for. Further, current application verification tool approaches do not typically provide a standardized infrastructure for allowing additional tests (checks) to be integrated into the tool. Such lacking of the standardized infrastructure severely limits the scalability of the application verification tool.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
A verifier infrastructure for a runtime verification tool is provided. The verifier infrastructure includes a managing module and one or more verifier modules. The verifier modules provide the verification for checks (tests) to predict how well an application will perform under a certain runtime setting. Each verifier module is configured to be plugged into the runtime verification tool in a standardized way. The managing module manages the verifier modules by transmitting runtime settings to the verifier modules and collecting information regarding testing of an application from the verifier modules. The collected information is reported back to users for detecting and/or debugging runtime errors.
In accordance with an aspect of the invention, a method for collecting information regarding execution of an application is provided. A computing device obtains verifier information from at least one verifier layer. The computing device then advertises a descriptor of the verifier layer to the user. The computing device obtains runtime settings associated with the execution of a software application on a computer system. The computing device transmits the runtime settings to the verifier layer. The communication between the verifier layer and the computing device is normalized and structured in such a way that the knowledge of how settings are stored and applied is not specific to the implementation of the verification layer.
The computing device further collects verification events information from the verifier layer. The computing device then processes the collected verification events information and reinterprets the processed information. In one embodiment, the computing device may provide a log of information to users. The runtime settings may be used to ensure that the application does not hide access violation by use of exception handling, to verify that the application properly manages virtual memory space, or to track unsafe use of API.
In accordance with another aspect of the invention, a method for providing information regarding execution of an application is provided. A computing device transmits verification information to a managing module. The computing device obtains a runtime setting and then evaluates the obtained runtime setting. Upon evaluation, the computing device updates verification code based on the runtime setting. This allows the verification code to be enabled or disabled based on the runtime setting. While the application is running, the enabled verification code will be executed. The computing device collects execution tracing information (which may be processed and converted in log entries or in debugger events, according to the user choice and preference) regarding the execution of the enabled verification code. Then, the computing device provides normalized execution tracing information to the managing module.
In accordance with yet another aspect of the invention, a computer-readable medium having computer-executable components for implementing runtime verification of an application is provided. The computer-readable medium includes verifier layer components that provide verification code. The computer-readable medium further includes a core module component that manages the verifier layer components. The core module component passes runtime settings to the verifier layer components. The verifier layer components dynamically evaluate the received runtime settings to determine whether to execute verification code associated with the verifier layer components. The core module component causes the application to continue to run after the verifier layer components have dynamically evaluated the runtime setting. The core module component collects verification events information from the verifier layer components and then processes the verification events information. The core module component transmits the verification events information to help users to detect and/or debug runtime errors.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
Generally described, the present invention relates to a system and method for providing runtime verification utilizing a flexible and scalable verifier infrastructure. More specifically, the present invention relates to verifier modules utilized in conjunction with a verifier managing module for providing a flexible and scalable verifier infrastructure. Although the present invention will be described with relation to illustrative embodiments utilizing an illustrative verifier infrastructure and verifier modules, one skilled in the relevant art will appreciate that the disclosed embodiments are illustrative in nature and should not be construed as limiting.
With continued reference to
The computer system 100 further includes a front engine module 108 that allows users to specify an application to be verified and to select the desired testing for the application. Based on the user selection for the desired testing, a loader in conjunction with the engine module 104, loads the core module 106 and several verifier layers 102 corresponding to the desired testing. Generally described, a loader, a part of the operating system, is automatically invoked when a processor is run. The loader typically loads the executable code of a program into memory for execution. In one embodiment, the core module and the verifier layers are loaded earlier than any other executable code. In this embodiment, the verifier layers 102 can be suitable for verifying checks such as security checks, list of user accounts (LUA), or the like. As described above, the engine module 104 deploys the core module 106 and the verifier layers 102 after the core module 106 and the verifier layers 102 have been loaded into memory. The verifier layers 102 include a simple protocol to declare to the engine module 104 and the core module 106 what interfaces are verified at the time of deployment. After the core module 106 and the verifier layers 102 are deployed, the verifier layers 102 register with the core module 106. This registration advertises a set of configurable properties of verification layers, a set of verification events, a set of interfaces to be verified, and the like. The core module 106 may pass runtime settings to the verifier layers 102, based upon the set of advertised configurable properties. Furthermore, the core module 106 may instruct the verifier layers 102 about actions of the verification events. The verification events may cause generation of a log entry and/or a debugger event in accordance with the instruction.
In one embodiment, the front engine module 108 may provide various user interfaces suitable for each user's display device. In an alternative embodiment, the computer system 100 may include an additional user interface module to allow a user to create and/or modify a command line for debugging errors that have been detected by the verifier layers. The command line may include runtime settings which will be used to change behaviors of APIs via the verifier layers.
With reference to
In an illustrative embodiment, more than one verifier layer may include corresponding verification code which defines replacement functions for a certain API. In such a case, verification code in each verifier layer may form a linked list to redirect the entry point from the verification code in one verifier layer to the verification code in the next verifier layer, thereby redirecting the API to all of the available verification code in the verifier layers. Alternatively, the engine module 104 may inject several verifier layers in a hierarchical order.
As will be described in greater detail, the core module 106 manages the verifier layers 102 and provides various services to the verifier layers 102. Examples of the services provided by the core module 106 include providing instructions for which verification events information is to be collected, logging information provided by the verifier layers 102, processing the log information, and reporting the processed log information to users, among others. Further, the core module 106 provides common facilities which are needed by all verifier layers 102. For example, the core module 106 maintains the global knowledge relating to active verifier layers in the computer system 100. Although each verifier layer does not have information regarding activities of other verifier layers, a verifier layer can receive such information from the core module 106 when it is necessary.
After having been loaded, the verifier layers 102 register with the core module 106 so that runtime settings and instructions are passed along to the verifier layers 102. In an embodiment, the verifier layers 102 register a callback function to the core module 106, which will be called by the core module to pass runtime settings when the core module receives the runtime settings. In this manner, the core module 106 and the verifier layers 102 are loosely coupled and various numbers of verifier layers can be plugged into the verifier infrastructure of the computer system 100 in a standardized way without requiring any code changes in the core module 106.
In an embodiment, runtime setting can be used to change behaviors of certain APIs in order to verify desired tests (checks). Based on the instructions from the core module, each of the verifier layers 102 obtains selective information regarding the behaviors of the API's as the application 112 is running. Each of the verifier layers 102 provides the obtained information to the core module 106. The core module 106 receives verification events information from the verifier layers 102 and then process the verification event information. In one embodiment, the core module 106 generates log information based on the processed verification events information and transmits the log information to users 114. In alternative embodiment, the core module 106 transmits the verification event information to the front engine module 108 which processes the verification event information to produce a desired report for the users 114.
As shown in
With reference to
In an illustrative embodiment, the verifier provider 400 can define any suitable verifier layers to be injected into the verifier infrastructure. For example, the verifier provider 400 may define a verifier layer that is configured to perform the verification for checks to determine memory corruptions issued in a heap allocation, to ensure the correct use of critical sections, Thread Lock Storage (TLS) APIs, and the like. The verifier provider 400 may define a verifier layer to perform the verification for checks to ensure that the application is not attempting to use invalid handles, or that returned thread in a thread pool is in a safe condition of reuse. In operation, the verifier layer includes various replacements 404 with which the engine module 104 hooks (replaces or redirects) system interfaces in order to track various objects or verify correctness of operations. For example, the replacements 404 may be used to track dangerous API to check if the application is using unsafe APIs, or to ensure that the application does not hide an access violation using structured exception handling. Similarly, the replacements 404 may be used to ensure that APIs for virtual space manipulation are correctly used in the application. Other examples of replacements 404 may include replacements for ensuring a proper user privilege by tracking LUA and a proper security check. One of ordinary skill in the art will appreciate that the verifier provider 400 is application environment independent. Further, any number of verifier providers can be added on to the verifier infrastructure, defining various verifier layers. One verifier provider may define one or more verifier layers. In one embodiment, the front engine module 108 discovers new verifier providers. As described above, the information relating to the verifier providers may be exposed through a set of normalized interfaces between various components in the computing system 100. In this embodiment, the front engine module 108 may provide a user interface to advertise the information relating to verifier layers, such as a set of configurable properties and a set of verification events, and to allow the user to configure desired verifier layers, a set of configurable properties of the configured verifier layers, and the like.
With reference to
With reference to
In accordance with an illustrative embodiment, each verifier layer 102 dynamically evaluates the runtime setting to determine whether to execute its corresponding verification code. Upon evaluation, each of the verifier layers 102 enables or disables its corresponding verification code (i.e., replacement functions). For example, replacement function 132 in verifier layer 1 is enabled and replacement function 134 in verifier layer N is disabled after the runtime setting evaluation. In this manner, the runtime setting is used to verify the desired checks by executing replacement function 132 but not replacement function 134. In one embodiment, the core module 106 transmits the runtime settings to a particular verifier layer in order to change one of its runtime configurable properties. In this case, the replacement function is always executed, but its specific behavior is changed. For example, replacement function for a memory allocation API can change the rate of injected failures. After the execution, information regarding execution of the replacement function 132 is gathered and provided to the core module 106.
With reference new to
At block 604, the core module 106 obtains desired runtime settings associated with a processor (a runtime application) on a computer system. As described above, the desired runtime settings may include a runtime setting to ensure that the application does not hide access violation caused by use of exception handling, that the application properly manages virtual memory space, and the like. The desired runtime settings further include a runtime setting to track unsafe use of API. In some instances, the desired runtime setting includes a runtime setting to create a failure in order to test the behavior of the processor at the time of the failure. Further, the desired runtime settings include a runtime setting to simulate a low resource condition, such as being low on a class of resources and the like, in the computer system 100.
At block 606, the core module 106 transmits the runtime settings to the verifier layers 102. In an embodiment of the present invention, each registered callback function may be used to transmit the runtime settings. At block 608, the core module 106 collects verification event information from the verifier layers 102. As will be described in greater detail, each of the verifier layers 102 obtains the configuration information regarding the execution of its corresponding verification code, such as information regarding the behaviors of the replacement functions and the verification events. Based on the configuration information of the verification events, the verifier layers 102 gather information regarding the execution of the enabled verification code and provide the verification event information to the core module. The core module 106 then processes the verification event information, applies normalized processing (such as collection of context information and stack traces). The core module 106 generates and transmits the log information to users at block 610. In one embodiment, the log information is contained in a file format, such as XML format, suitable for being used in each user's display device. The verification routine 600 terminates at block 612.
With reference to
While the application 112 is running, a verifier layer examines whether its corresponding verification code is enabled with respect to the runtime setting at decision block 706. If the answer is no, indicating the corresponding verification code is disabled, the routine proceeds to decision block 710. If the answer is yes indicating that the corresponding verification code is enabled with respect to the runtime setting, the enabled verification code is executed to replace the API call in the application 112 at block 708. In an illustrative embodiment of the present invention, the core module 106 obtains a range of desired action and settings for the verification events as verification event configuration selected by a user. The core module 106 sends log instructions to the verifier layers for instructing which verification event information should be collected. Based on the log instructions, the verifier layer collects verification event information regarding the execution of the enabled verification code. The verifier layer transmits the collected verification event information to the core module 106 at block 709. The runtime setting evaluation routine 700 proceeds to decision block 710 to determine whether the verifier layer is the last verifier layer in the hierarchy. If the answer is no, the runtime setting evaluation routine 700 continues to block 712 where the next verifier layer is selected. The runtime setting evaluation routine 700 repeats the previously discussed processing steps. If the answer is yes, the runtime setting evaluation routine 700 terminates at block 714.
While illustrative embodiments have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.
