Fuzzing refers to a process of altering the data of test cases so as to generate new test cases. Test data may be fuzzed to test a variety of different types of software. Typically, the fuzzing process is automated, guided by alterations programmed by a human tester.
Existing fuzzing techniques may not collect the data that results from the fuzzing operations in one location. Thus, it may be difficult to correlate the results from a variety of different test runs, and determine which set of test conditions caused a particular error to occur. In the context of testing an application in a client-server environment, a server application may be tested by submitting a large number of test requests to it. If one of these numerous requests crashes the server or causes the server to fail, it may be difficult to isolate which particular request, sequence of requests, or other circumstances led to the failure.
Typical fuzzing techniques involve testing, for example, a server application by creating an “artificial” client application that sends fuzzed requests to the server application under test. Therefore, the testers typically would possess detailed knowledge regarding the format and content of the services offered by the server under test, and would implement the artificial client application accordingly. However, this detailed knowledge may be expensive to acquire, and may further limit how widely a given fuzzing test application may be deployed. If another server application is to be tested, the artificial client application may need to be recreated, at least in part.
Systems, methods, and/or techniques (“tools”) for fuzzing requests and responses using a proxy are described herein. A system for fuzzing requests and responses using a proxy interface includes a client that, in turn, includes a client application. The system can also include a server that includes a server application. The system also includes a proxy coupled between the client and the server. The proxy communicates message traffic between the client and the server, and the message traffic may be related to testing the client application or the server application. The proxy stores a template resulting from the message traffic into a data store to facilitate later fuzzing of requests or responses that are contained in the message traffic. A user interface for presenting events resulting from the fuzzing is also described.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Tools for fuzzing requests and responses using a proxy are described in connection with the following drawing figures. The same numbers are used throughout the disclosure and figures to reference like components and features. The first digit in a reference number indicates the drawing figure in which that reference number is introduced.
Overview
The following document describes tools capable of performing and/or supporting many techniques and processes. The following discussion describes exemplary ways in which the tools provide for fuzzing requests and responses using a proxy. This discussion also describes other techniques and/or processes that may be performed by the tools.
For convenience only, but not limitation, this document is organized into sections, with the sections introduced by corresponding headings. First, Operating Environments are described in connection with
Operating Environments
In general, the operating environment 100 may include one or more clients 102. The client 102 may include one or more processor(s) 108 and computer-readable media 110. The computer-readable media 110 may contain instructions that, when executed by the processor 108, perform any of the tools described herein. The processor 108 may be configured to access and/or execute the instructions embedded or encoded onto the computer-readable media 110. The processor 108 may also be categorized or characterized as having a given architecture. The client 102 may comprise a computing device, such as a network or other server, a desktop computer, a laptop or notebook computer, or any other computing device configured to perform the functions described herein in connection with the client 102.
The computer-readable media 110 may include one or more client applications 112;
The proxy 104 may also include a processor and computer-readable media, referenced at 114 and 116, respectively. The computer-readable media 116 may include a proxy interface 118 that is adapted to receive requests from the client 102, and to pass responses to the client 102. For convenience only, the requests and responses passing between the client 102 and the proxy 104 are represented generally as message traffic 120a.
The proxy interface 118 may also be adapted to pass requests to the server 106, and to receive responses from the server 106. For convenience only, the requests and responses passing between the proxy 104 and the server 106 are represented generally as message traffic 120b.
The server 106 may include a processor and computer-readable media, referenced at 122 and 124, respectively. The computer-readable media 124 may include one or more server applications 126;
It is noted that the client 102, the proxy 104, and the server 106 are shown as separate entities in
Additionally, implementations of the operating environment 100, as well as other operating environments described herein, may include more than one proxy 104. For example, the various functions described herein as being performed by the proxy 104 may be distributed across more than one proxy 104 without departing from the scope and spirit of the description herein.
The proxy 104 may include a proxy logging component 128 that is operative to receive respective sets of requests and corresponding responses that pass between the client 102 and the server 106, through the proxy interface 118. Recall that
As discussed in further detail below, the data store 132 may store the results of submitting one or more requests to the server 106, and receiving responses thereto. The data store 132 may also store the results of fuzzing one or more requests submitted to the server 106, or of fuzzing one or more responses provided to the client 102. Without loss of generality, term “fuzzing” refers to altering or transforming at least part of the data used to test a computer program, so as to generate a new test case.
The templates 130 that result from the message traffic 120 may be input to the proxy 104 as potential test cases for testing the components of the client 102 and/or the server 106. More particularly, the client application 112 and/or the server application 126 may be tested, as described further below in connection with
The client 102 may obtain one or more templates 130 from the data store 132. The templates 130 represent ordinary, expected message traffic 120 that would pass between the client 102 and the server 106. The message traffic 120 may take the form of messages exchanged under, for example, the HTTP, SOAP, or other protocols. The client 102 may include a traffic replay component 202 that is adapted to receive the templates 130 from the data store 132. The traffic replay component 202 may submit one or more requests 204 to the proxy 104, for example, the proxy interface 118.
If fuzzing is enabled, the proxy 104 evaluates the request 204 to determine whether it meets any criteria applicable to fuzzing requests. If the request 204 meets the criteria, the proxy 104 fuzzes the request 204 and passes it to the server 106. The request 204 as fuzzed and passed to the server 106 is denoted as fuzzed request 206. The proxy interface 118 may log the non-fuzzed request 204 and the fuzzed request 206 in the data store 132.
The server 106 receives the fuzzed request 206. At the server 106, the fuzzed request 206 is processed by, for example, the server application 126. If the server application 126 processes the fuzzed request 206 successfully, the server 106 may return a response 208 to the proxy 104. In some instances, the proxy interface 118 may log the response 208 in the data store 132, to be associated with the non-fuzzed request 204 and the fuzzed request 206 stored previously. For convenience of illustration only, the non-fuzzed request 204, the fuzzed request 206, and/or the response 208 as stored in the data store 132 are represented in
Returning to the server 106, the server 106 may include a server status reporting component 214 and a server monitoring component 216. At least these components 214 and 216 may be implemented in the computer-readable media 124. However,
The server status component 214 may take the form of, for example, an application debugger, a memory profiling tool, or the like. Respective instances of the server status component 214 may be associated with corresponding instances of the server application 126. For example, if the server application 126 spawns several different executing threads or processes, instances of the server status component 214 may be associated with these different threads or processes.
The server status component 214 may be configured to detect occurrences of pre-defined events 218, such as memory faults, access violations, buffer overflows, allocations or deallocations of memory, or the like. The events 218 may be reported in connection with details relating to the events 218, such as call stacks, contents of debug registers, exception types, or the like.
If the server 106 and/or the server application 126 generates an event 218, the server status component 214 captures these events 218, and reports them to the server monitoring component 216. Within a given server 106, the server monitoring component 216 may receive reports of events 218 from one or more server status components 214. For convenience, the events 218 as reported to the server monitoring component 216 are denoted at reference 220. The events 220 may be considered to represent all events of interest that occur in any process executing on the server 106.
The server monitoring component 216 may notify the proxy 104 of any events 220 reported by the server status component 214. The proxy 104 may include a proxy debugging component 222 that is adapted to receive notifications of the events 220 as they occur on the server 106. For convenience, the events 220 as reported to the proxy 104 are denoted at reference 224. The events 224 may be considered to represent all events of interest that occur in any process executing on the server 106, as reported to the proxy 104. These events 224 may also include cases where processes running on the server 106 stop executing without warning. Additionally, the server monitor 216 may report events that are not specific to a particular process, such as overall CPU usage, non-paged memory usage, free disk space, or the like. For example, such events may be reported when predefined thresholds have been met or exceeded.
The proxy 104 may also include a proxy logging component 226 that is operative to receive notifications of events 224 and related details, which are denoted generally in
In the foregoing manner, the traffic replay component 202 emulates the client application 112, and enables testing of the server application 126. In some instances, the server application 126 may be tested without the client application 112 running on the client 102.
The foregoing discussion of
To facilitate testing the client 102 and/or the client application 112, the server 106 may include a traffic replay component 302, which may be implemented in the computer-readable media 124 shown in
The traffic replay component 302 may be implemented similarly to the traffic replay component 202 shown in
The traffic replay component 302 may analyze the templates 130 to extract the responses received to certain requests. For convenience, the responses as passed from the server 106 to the proxy 104 are denoted as responses 304a. The responses 304a are provided to the proxy interface 118. One or more criteria may specify which responses 304a are eligible for fuzzing and presentation to the client 102 as test cases. The proxy interface 118 evaluates whether the responses 304a meet these eligibility criteria. If so, the proxy interface 118 fuzzes the responses 304a and presents them to the client 102. For convenience, the fuzzed responses are denoted in
The client 102 may receive and process the fuzzed responses 304b, for example, using the client application 112. For example, the client application 112 may be a web browser application, and the fuzzed responses 304b may be web pages requested by the web browser application that are fuzzed by the proxy 104. The client application 112 then processes the fuzzed responses 304b.
The client 102 may include a client status reporting component 306, which may be implemented similarly to the server status reporting component 214 shown in
If the client application 112 generates one or more events 308 of interest to the proxy 104 when processing the fuzzed response 304b, the client status reporting component 306 may capture the events 308. Examples of the events 308 may include memory faults, access violations, buffer overflows, allocations or deallocations of memory, or the like. The events 308 may be reported along with related details, such as call stacks, contents of debug registers, exception types, or the like.
One or more of the client status reporting components 306 may report events 308 to a client monitoring component 310. The client monitoring component 310 may be implemented to function similarly to the server monitoring component 216 shown in
The client monitoring component 310 may be adapted to report the events 312 that are captured on the client 102 to the proxy 104. For example, the client monitoring component 310 may report events 314 to the proxy debugging component 222, which was discussed above with
The proxy logging component 226 may be adapted to log the events 316 and any related details (e.g., call stacks, contents of debug registers, exception types, or the like) in the data store 132. For convenience, the events 316 and related details as stored in the data store 132 are denoted at reference 318. In the data store 132, the events 316 and related details can be correlated with the non-fuzzed responses 304a and the fuzzed responses 304b that triggered the events 316. For convenience, the non-fuzzed responses 304a and the fuzzed responses 304b as stored in the data store 132 are referenced at 320. As discussed above with the events 230 shown in
In the foregoing manner, the traffic replay component 302 emulates the server application 126, and enables testing of the client application 112. In some instances, the client application 112 may be tested without the server application 126 running on the server 106.
Having described the above operating environments, the discussion now turns to a description of illustrative process flows for performing various aspects of fuzzing requests and responses using a proxy.
Process Flows
Block 402 installs a legitimate client application, such as a web browser, SOAP client, test application, compatible HTTP client application, or the like. An example client application is shown in
Block 404 configuring the client application to use a proxy, such as the proxy 104 shown in
Block 406 starts a target application running on a server. For example, the
Block 408 configures the proxy to log the message traffic. The message traffic may include requests from the client application to the server application, and responses thereto from the server application.
Block 410 exercises the functionality of the server of interest, by causing the client to send appropriate requests through the proxy to the server. An example of message traffic from the proxy to the server is shown at 120b in
Block 412 receives the responses from the server. Block 414 saves a log of the requests and corresponding responses into a data store for use as a template for the fuzzed requests and/or responses.
Having described a process for creating the template, the discussion now turns to a description of configuring a client, proxy, and/or server for replaying message traffic and fuzzing.
Block 502 starts a status reporting component on the client or server under test. For example, the status reporting component may take the form of an application debugger. Examples of the status reporting component are shown in
Block 504 starts a monitoring component on the client or server under test. Examples of the monitoring component are shown in
Block 506 configures and enables fuzzing in the proxy. Block 508 configures and enables debugging or status monitoring options on the proxy. Examples of such options include which processes to watch, what commands to run to restart them, or the like. Note that in one implementation, processes can be monitored by process id, to select a process individually, or by name, which allows automatically attaching to new processes created with the same name. This feature may be especially suitable for monitoring servers that create new processes under load or restart processes. Block 510 starts the proxy.
Once the proxy starts, block 602 initiates communication with the status reporting and monitoring components on the server or client under test. Block 604 initiates the status reporting components (e.g., application debuggers) and attaches them to the processes of interest running on the server or client under test.
Block 606 starts a traffic replay utility on the server or client under test.
Block 608 looks up information in the proxy log or template, and selects requests or responses to replay. More particularly, if a server application is being tested, block 608 may include selecting requests from the template to be replayed, such that the requests are re-presented to the server after being fuzzed. If a client application is being tested, block 608 may include selecting responses from the template to be replayed, such that the responses are re-presented to the client after being fuzzed.
Block 610 automatically replays the requests and/or responses selected or obtained in block 608. In but one possible implementation, block 610 may be performed by the traffic replay utility (e.g., 202 in
When requests or responses arrive at the proxy, block 612 evaluates them to determine whether they qualify to be fuzzed, based on criteria that may be set by a user of the tools, for example a network-aware software tester (hereafter “tester”). If the request or response qualifies to be fuzzed, then the process flow takes Yes branch 614 to block 616 to fuzz the request or response. Block 616 may include calling a fuzzing add-in API, which may determine which part of the request or response to fuzz, as configured by the tester. The API may also specify what fuzzing strategy to use. For example, the tester can specify whether to extend the information present in the request or response, replace such information, or the like. The API may also tell the proxy what change to make, and the change is made automatically.
Block 618 stores the details of how a particular request or response was fuzzed in a data store that contains combined results data. An example of such a data store is the data store 132. In some implementations, the combined results data is interchangeable or compatible with the proxy log or templates, to facilitate replay scenarios more readily.
Block 620 sends the fuzzed request to the server, or sends the fuzzed response to the client, depending on whether a server application or a client application is being tested.
Returning to block 612, if the request or response does not meet the criteria applicable to fuzzing, then the process flow 600 takes No branch 622 to block 620. In this case, blocks 616 and 618 are bypassed, and the request or response is not fuzzed. Instead, the request or response is sent as-is to the server or client.
The processing of the fuzzed request or response at the server or client is described in connection with
Block 702 evaluates whether the fuzzed request or response was handled successfully by the server or client under test. If so, the process flow 700 takes Yes branch 704 to block 706, which reports a status back to the proxy.
Block 708 then logs the status, indicating that the fuzzed request or response was handled successfully. Block 708 may be performed by the proxy. For example, the status can be logged into a data store such as the data store 132.
Block 710 forwards the response or request to the client or server, respectively. For example, the proxy may be testing a server application by sending it a fuzzed request. If the server application successfully processes the fuzzed request and provides a valid response back to the proxy, then block 710 includes forwarding the valid response to the client. Similarly, the proxy may be testing a client application by returning it a fuzzed response. If the client application successfully processes the fuzzed response, then block 710 includes notifying the server that the response was processed successfully.
After performing blocks 706-710, the process may be repeated for the request or response extracted from the template. Thus, a process flow may return to block 610 for this next request or response, as represented in block 712.
Returning to block 702, if the request or response is not handled successfully by the client or server application under test, then the process flow 700 takes No branch 714 to block 716. Block 716 communicates the details of the error (e.g., call stack, debug registers, type of exception, or the like) to a monitor component executing on the client or server under test. Examples of the monitor components are the server monitoring component 216 shown in
Block 718 assesses the severity of the error, and may be performed by, for example, the monitor component executing on the client or server under test. If the error is sufficiently severe, block 720 communicates the event back to the proxy debug components. In but one possible implementation, the event can be communicated via a separate means from the network message traffic being targeted. The targeted network message traffic is represented by 120 in
Block 722 adds the event details to the combined results data stored in the data store 132, associating the event details with the non-fuzzed and fuzzed request or response that triggered the event. Block 722 may be performed by the proxy logging component 226.
Block 724 evaluates the type of event that occurred, and determines whether to suspend test message traffic. For example, the server or client application under test may have crashed when processing the fuzzed request or response, rendering further testing pointless until the server or client application is restarted. If test message traffic is not to be suspended, the process flow 700 takes No branch 726 to block 712, which was described above.
Returning to block 724, if test message traffic is to be suspended, then the process flow 700 takes Yes branch 728 to block 730, where the client or server application under test is restarted. For example, the status reporting component (e.g., 216 and 310 in
In some implementations, when the client or server application under test has been restarted and is ready to accept more test cases, block 732 may notify the proxy accordingly. The proxy, upon receiving a message that the process has restarted, can optionally delay accepting new client requests for a configurable amount of time, allowing time for the process restart to complete. For example, block 732 may be performed using the same means as block 720 uses as described above, such as a customized TCP protocol. At this point, testing may resume, either with the current test case or the next test case. Alternatively, testing may be suspending pending further investigation.
The process flows 500, 600, and 700 may be repeated or continued for as long as the tester configured, until the tester intervenes, or a more severe event prevents operation of the process flows 500, 600, and 700, such as the server or client running out of disk space.
For convenience only, the process flows 500, 600, and 700 are described above in connection with certain components shown in the operating environments 100, 200, and 300. However, it is noted that the process flows 500, 600, and 700 may be performed, in whole or in part, using components other than those described herein without departing from the scope and spirit of the description herein.
Having described the above process flows, the discussion now turns to an illustrative user interface that may present results of the testing described above.
User Interface
Turning to the user interface 800 in more detail, an area 802 may contain representations of various events that occurred during the testing runs described above. The area 802 may be populated by extracting event data from a data store, such as the data store 132. Note that the area 802 can indicate whether particular events are requests or responses, or whether the request or response was changed or unchanged (i.e., fizzed or non-fuzzed).
The area 802 may be responsive to user input to select one or more of the events. Examples of user selection input may include mouse and/or keyboard input. More detailed data related to these selected events may be presented in an area such as the area 804. As shown in
The user interface 800 may also include a series of tools that enable the tester to adjust the data presented in the user interface 800. For example, checkboxes 808 enable the tester to select whether or not to display date and time data in the area 802. Custom filtering tool 810 enables the tester to specify strings that may be used to either include or exclude events from being displayed in the area 802. Search/highlight tool 812 enables the tester to locate particular events, and to select a next or previous event.
Checkboxes 814 enable the tester to select event types to display. Individual ones of the checkboxes 814 either enable or disable display of requests, responses, pre-edited (fuzzed) events, post-edited (fuzzed) events, server monitor 9 events, or events corresponding to known issues.
Export tool 816 enables the tester to export selected items, for example, in binary format or XML format. Known issue tool 818 enables the tester to edit descriptions of known events, or to add additional known events. Replay tool 820 enables the tester to replay one or more items that are selected in the area 802 against one or more specified servers or clients.
Area 822 presents an itemized summary of debug events as reported in the area 802. Button 824 enables the tester to refresh the view of events shown in the area 802. Button 826 enables the tester to delete the contents of the data store from which the user interface 800 reads, such as the data store 132.
The above illustration and description of the user interface 800 is presented only for convenience, but not limitation. Thus, it is noted that implementations of the user interface 800 could differ visually from that shown in
Although the system and method has been described in language specific to structural features and/or methodological acts, it is to be understood that the system and method defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed system and method.
Certain data structures are described herein in block form. It is understood that the block elements shown herein are shown and labeled only for convenience. Implementations of these data structures could include sub-sets or super-sets of the elements described and illustrated herein, and could include elements labeled or referenced differently than the elements herein, without departing from the spirit and scope of the description herein
In addition, regarding certain flow diagrams described and illustrated herein, it is noted that the processes and sub-processes depicted therein may be performed in orders other than those illustrated without departing from the spirit and scope of the description herein.