Patent Application
20150067031
The World Wide Web has expanded to provide web services faster to consumers. Web services may be provided by a web application which uses one or more services to handle a transaction. The applications may be distributed over several machines, making the topology of the machines that provides the service more difficult to track and monitor.
Monitoring a web application helps to provide insight regarding bottle necks in communication, communication failures and other information regarding performance of the services that provide the web application. A web application may be initiated by a request received from a client. To provide a complete picture of the performance of a web application, it is helpful to monitor both the backend processing of a request by the web application as well as the network and other processing of the request.
Typically, monitoring systems determine the time it takes to process a request on a back-end system and the total time to handle a request from the point of view of a client which initiates the request. The difference of the two times is typically attributed to network latency. Attributing time difference to the network does not provide a complete explanation of how the processing of a request is distributed in time.
There is a need in the art for web service monitoring which may accurately and efficiently monitor the performance of distributed applications from the point of view of a backend server and a client device sending a request to the server, as well as a need for correlating data derived from monitoring each end of a transaction.
The present technology monitors a client request received through a rendered content page and a backend business transaction performed in response to the request and correlates the data generated from monitoring the request and business transaction. A request may be monitored in detail to determine several metrics. The metrics may include server connection time, response receipt completion time, document download time, document processing time, and page rendering time. The metrics may be determined for an entire content page as well as individual page elements, such as Iframes, frames, and elements constructed using AJAX techniques.
When a request is received through a content page rendered by a network browser, a corresponding request may be sent to a backend server, which results in performance of a business transaction. The monitoring data for the business transaction and metrics obtained from monitoring the processing of the request at the client may be correlated using a unique identifier. The unique identifier may created by an agent on the backend server or by an agent in the browser application. A controller may access business transaction monitoring data and request to correlate the data and report it to a user.
In an embodiment, a method for reporting data may receive, from a client, a client request identifier and data associated with processing the client request at the client device. The client request identifier and data associated with processing a business transaction associated with the client request may be received from one or more servers. Data associated with processing the client request at the client device and the data associated with processing a business transaction at the one or more servers may be reported.
In an embodiment, a method for reporting data may send a request to a remote server. A unique request identifier associated with the request may be received from the server, wherein the unique request identifier created by the server in response to receiving the request. Performance data associated with processing the request by the network browser may be captured. The performance data and the unique request identifier associated with the request may be reported.
In an embodiment, a method for reporting data may receive a request from a client, the request associated with one of a plurality of content page elements requesting data from one or more remote servers. A unique request identifier for the received request may be generated, and the unique request identifier may be transmitted to the requesting client.
In an embodiment, a method for reporting data may receive input associated with a content page through a network browser on a client device. A first time at which the client device connects with a server that creates a response having content for a content page may be detected, wherein the response created in response to a request sent based on the received input. A second time at which the response completed at the client device may be detected.
In embodiments, the methods described above may be performed by systems. Additionally, one or more modules stored on memory may be executable by one or more processors to perform the above methods.
Embodiments of the present invention monitor a client request and a backend business transaction performed in response to the request. Data generated from monitoring the request and the business transaction may be correlated and reported together. A request may be monitored in detail to determine metrics such as for example sever connection time, response receipt completion time, document download time, document processing time, and page rendering time. The metrics may be determined for an entire content page as well as individual page elements, such as Iframes and elements constructed from AJAX techniques.
A request for content may be sent to a backend server. The request may be generated by a user, for example when a request is received through a content page (e.g., selecting a link) rendered by a network browser or the browser itself (e.g., selecting a bookmark), or may be automatically generated (e.g., an automated update to the page). The request may be generated by a network browser or other client software or hardware that loads content provided by a remote server. The request to the backend sever may result in performance of a business transaction. The monitoring data for the business transaction and metrics obtained from monitoring the processing of the request at the client may be correlated using a unique identifier. The unique identifier may created by an agent on the backend server or by an agent in the browser application. A controller may access business transaction monitoring data and request to correlate the data and report it to a user.
An agent may be provided to monitor a request. The agent may be embedded in the HTML being downloaded to the client, inserted into a content page, or otherwise provided on the client to monitor the request. In some embodiments, a server agent may embed an agent in the HTML code provided to a client. The browser application agent may retrieve data and timing information upon completion of certain tasks or events occurring at the browser application. Agents installed on one more servers may monitor performance of an application. Agents at both the browser application and the server(s) may report data to a controller. The controller may receive the data, correlate data from the browser application and servers together, and report correlated data.
Client device 105 may include network browser 110 and be implemented as a computing device, such as for example a laptop, desktop, workstation, or some other computing device. Network browser 110 may be a client application for viewing content provided by an application server, such as application server 130 via network server 125 over network 120.
Network browser 110 may include agent 112. Agent 112 may be embedded, installed or otherwise provided on network browser 110 and/or client 105, for example as a network browser add-on, downloading the agent to the network browser as part of HTML, or in some other manner. Agent 112 may be executed to monitor network browser 110, the operation system of client 105, and any other application, API, or other component of client 105. Agent 112 may determine network browser navigation timing metrics, access browser cookies, and transmit data to data collection 160, controller 190, or another device. Agent 112 may perform other operations related to monitoring a request at client 105 as discussed herein.
Mobile device 115 is connected to network 120 and may be implemented as a portable device suitable for sending and receiving content over a network, such as for example a mobile phone, smart phone, tablet computer, or other portable device. Both client device 105 and mobile device 115 may include hardware and/or software configured to access a web service provided by network server 125.
Mobile device 115 may include network browser 117 and an agent 119. Agent 119 may reside in and/or communicate with network browser 117, as well as communicate with other applications, an operating system, APIs and other hardware and software on mobile device 115. Agent 119 may have similar functionality as that described herein for agent 112 on client 105, and may repot data to data collection server 160 and/or controller 190.
Network 120 may facilitate communication of data between different servers, devices and machines. The network may be implemented as a private network, public network, intranet, the Internet, a cellular network, Wi-Fi network, VoIP network, or a combination of one or more of these networks.
Network server 125 is connected to network 120 and may receive and process requests received over network 120. Network server 125 may be implemented as one or more servers implementing a network service. When network 120 is the Internet, network server 125 may be implemented as a web server. In some embodiments, network server 125 and application server 130 may be a single server, or include multiple machines that each implement a network server and an application server.
Application server 130 may communicate with network server 125, application servers 140 and 150, controller 190. Application server 130 may also communicate with other machines and devices (not illustrated in
Virtual machine 132 may be implemented by code running on one or more application servers. The code may implement computer programs, modules and data structures to implement a virtual machine mode for executing programs and applications. In some embodiments, more than one virtual machine 132 may execute on an application server 130. A virtual machine may be implemented as a Java Virtual Machine (JVM). Virtual machine 132 may perform all or a portion of a business transaction performed by application servers comprising system 100. A virtual machine may be considered one of several services that implement a web service.
In embodiments, applications may execute on servers in program containers other than a virtual machine. For example, applications may be executed in PHP on any of servers 130-160.
Virtual machine 132 may be instrumented using byte code insertion, or byte code instrumentation, to modify the object code of the virtual machine. The instrumented object code may include code used to detect calls received by virtual machine 132, calls sent by virtual machine 132, and communicate with agent 134 during execution of an application on virtual machine 132.
Agent 134 on application server 130 may be installed on application server 130 by instrumentation of object code, downloading the agent to the server, or in some other manner. Agent 134 may be executed to monitor application server 130, monitor virtual machine 132, and communicate with byte instrumented code on application server 130, virtual machine 132 or another application on application server 130. Agent 134 may detect operations such as receiving calls and sending requests by application server 130 and virtual machine 132. Agent 134 may receive data from instrumented code of the virtual machine 132, process the data and transmit the data to controller 190. Agent 134 may perform other operations related to monitoring virtual machine 132 and application server 130 as discussed herein. For example, agent 134 may identify other applications, share business transaction data, aggregate detected runtime data, and other operations.
Agent 134 may create a request identifier for a request received by server 130. The request identifier may be sent to client 105 or mobile device 115, whichever device sent the request. In embodiments, the request identifier may be created when a data is collected and analyzed for a particular business transaction. Additional information regarding collecting data for analysis is discussed in U.S. patent application no. U.S. patent application Ser. No. 12/878,919, titled “Monitoring Distributed Web Application Transactions,” filed on Sep. 9, 2010, U.S. patent application Ser. No. 13/189,360, titled “Automatic Capture of Diagnostic Data Based on Transaction Behavior Learning,” filed on Jul. 22, 2011, and U.S. patent application Ser. No. 13/365,171, titled “Automatic Capture of Detailed Analysis Information for Web Application Outliers with Very Low Overhead,” filed on Feb. 2, 2012, the disclosures of which are incorporated herein by reference.
Each of application servers 140 and 150 may include an application and an agent. Each application may run on the corresponding application server or a virtual machine. Each of virtual machines 142 and 152 on application servers 140-150 may operate similarly to virtual machine 132. Agents 144 and 154 may operate similarly to agent 134.
Asynchronous network machine 170 may engage in asynchronous communications with one or more application servers, such as application server 150 and 160. For example, application server 150 may transmit several calls or messages to an asynchronous network machine. Rather than communicate back to application server 150, the asynchronous network machine may process the messages and eventually provide a response, such as a processed message, to application server 160. Because there is no return message from the asynchronous network machine to application server 150, the communications between them are asynchronous.
Data stores 180 and 185 may each be accessed by a machine, such as for example an application servers such as application server 150 or client 110 Data store 185 may also be accessed by application server 150. Each of data stores 180 and 185 may store data, process data, and return queries received from an application server. Each of data stores 180 and 185 may or may not include an agent.
Controller 190 may control and manage monitoring of business transactions distributed over application servers 130-160. In some embodiments, controller 190 may receive runtime data, including data associated with monitoring client requests at client 105 and mobile device 115, from data collection server 160. In some embodiments, controller 190 may receive runtime data from each of agents 112, 119, 134, 144 and 154. Controller 190 may associate portions of business transaction data with other portions of business transaction data and virtual machines, applications, and other nodes and hardware that the business transaction data is generated from monitoring, communicate with agents to configure collection of runtime data, and provide performance data and reporting through an interface. Performance data may include metrics, errors, and other data and events which may be captured and/or generated during the monitoring of a distributed transaction. The interface may be viewed as a web-based interface viewable by client device 192, which may be a mobile device, client device, or any other platform for viewing an interface provided by controller 190. In some embodiments, a client device 192 may directly communicate with controller 190 to view an interface for monitoring data.
Controller 190 may also correlate request metrics with business transaction data received from client 105 and mobile device 115 and from application servers 130-150, respectively.
Data collection server 195 may communicate with client 105, 115 (not shown in
A content page request may be generated at step 315. The content page request may be received through a content page rendered by a browser application, through a selection of the browser application itself, or generated based on an event not initiated by a user. For example, the request may be generated from received input selecting a link, button, or media for display, submission of text within a text box, selection of a bookmark, an automated update, or some other event. The generated content page request may be associated with a particular content page element, such as an Iframe, frame, or an element generated using AJAX techniques, or the content page in general.
Steps 320-350 may be initiated or performed on a browser application and may be monitored by an agent within the application or the device hosting the application. Steps 320-350 may be performed for each content page element of a content page rendered by network browser 110, such as for example each Iframe, frame, element generated using AJAX techniques, and so on. Though steps 320-350 are discussed below with respect to agent 112 on network browser 110, the steps may be performed by agent 119 on mobile device browser 117 and other client devices.
The content page input may be processed to connect with a server that may process the request at step 320. Processing the request may include collecting data to determine a server connection time—e.g., the time it takes client 110 to establish a connection with server 130 which can process the request. Put another way, the server connection time may include the time it takes the browser to resolve and start fetching the document from a server or application cache, including connection re-directs, domain server look-up operations, TCP communications and SSL communications. The time to determine server connection time is the time at which the connection with the server is complete, but before any response data is transmitted to the server from the client. In embodiments, the present technology may utilize W3C navigation timing metrics to determine redirects, DNS, and TCP. An agent within the network browser that originated the request may store the start time and end time associated with connecting to the server.
Names for the base content page and content page elements are stored at step 325. The names are used to identify content page element hierarchy. The page name may be created based on any of several factors, including a URI of the page, Iframe or AJAX request, the domain name of the request (e.g., www.costco.com), the protocol used for the request (http), a user defined page name, or other source of information. Names may be provided upon page rendering, upon receiving a request from a particular content page element, or at some other time. In some embodiments, a base content page and/or other content page element names may be determined by a collector that receives data from an agent monitoring performance at a client.
Transmittal of the client request may begin at step 330. Agent 112 may detect the initial transmission of the client request, which may occur as soon as the server connection is completed. In embodiments, the end of the client request transmission may or may not be tracked. The time at which the content page is first transmitted may be stored by agent 112 or by a network browser which makes it available to the agent.
The start of a response received by the server by client 105 is detected at step 335. The response is transmitted by server 130 in response to the request sent at step 330. Receipt of the response may extend over a period of time, and may overlap with a portion of the request sent to the server providing the response. The time at which the response is first received may be stored by agent 112 or by a network browser which makes it available to the agent.
A request identifier associated with the request at step 330 may be stored at step 340. The request identifier may be generated by a server processing the request and sent to the client 105. For example, agent 134 on application server 130 may detect receipt of the request, generate a request identifier, and transmit the request identifier client 105. In some instances, a client which sends the original request may generate a request identifier. Storing the request at step 340 is discussed in more detail with respect to the method of
The receipt of the response is completed at client 105 at step 345. After receiving the response, which may be received incrementally, the document contained in the response is loaded at step 345. While loading the document (which may be done incrementally), the start time of the document load and the end time of the document may be stored by agent 112 or may be made available to the agent by a network browser. As it is loading, the document may be available for Java scripts to apply rendering logic.
The content page may be rendered, for example incrementally as it is received, at step 350. Rendering the content page may include downloading and displaying remaining resources such as image and other rendering tasks. Agent 112 may store the start time and end time of rendering the content page or these may be available to the agent by a network browser.
Data may be sent by agent 112 to data collection 160 and ultimately to controller 190 at step 355. A data collection application may be implemented remotely on server 160 (discussed herein for purposes of example) or, in some embodiments, at the client device The data collection application may aggregate received data and transmit data to controller 190. The data sent to data collection 160 by agent 112 may include runtime data associated with server connect time, response available time, document download and processing time, and page rendering time. The data sent to data collection 160 may also include request identification information for the data associated with the request, a page name to identify the source of the request, the name of the parent page to the requesting page, and the base page name for the requesting page. The data may be transmitted periodically, in response to a request for data from the controller, in response to the completion of a client request, or based on some other event. The data may be aggregated and transmitted to controller 190, where it is received and processed. Operation of controller 190 is discussed below with respect to
If the request was a client request, the request identifier may be provided in one or more of several ways, including a cookie or response body, for example in embedded Java script code. Agent 112 may retrieve the request identifier from a cookie at step 420.
In addition to client request ID, a cookie may also include (as written by a server processing the request) one or more tuples of one or more business transaction identifiers and time for each of one or more responses to be processed at the server. The tuple may be written to the cookie as well as a body, footer or other portion of the response.
Agent 430 may retrieve the request identifier from a content page response body at step 430. In some embodiments, the request identifier may be in one of a cookie or a response body, based on the whether the response is provided or whether the cookie has been sent, but may not be in both. As such, both a cookie and response body may be searched to retrieve the request identifier. The request identifier is then stored at step 450.
A request identifier may be created for the detected request at step 525. The request identifier may be a global unique identifier that identifies the particular request. The request identifier may be transmitted to the requesting client in several ways, depending in part on the type of request that was received. If the request received was a client request (determined at step 530), the request identifier may be sent to the client in a response body and/or in a cookie at step 535. When sent in a cookie, the cookie may be sent at some point between receiving the initial request data and completing transmission of the response to the request. The cookie may be sent soon after receiving the request and at some point before the HTTP header is flushed. When sent in a request body, the request identifier may be included in the body footer, header, or other portions in the body or request. The method of
Business transaction data may be collected and aggregated by agent 134 at step 545. The business transaction data and the request identifier may be sent from the server agent to the controller at step 550. The request identifier may be associated with a particular “snap shot” of data, a collection data for a particular business transaction.
In embodiments, the relationship between a page element, such as a content page request or AJAX request, and a business transaction may be a many to many relationship. Hence, a page may include none, one or more Iframes and may be associated with zero or more business transactions, and a business transaction may be associated with multiple pages.
A request identifier and business transaction data may be received from one or more servers at step 620. The business transaction data may include data for creating a business transaction snapshot. The client request data and business transaction data may be correlated using request identifiers. Client request data associated with a particular request identifier may be correlated with business transaction data associated with a matching request identifier.
Client request data may be correlated with business transaction data to re-assemble a complete picture of a base page. The data may be correlated together using information about the iframes, requests using AJAX techniques, the content page and other elements make up the base page and send network requests. When client request data is ultimately transmitted from a client agent to the controller, the client request data may include an identifier for a particular request (content page, iframe, AJAX generated, or other request) as well as the parent page identifier and the content page base identifier. As client request data is received by the controller for each content page element or AJAX based request, a hierarchy of the requesting elements for a particular content page can be constructed using the request identifiers and parent page identifiers for the particular base content page. Once the hierarchy is constructed, the performance of each content page request can be related to other content page requests in the hierarchy, and the overall picture of performance for providing the base page can be constructed.
Additionally, the performance of each request of a content page may be correlated to one or more business transactions associated with each request associated the content page's hierarchy. For example, each iframe, AJAX generated request, and other page element may be correlated to a business transaction performed by a distributed server system, with the performance on each side of the network correlated together. The correlated information for the content page request and business transaction processing may be provided through a series of correlated reports, such as for example a transaction flow map (see
In addition to correlating performance data, the present system may detect HTTP or Javascript errors and correlate them between a client content page request and a distributed transaction. The HTTP errors and Javascript errors may be detected at the client. The particular error may be detected for a particular request made from a content page element, such as for example a particular iframe, AJAX generated request, or other request. On the server side, any processing performed for the request may be correlated to the request for which the error was detected. As a result, rather than reporting that an entire content page is associated with an error, the particular request may be associated with the particular HTTP or Javascript error and the processing of the request associated with an error may be reported for the particular request. This allows an administrator to a much higher level of visibility and granularity with respect to what caused the error and the details of when the error occurred.
The correlated content page data and business transaction data may be reported at step 640. The data may be reported via email, SMS, or graphically through an interface. Examples of graphical reporting are provided in
The interface of
The summary tab is selected in
The components shown in
Mass storage device 930, which may be implemented with a magnetic disk drive or an optical disk drive, is a non-volatile storage device for storing data and instructions for use by processor unit 910. Mass storage device 930 can store the system software for implementing embodiments of the present invention for purposes of loading that software into main memory 920.
Portable storage device 940 operates in conjunction with a portable non-volatile storage medium, such as a floppy disk, compact disk or Digital video disc, to input and output data and code to and from the computer system 900 of
Input devices 960 provide a portion of a user interface. Input devices 960 may include an alpha-numeric keypad, such as a keyboard, for inputting alpha-numeric and other information, or a pointing device, such as a mouse, a trackball, stylus, or cursor direction keys. Additionally, the system 900 as shown in
Display system 970 may include a liquid crystal display (LCD) or other suitable display device. Display system 970 receives textual and graphical information, and processes the information for output to the display device.
Peripherals 980 may include any type of computer support device to add additional functionality to the computer system. For example, peripheral device(s) 980 may include a modem or a router.
The components contained in the computer system 900 of
The components shown in
Memory 1012 may include local memory such as RAM and ROM, portable memory in the form of an insertable memory card or other attachment (e.g., via universal serial bus), a magnetic disk drive or an optical disk drive, a form of FLASH or PROM memory, or other electronic storage medium. Memory 1012 can store the system software for implementing embodiments of the present invention for purposes of loading that software into main memory 1010.
Antenna 1016 may include one or more antennas for communicating wirelessly with another device. Antenna 1016 may be used, for example, to communicate wirelessly via Wi-Fi, Bluetooth, with a cellular network, or with other wireless protocols and systems. The one or more antennas may be controlled by a processor 1010, which may include a controller, to transmit and receive wireless signals. For example, processor 1010 execute programs stored in memory 1012 to control antenna 1016 transmit a wireless signal to a cellular network and receive a wireless signal from a cellular network.
Display system 1018 may include a liquid crystal display (LCD), a touch screen display, or other suitable display device. Display system 1018 may be controlled to display textual and graphical information and output to text and graphics through a display device. When implemented with a touch screen display, the display system may receive input and transmit the input to processor 1010 and memory 1012.
Input devices 1020 provide a portion of a user interface. Input devices 1020 may include an alpha-numeric keypad, such as a keyboard, for inputting alpha-numeric and other information, a touch-screen, microphone, camera, buttons or switches, a trackball, stylus, or cursor direction keys.
Microphone 1022 may include one or more microphone devices which transmit captured acoustic signals to processor 1010 and memory 1012. The acoustic signals may be processed to transmit over a network via antenna 1016.
Speaker 1024 may provide an audio output for mobile device 1000. For example, a signal received at antenna 1016 may be processed by a program stored in memory 1012 and executed by processor 1010. The output of the executed program may be provided to speaker 1024 which provides audio. Additionally, processor 1010 may generate an audio signal, for example an audible alert, and output the audible alert through speaker 1024.
The mobile device system 1000 as shown in
The components contained in the computer system 1000 of
The foregoing detailed description of the technology herein has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the technology and its practical application to thereby enable others skilled in the art to best utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the technology be defined by the claims appended hereto.
