The present invention is directed to a method, apparatus, and system for capturing data. More particularly, the present invention is directed to a method, apparatus, and system for capturing data exchanged between a server and a user.
For systems employing interactions between a user and server, it is often desirable to be able to view the interactions, ideally in a manner that is transparent to the user. This is particularly desirable in a context such as sales, customer service, or e-commerce, where interactions between customers and a service provider are important indicators of customer satisfaction.
Attempts have been made to recreate interactions between a user and a server. For example, click stream analysis procedures have been used to recreate interactions between a web user and a web service provider. This type of procedure is analogous to reviewing and analyzing the script to a movie. While this procedure reveals some information about the interaction between the server and the user, it does not provide a clear tangible picture of special effects, the environment, chemistry between the user and the server, etc.
Other attempts have been made to replay recorded interactions between a server and a user. However, these attempts are typically implemented at the server and are thus suitable only for a particular type of server.
There is thus a need for a way of capturing data exchanged between a server and a user in a manner that is independent of the server and transparent to the user and that provides a full picture of the interaction between the server and the user.
The present invention is directed to a method, apparatus and system for capturing data exchanged between at least one server and at least one user.
According to exemplary embodiments, data from the user destined for the server or data from the server destined for the user is intercepted in a capturing module that operates independently from the server and the user. A copy of the intercepted data is stored in a database. The intercepted data destined for the server is forwarded to the server, or the intercepted data destined for the user is forwarded to the user. The intercepting and storing are performed transparently to the user. Interception may be performed continuously. Storing of data may be performed during a predetermined time window or at predetermined time intervals.
According to exemplary embodiments, the intercepted data includes events, attributes, images, user identifications, requests, and/or responses. Only a predetermined portion of the data and/or data that satisfies predefined rules may be stored.
According to an exemplary embodiment, the user is a web server or a web browser, and the server is a web server. The data may be exchanged in the form of an Internet protocol and assembled into a web page view for the user.
Data may be exchanged between a plurality of servers and the user, the server and a plurality of users, or a plurality of servers and a plurality of users. The intercepting and storing may be performed concurrently for capturing data exchanged simultaneously between the plurality of servers and the user, the server and the plurality of users, or the plurality of servers and the plurality of users.
Further objects, advantages and features of the present invention will become more apparent when reference is made to the following description taken in conjunction with the accompanying drawings.
According to exemplary embodiments, data exchanged between a server and a user is captured in a manner that is independent of the server and transparent to the user. In the following description, the server is referred to as a web server, and the user is referred to as a web browser. It will be appreciated, however, that the invention may be applicable to other types of servers and users.
The web browser 120 may be implemented in a personal computer, a telephone, etc. The web server 100 may be implemented as a server supporting any operating system, e.g., Unix, Linux, NT or Windows 2000.
The page capture module 110 is arranged between the web server 100 and the web browser 120. For security purposes, a firewall 115 may separate the web browser 120 and the page capture module 110.
The page capture module 110 operates independently from the web server 100 and the web browser 120. Thus, the page capture module 110 does not need to be customized for each type of web server but may be used with any web server, supporting any operating system.
Although the page capture module 110 operates independently from the web server 100 and the web browser, it may be implemented in the same device as the web server 100 or the web browser 120.
According to an exemplary embodiment, the page capture module 110 intercepts data exchanged over the Internet using the HyperText Transfer Protocol (HTTP). Both HTTP (unsecure) and HTTPS (secure) protocols may be supported by the page capture module 110. For secure protocols, a security certificate is shared between the web server 100 and the page capturing module 110. In addition, other types of data stream protocols may be supported, e.g., extensible Markup Language (XML) and socket based data transfers.
According to exemplary embodiments, the page capture module 110 acts as a redirection or proxy server from the user's perspective. The page capture module 110 listens on a specified port, such as port 80 for HTTP or port 443 for HTPPS, and then redirects all browser requests to the web server 100, which is configured to listen on a port other than ports 80 or 443. Of course, if the page capture module 110 is implemented in the same device as the web server 100, the web server may listen on the same port. Web server responses are intercepted by the page capture module 110 and redirected back down to the web browser 120.
The page capture module 110 captures pages and other data exchanged between the web server 100 and the browser 120. Pages and other data may be captured continually or at designated intervals or time windows. The page capture module 110 may also record these pages and other data, or recording may be performed in a separate recorder server connected to the page capture module.
Each web browser 120 is assigned a unique machine identity (ID) by the web server 100. A persistent machine ID cookie may be created by the web server 110 and stored at the web browser 120 for this purpose. All pages served to a particular web browser 120 are identified and grouped by the machine ID.
Although the module 110 is described as a page capture module, according to exemplary embodiments, other types of data may also be captured. For example, events and attributes may be captured. Attributes may be captured in a manner similar to that in which pages are captured, as described above.
For event capturing, according to an exemplary embodiment an event capture module captures user side events and delivers these to the page capture module 110. The event capture module may be implemented as an applet 130 that is downloaded to the web browser 120. Although shown as a separate component, the event capture applet 130 is stored at the browser, with parameters such as the web browser machine ID, the host Internet Protocol (IP) address, and the current page name. The event capture applet 130 may be notified, for example, by JavaScript embedded in the current page, whenever an event needs to be recorded. The event capture applet 130 records events such as: page load, page unload, page scroll, page resize, and browser exit. The event capture applet 130 sends captured events to the page capturing module 110 via, for example, a Transmission Control Protocol/Internet Protocol (TCP/IP) socket connection on port 80 (or port 443 for secure exchanges).
For event capturing, an HTTP request header containing a unique signature or identifier may be used to send a captured event to the page capture module 110. The unique signature may take the form of an application defined request header. Captured event data may include a browser machine ID, a page name, an event type ID, and event data.
According to an exemplary embodiment, the page capture module 110 intercepts HTTP requests that are identified as event captured HTTP requests and does not send the event captured HTTP request to the web server 110.
According to an exemplary embodiment, each captured page is assigned a unique page ID and is associated with a specific browser user machine ID. Each page may also contain the date and time that the page was captured and the page status (recording, processing, playback, etc.) After pages are captured, this information is extracted from the captured page, and a new record is inserted into a database 145.
The page preprocessor 125 acts as a recorder server and stores the captured data in a device such as a database 145. The pages 135 are passed on to the page post-processor 140. Alternatively, the page capture module 110 may perform this recording. To reduce the amount of storage necessary, only predetermined portions of data may be stored, e.g., the request portion or the response portion. Also, only data satisfying predetermined rules, e.g., rules indicating timing, may be stored. When the captured pages are recorded, identifying information may also be recorded, e.g., a session record ID, a date/time of recording, a machine ID, etc.
The post-processing module 140 determines which captured data satisfies predefined rules, e.g., business rules, and records this data in a file 180, such as a Java ARchive (JAR) file. The database 145 is updated to indicate what captured data has been selected and recorded for playback. An exemplary post-processor is described in more detail in the afore-mentioned application entitled “Method, Apparatus, and System for Processing Data Captured During Exchanges Between a Server and a User”.
A playback tool 190 selects recorded data from the database 180, using the information in the database 145. An exemplary playback tool is described in more detail in the afore-mentioned application entitled “Method, Apparatus, and System for Replaying Data Selected From Among Data Captured During Exchanges Between a Server and a User”.
Although not shown in the interest of simplifying the illustrations, it will be appreciated that the system in
If the cookie (or other persistent ID) is created successfully or the browser machine ID cookie exists, a determination is made whether event capturing is enabled, e.g., whether the event capture applet 130 has been notified that an events needs to be recorded, at step 235. If not, the page is sent to the page preprocessor at step 240. Otherwise, the event is captured at step 245.
An exemplary process for recording captured requests and responses is shown in
According to exemplary embodiments, a user interaction with a server is captured. For a web user and a web server, this information may be used to recreate the web experience of the user. For example, in a customer service context, the flow of customer inputs and the text and images that are displayed on the pages that are viewed by the customer may be captured and recorded. In such a context, this recreation will provide contact center personnel the ability to observe the customer's experience for the purpose of analyzing how well service was delivered and whether or not the needs of the customer were met. The invention may also be useful in other contexts, such as sales or e-commerce.
According to exemplary embodiments, all the user actions within a browser session may be captured, without degrading the performance of the browser session or web host environment. Also, all captured data may be transmitted efficiently with low network bandwidth utilization. Thousands of concurrent users may be supported simultaneously, with the recording of a large number of sessions that are simultaneously occurring on different web sites provided by various web servers.
It should be understood that the foregoing description and accompanying drawings are by example only. A variety of modifications are envisioned that do not depart from the scope and spirit of the invention. The above description is intended by way of example only and is not intended to limit the present invention in any way.
This application is a continuation of U.S. Ser. No. 11/534,410, filed Sep. 22, 2006, which is a continuation of U.S. Ser. No. 10/061,469 filed Jan. 31, 2002, which are both incorporated herein by reference, and is related to commonly assigned U.S. Ser. Nos. 10/061,489 and 10/061,491, both of which were filed on Jan. 31, 2002 and are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
3594919 | De Bell et al. | Jul 1971 | A |
3705271 | De Bell et al. | Dec 1972 | A |
4684349 | Ferguson et al. | Aug 1987 | A |
5101402 | Chiu et al. | Mar 1992 | A |
5267865 | Lee et al. | Dec 1993 | A |
5311422 | Loftin et al. | May 1994 | A |
5511165 | Brady et al. | Apr 1996 | A |
5526407 | Russell et al. | Jun 1996 | A |
5581614 | Ng et al. | Dec 1996 | A |
5597312 | Bloom et al. | Jan 1997 | A |
5710591 | Bruno et al. | Jan 1998 | A |
5717879 | Moran et al. | Feb 1998 | A |
5742736 | Haddock | Apr 1998 | A |
5790798 | Beckett, II et al. | Aug 1998 | A |
5809250 | Kisor | Sep 1998 | A |
5825869 | Brooks et al. | Oct 1998 | A |
5926836 | Blumenau | Jul 1999 | A |
5958016 | Chang et al. | Sep 1999 | A |
5964836 | Rowe et al. | Oct 1999 | A |
5970455 | Wilcox et al. | Oct 1999 | A |
5978648 | George et al. | Nov 1999 | A |
6009429 | Greer et al. | Dec 1999 | A |
6014134 | Bell et al. | Jan 2000 | A |
6014647 | Nizzari et al. | Jan 2000 | A |
6018619 | Allard et al. | Jan 2000 | A |
6035332 | Ingrassia et al. | Mar 2000 | A |
6038544 | Machin et al. | Mar 2000 | A |
6039575 | L'Allier et al. | Mar 2000 | A |
6057841 | Thurlow et al. | May 2000 | A |
6061798 | Coley et al. | May 2000 | A |
6076099 | Chen et al. | Jun 2000 | A |
6078894 | Clawson et al. | Jun 2000 | A |
6108711 | Beck et al. | Aug 2000 | A |
6115751 | Tam et al. | Sep 2000 | A |
6122665 | Bar et al. | Sep 2000 | A |
6122668 | Teng et al. | Sep 2000 | A |
6138139 | Beck et al. | Oct 2000 | A |
6144991 | England | Nov 2000 | A |
6146148 | Stuppy | Nov 2000 | A |
6154771 | Rangan et al. | Nov 2000 | A |
6157808 | Hollingsworth | Dec 2000 | A |
6171109 | Ohsuga | Jan 2001 | B1 |
6182094 | Humpleman et al. | Jan 2001 | B1 |
6195679 | Bauersfeld et al. | Feb 2001 | B1 |
6201948 | Cook et al. | Mar 2001 | B1 |
6211451 | Tohgi et al. | Apr 2001 | B1 |
6225993 | Lindblad et al. | May 2001 | B1 |
6230197 | Beck et al. | May 2001 | B1 |
6236977 | Verba et al. | May 2001 | B1 |
6244758 | Solymar et al. | Jun 2001 | B1 |
6286030 | Wenig et al. | Sep 2001 | B1 |
6286046 | Bryant | Sep 2001 | B1 |
6288753 | DeNicola et al. | Sep 2001 | B1 |
6289340 | Puram et al. | Sep 2001 | B1 |
6301462 | Freeman et al. | Oct 2001 | B1 |
6301573 | McIlwaine et al. | Oct 2001 | B1 |
6324282 | McIlwaine et al. | Nov 2001 | B1 |
6336093 | Fasciano | Jan 2002 | B2 |
6347374 | Drake et al. | Feb 2002 | B1 |
6351467 | Dillon | Feb 2002 | B1 |
6370574 | House et al. | Apr 2002 | B1 |
6404857 | Blair et al. | Jun 2002 | B1 |
6408064 | Fedorov et al. | Jun 2002 | B1 |
6418471 | Shelton et al. | Jul 2002 | B1 |
6421427 | Hill et al. | Jul 2002 | B1 |
6437818 | Ludwig et al. | Aug 2002 | B1 |
6445776 | Shank et al. | Sep 2002 | B1 |
6453345 | Trcka et al. | Sep 2002 | B2 |
6459787 | McIlwaine et al. | Oct 2002 | B2 |
6493758 | McLain | Dec 2002 | B1 |
6502131 | Vaid et al. | Dec 2002 | B1 |
6510220 | Beckett, II et al. | Jan 2003 | B1 |
6510469 | Starnes et al. | Jan 2003 | B1 |
6535909 | Rust | Mar 2003 | B1 |
6542602 | Elazar | Apr 2003 | B1 |
6543004 | Cagle et al. | Apr 2003 | B1 |
6546405 | Gupta et al. | Apr 2003 | B2 |
6583806 | Ludwig et al. | Jun 2003 | B2 |
6584179 | Fortier et al. | Jun 2003 | B1 |
6603854 | Judkins et al. | Aug 2003 | B1 |
6606657 | Zilberstein et al. | Aug 2003 | B1 |
6665644 | Kanevsky et al. | Dec 2003 | B1 |
6674447 | Chiang et al. | Jan 2004 | B1 |
6683633 | Holtzblatt et al. | Jan 2004 | B2 |
6697858 | Ezerzer et al. | Feb 2004 | B1 |
6724887 | Eilbacher et al. | Apr 2004 | B1 |
6738456 | Wrona et al. | May 2004 | B2 |
6757361 | Blair et al. | Jun 2004 | B2 |
6772396 | Cronin et al. | Aug 2004 | B1 |
6775377 | McIlwaine et al. | Aug 2004 | B2 |
6785370 | Glowny et al. | Aug 2004 | B2 |
6792575 | Samaniego et al. | Sep 2004 | B1 |
6810414 | Brittain | Oct 2004 | B1 |
6820083 | Nagy et al. | Nov 2004 | B1 |
6823384 | Wilson et al. | Nov 2004 | B1 |
6870916 | Henrikson et al. | Mar 2005 | B2 |
6901438 | Davis et al. | May 2005 | B1 |
6915314 | Jackson et al. | Jul 2005 | B2 |
6937706 | Bscheider et al. | Aug 2005 | B2 |
6959078 | Eilbacher et al. | Oct 2005 | B1 |
6965886 | Govrin et al. | Nov 2005 | B2 |
6993494 | Boushy et al. | Jan 2006 | B1 |
7003517 | Seibel et al. | Feb 2006 | B1 |
7047297 | Huntington et al. | May 2006 | B2 |
20010000962 | Rajan | May 2001 | A1 |
20010014143 | Kuhn | Aug 2001 | A1 |
20010032335 | Jones | Oct 2001 | A1 |
20010043697 | Cox et al. | Nov 2001 | A1 |
20020038363 | MacLean | Mar 2002 | A1 |
20020052948 | Baudu et al. | May 2002 | A1 |
20020065911 | Von Klopp et al. | May 2002 | A1 |
20020065912 | Catchpole et al. | May 2002 | A1 |
20020112048 | Gruyer et al. | Aug 2002 | A1 |
20020128925 | Angeles | Sep 2002 | A1 |
20020143925 | Pricer et al. | Oct 2002 | A1 |
20020165954 | Eshghi et al. | Nov 2002 | A1 |
20020184527 | Chun et al. | Dec 2002 | A1 |
20030055883 | Wiles et al. | Mar 2003 | A1 |
20030079020 | Gourraud et al. | Apr 2003 | A1 |
20030140121 | Adams | Jul 2003 | A1 |
20030144900 | Whitmer | Jul 2003 | A1 |
20030154240 | Nygren et al. | Aug 2003 | A1 |
20040100507 | Hayner et al. | May 2004 | A1 |
20040165717 | McIlwaine et al. | Aug 2004 | A1 |
20040249650 | Freedman et al. | Dec 2004 | A1 |
20050138560 | Lee et al. | Jun 2005 | A1 |
20050240656 | Blair | Oct 2005 | A1 |
20060165003 | Partridge | Jul 2006 | A1 |
Number | Date | Country | |
---|---|---|---|
20080281870 A1 | Nov 2008 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11534410 | Sep 2006 | US |
Child | 12118781 | US | |
Parent | 10061469 | Jan 2002 | US |
Child | 11534410 | US |