The current invention relates generally to computer system applications and more specifically protecting privacy.
The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also be inventions.
In conventional database systems, users access their data resources in one logical database. A user of such a conventional system typically retrieves data from and stores data on the system using the user's own systems. A user system might remotely access one of a plurality of server systems that might in turn access the database system. Data retrieval from the system might include the issuance of a query from the user system to the database system. The database system might process the request for information received in the query and send to the user system information relevant to the request. In conventional production software debugging, software developers use a variety of tools to browse, inspect, and analyze heap dumps. In general, a heap dump is a compilation of the output of all the live objects and classes in the system memory at a particular point in time. The ability to view and analyze heap dumps may allow software developers to debug system crashes due to memory shortage errors and determine application memory footprints, among other things.
Unfortunately, memory heap dumps may often contain sensitive or secure client data that developers have an obligation to keep private and confidential. Conventional debugging tools and approaches might disclose and compromise sensitive and secure client data to those assigned to debug and fix production application issues.
Accordingly, it is desirable to provide techniques enabling the scrubbing of sensitive and/or secure client data information from production application heap dumps without eliminating non-sensitive technical information.
In accordance with embodiments, there are provided mechanisms and methods for scrubbing portions of information, which are not to be used in debugging processes, from files intended for use in debugging. These mechanisms and methods for scrubbing information can enable embodiments to scrub (remove and/or mask) secured and sensitive client data, that is to remain confidential, while retaining the data that is not sensitive in the file. The ability of embodiments to scrub and/or mask sensitive and confidential customer data that is present within heap dump information, can enable the use of the non-sensitive portions of heap dump information by those assigned to debug and fix production application issues, without compromising the confidential client data.
In an embodiment and by way of example, a method for scrubbing information is provided. The method embodiment includes parsing a heap dump, mapping the heap dump for class dependencies and inheritance, scrubbing certain information based on heuristic rules, and rebuilding the heap dump.
While the present invention is described with reference to an embodiment in which techniques for scrubbing information are implemented in a system having an application server providing a front end for an on-demand database service capable of supporting multiple tenants, the present invention is not limited to multi-tenant databases, is not limited to databases, and is not limited to deployment on application servers. Embodiments may be practiced using other database architectures, e.g., ORACLE®, DB2® by IBM and the like without departing from the scope of the embodiments claimed.
Any of the embodiments described in the specification may be used alone or together with one another in any combination. Inventions encompassed within this specification may also include embodiments that are only partially mentioned or alluded to or are not mentioned or alluded to at all in this brief summary or in the abstract. Although various embodiments of the invention may have been motivated by various deficiencies with the prior art, which may be discussed or alluded to in one or more places in the specification, the embodiments of the invention do not necessarily address any of these deficiencies. In other words, different embodiments of the invention may address different deficiencies that may be discussed in the specification. Some embodiments may only partially address some deficiencies or just one deficiency that may be discussed in this specification, and some embodiments may not address any of these deficiencies.
In the following drawings like reference numbers are used to refer to like elements. Although the following figures depict various examples of the invention, the invention is not limited to the examples depicted in the figures.
Systems and methods are provided for scrubbing information from a production server environment.
As used herein, the term multi-tenant database system refers to those systems in which various elements of hardware and software of the database system may be shared by one or more customers. For example, a given application server may simultaneously process requests for a great number of customers, and a given database table may store rows for a potentially much greater number of customers. As used herein, the term query plan refers to a set of steps used to access information in a database system. As used herein, the term view refers to any visual representation of information. For example, a view may be a web page, a dialog box, a window in a standalone application, or any other visual rendering of information in a user interface. As used herein, a selector refers to any visible object in an interface for indicating that a choice has been made between one or more options provided to a user of an interface. For example, a selector may be a link, an icon, a button, a tab, an item in a drop down list, an area for receiving a selection indicator such as a checkbox, a circle for entering a dot, or other marking, a field for receiving text, or any other means of indicating an option that has been chosen by a user of an interface.
System Overview
Environment 10 is an environment in which an on-demand database service exists. User system 12 may be any machine or system that is used by a user to access a database user system. For example, any of user systems 12 can be a handheld computing device, a mobile phone, a laptop computer, a work station, and/or a network of computing devices. As illustrated in
An on-demand database service, such as system 16, is a pre-established database system that is made available to outside users that do not need to necessarily be concerned with building and/or maintaining the database system, but instead may be available for their use when the users need the database system (e.g., on the demand of the users). Some on-demand database services may store information from one or more tenants stored into tables of a common database image to form a multi-tenant database system (MTS). Accordingly, “on-demand database service 16” and “system 16” will be used interchangeably herein. A database image may include one or more database objects. A relational database management system (RDMS) or the equivalent may execute storage and retrieval of information against the database object(s). Application platform 18 may be a framework that allows the applications of system 16 to run, such as the hardware and/or software, e.g., the operating system. In an embodiment, on-demand database service 16 may include an application platform 18 that enables creation, managing and executing one or more applications developed by the provider of the on-demand database service, users accessing the on-demand database service via user systems 12, or third party application developers accessing the on-demand database service via user systems 12.
The users of user systems 12 may differ in their respective capacities, and the capacity of a particular user system 12 might be entirely determined by permissions (permission levels) for the current user. For example, where a salesperson is using a particular user system 12 to interact with system 16, that user system has the capacities allotted to that salesperson. However, while an administrator is using that user system to interact with system 16, that user system has the capacities allotted to that administrator. In systems with a hierarchical role model, users at one permission level may have access to applications, data, and database information accessible by a lower permission level user, but may not have access to certain applications, database information, and data accessible by a user at a higher permission level. Thus, different users will have different capabilities with regard to accessing and modifying application and database information, depending on a user's security or permission level.
Network 14 is any network or combination of networks of devices that communicate with one another. For example, network 14 can be any one or any combination of a LAN (local area network), WAN (wide area network), telephone network, wireless network, point-to-point network, star network, token ring network, hub network, or other appropriate configuration. As the most common type of computer network in current use is a TCP/IP (Transfer Control Protocol and Internet Protocol) network, such as the global internetwork of networks often referred to as the “Internet” with a capital “I,” that network will be used in many of the examples herein. However, it should be understood that the networks that the present invention might use are not so limited, although TCP/IP is a frequently implemented protocol.
User systems 12 might communicate with system 16 using TCP/IP and, at a higher network level, use other common Internet protocols to communicate, such as HTTP, FTP, AFS, WAP, etc. In an example where HTTP is used, user system 12 might include an HTTP client commonly referred to as a “browser” for sending and receiving HTTP messages to and from an HTTP server at system 16. Such an HTTP server might be implemented as the sole network interface between system 16 and network 14, but other techniques might be used as well or instead. In some implementations, the interface between system 16 and network 14 includes load sharing functionality, such as round-robin HTTP request distributors to balance loads and distribute incoming HTTP requests evenly over a plurality of servers. At least as for the users that are accessing that server, each of the plurality of servers has access to the MTS' data; however, other alternative configurations may be used instead.
In one embodiment, system 16, shown in
One arrangement for elements of system 16 is shown in
Several elements in the system shown in
According to one embodiment, each user system 12 and all of its components are operator configurable using applications, such as a browser, including computer code run using a central processing unit such as an Intel Pentium® processor or the like. Similarly, system 16 (and additional instances of an MTS, where more than one is present) and all of their components might be operator configurable using application(s) including computer code to run using a central processing unit such as processor system 17, which may include an Intel Pentium® processor or the like, and/or multiple processor units. A computer program product embodiment includes a machine-readable storage medium (media) having instructions stored thereon/in which can be used to program a computer to perform any of the processes of the embodiments described herein. Computer code for operating and configuring system 16 to intercommunicate and to process webpages, applications and other data and media content as described herein are preferably downloaded and stored on a hard disk, but the entire program code, or portions thereof, may also be stored in any other volatile or non-volatile memory medium or device as is well known, such as a ROM or RAM, or provided on any media capable of storing program code, such as any type of rotating media including floppy disks, optical discs, digital versatile disk (DVD), compact disk (CD), microdrive, and magneto-optical disks, and magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data. Additionally, the entire program code, or portions thereof, may be transmitted and downloaded from a software source over a transmission medium, e.g., over the Internet, or from another server, as is well known, or transmitted over any other conventional network connection as is well known (e.g., extranet, VPN, LAN, etc.) using any communication medium and protocols (e.g., TCP/IP, HTTP, HTTPS, Ethernet, etc.) as are well known. It will also be appreciated that computer code for implementing embodiments of the present invention can be implemented in any programming language that can be executed on a client system and/or server or server system such as, for example, C, C++, HTML, any other markup language, Java™, JavaScript, ActiveX, any other scripting language, such as VBScript, and many other programming languages as are well known may be used. (Java™ is a trademark of Sun Microsystems, Inc.).
According to one embodiment, each system 16 is configured to provide webpages, forms, applications, data and media content to user (client) systems 12 to support the access by user systems 12 as tenants of system 16. As such, system 16 provides security mechanisms to keep each tenant's data separate unless the data is shared. If more than one MTS is used, they may be located in close proximity to one another (e.g., in a server farm located in a single building or campus), or they may be distributed at locations remote from one another (e.g., one or more servers located in city A and one or more servers located in city B). As used herein, each MTS could include one or more logically and/or physically connected servers distributed locally or across one or more geographic locations. Additionally, the term “server” is meant to include a computer system, including processing hardware and process space(s), and an associated storage system and database application (e.g., OODBMS or RDBMS) as is well known in the art. It should also be understood that “server system” and “server” are often used interchangeably herein. Similarly, the database object described herein can be implemented as single databases, a distributed database, a collection of distributed databases, a database with redundant online or offline backups or other redundancies, etc., and might include a distributed database or storage network and associated processing intelligence.
User system 12, network 14, system 16, tenant data storage 22, and system data storage 24 were discussed above in
Application platform 18 includes an application setup mechanism 38 that supports application developers' creation and management of applications, which may be saved as metadata into tenant data storage 22 by save routines 36 for execution by subscribers as one or more tenant process spaces 104 managed by tenant management process 110 for example. Invocations to such applications may be coded using PL/SOQL 34 that provides a programming language style interface extension to API 32. A detailed description of some PL/SOQL language embodiments is discussed in commonly owned co-pending U.S. Provisional Patent Application 60/828,192 entitled, PROGRAMMING LANGUAGE METHOD AND SYSTEM FOR EXTENDING APIS TO EXECUTE IN CONJUNCTION WITH DATABASE APIS, by Craig Weissman, filed Oct. 4, 2006, which is incorporated in its entirety herein for all purposes. Invocations to applications may be detected by one or more system processes, which manages retrieving application metadata 116 for the subscriber making the invocation and executing the metadata as an application in a virtual machine. In an embodiment, application platform 18 may include application monitoring system 40, which contains the machine readable instructions for scrubbing sensitive client information. Application monitoring system 40 might also track application performance or check for application errors.
Each application server 100 may be communicably coupled to database systems, e.g., having access to system data 25 and tenant data 23, via a different network connection. For example, one application server 1001 might be coupled via the network 14 (e.g., the Internet), another application server 100N-1 might be coupled via a direct network link, and another application server 100N might be coupled by yet a different network connection. Transfer Control Protocol and Internet Protocol (TCP/IP) are typical protocols for communicating between application servers 100 and the database system. However, it will be apparent to one skilled in the art that other transport protocols may be used to optimize the system depending on the network interconnect used.
In certain embodiments, each application server 100 is configured to handle requests for any user associated with any organization that is a tenant. Because it is desirable to be able to add and remove application servers from the server pool at any time for any reason, there is preferably no server affinity for a user and/or organization to a specific application server 100. In one embodiment, therefore, an interface system implementing a load balancing function (e.g., an F5 Big-IP load balancer) is communicably coupled between the application servers 100 and the user systems 12 to distribute requests to the application servers 100. In one embodiment, the load balancer uses a least connections algorithm to route user requests to the application servers 100. Other examples of load balancing algorithms, such as round robin and observed response time, also can be used. For example, in certain embodiments, three consecutive requests from the same user could hit three different application servers 100, and three requests from different users could hit the same application server 100. In this manner, system 16 is multi-tenant, wherein system 16 handles storage of, and access to, different objects, data and applications across disparate users and organizations.
As an example of storage, one tenant might be a company that employs a sales force where each salesperson uses system 16 to manage their sales process. Thus, a user might maintain contact data, leads data, customer follow-up data, performance data, goals and progress data, etc., all applicable to that user's personal sales process (e.g., in tenant data storage 22). In an example of a MTS arrangement, since all of the data and the applications to access, view, modify, report, transmit, calculate, etc., can be maintained and accessed by a user system having nothing more than network access, the user can manage his or her sales efforts and cycles from any of many different user systems. For example, if a salesperson is visiting a customer and the customer has Internet access in their lobby, the salesperson can obtain critical updates as to that customer while waiting for the customer to arrive in the lobby.
While each user's data might be separate from other users' data regardless of the employers of each user, some data might be organization-wide data shared or accessible by a plurality of users or all of the users for a given organization that is a tenant. Thus, there might be some data structures managed by system 16 that are allocated at the tenant level while other data structures might be managed at the user level. Because an MTS might support multiple tenants including possible competitors, the MTS should have security protocols that keep data, applications, and application use separate. Also, because many tenants may opt for access to an MTS rather than maintain their own system, redundancy, up-time, and backup are additional functions that may be implemented in the MTS. In addition to user-specific data and tenant-specific data, system 16 might also maintain system level data usable by multiple tenants or other data. Such system level data might include industry reports, news, postings, and the like that are sharable among tenants.
In certain embodiments, user systems 12 (which may be client systems) communicate with application servers 100 to request and update system-level and tenant-level data from system 16 that may require sending one or more queries to tenant data storage 22 and/or system data storage 24. System 16 (e.g., an application server 100 in system 16) automatically generates one or more SQL statements (e.g., one or more SQL queries) that are designed to access the desired information. System data storage 24 may generate query plans to access the requested data from the database.
Each database can generally be viewed as a collection of objects, such as a set of logical tables, containing data fitted into predefined categories. A “table” is one representation of a data object, and may be used herein to simplify the conceptual description of objects and custom objects according to the present invention. It should be understood that “table” and “object” may be used interchangeably herein. Each table generally contains one or more data categories logically arranged as columns or fields in a viewable schema. Each row or record of a table contains an instance of data for each category defined by the fields. For example, a CRM database may include a table that describes a customer with fields for basic contact information such as name, address, phone number, fax number, etc. Another table might describe a purchase order, including fields for information such as customer, product, sale price, date, etc. In some multi-tenant database systems, standard entity tables might be provided for use by all tenants. For CRM database applications, such standard entities might include tables for Account, Contact, Lead, and Opportunity data, each containing pre-defined fields. It should be understood that the word “entity” may also be used interchangeably herein with “object” and “table”.
In some multi-tenant database systems, tenants may be allowed to create and store custom objects, or they may be allowed to customize standard entities or objects, for example by creating custom fields for standard objects, including custom index fields. U.S. patent application Ser. No. 10/817,161, filed Apr. 2, 2004, entitled “Custom Entities and Fields in a Multi-Tenant Database System”, and which is hereby incorporated herein by reference, teaches systems and methods for creating custom objects as well as customizing standard objects in a multi-tenant database system. In certain embodiments, for example, all custom entity data rows are stored in a single multi-tenant physical table, which may contain multiple logical tables per organization. It is transparent to customers that their multiple “tables” are in fact stored in one large table or that their data may be stored in the same table as the data of other customers.
Next, mechanisms and methods for enabling the scrubbing of confidential information from a production server environment will be described with reference to example embodiments.
Overview of Parsing the Heap Dump File
In this specification, a heap dump is a data snapshot of the state of the heap of a software application, at a specific time (such as the time of a software crash). The heap may be a portion of the working memory, which may be volatile memory, such as random access memory, for example. The working memory may be memory that is used to perform computations necessary to accomplish tasks currently be performed. The heap dump may contain information about the elements of a software application that were in use at the time the heap dump was created, such as the classes that define the application's functionality and the objects that are instances of those classes.
In an embodiment, the language generating the heap dump data may be Java. Using Java, for example, when an Out of Memory occurs (that is when a particular object or routine uses all or essentially all of the memory allocated for the object or routine), as part of an error handling routine, the Java Virtual Machine heap may be dumped in ASCII or binary format, for example. In an embodiment, heap dump data in a binary format may be easier to process than were the heap dump data formatted using other formats. Accordingly, the binary format may be used by some heap dump analysis tools. Since the format of the heap dump may be binary, the heap dump may contain 1-2 gigabytes of data. In an alternative embodiment, the heap dump data may be formatted in a format other than the binary format.
In the specification, “objects in the heap dump” refers to information, in a heap dump, about the objects that existed in the memory of an application at the time a heap dump was performed. The information about the objects in the heap dump file may include details such as
(1) the dependencies of each object (e.g., other objects the current object relies on for at least some portion of the object's functionality),
(2) the referential data of each of the objects (e.g., the objects another particular object made references to, and the objects that particular object was referenced by),
(3) the type of class each object is an instance of,
(4) the amount of memory allocated for each object,
(5) the memory address of each object, etc.
Heap dump information may not be easy to understand without knowing the relationships between the objects in the heap dump, and the information about the classes the objects belong to. Parsing the heap dump is useful in determining the relationship between heap dump objects. The heap dump is parsed by examining the heap dump for information related to objects and recreating the objects, by loading the objects and their dependencies into active memory. The loading of the heap dump objects into memory enables the organization of heap dump objects and other heap dump data into a format suitable for viewing, analyzing, mapping and modification, as discussed below.
In an embodiment, the machine readable instructions for executing the heuristics rules are stored within application monitoring system 40 as well as the instructions for mapping, parsing, scrubbing, and rebuilding the heap dump. Application monitoring system 40 also contains machine readable instructions for monitoring applications.
Scrubbing heuristics 300 is the code that includes the code for implementing the scrubbing process. Rules 301 are the heuristic rules that determine what is scrubbed and/or what is allowed to be scrubbed. Object class 302 includes one or more rules that identify an object based on the object's class. For example the user may enter a list of classes that are expected to contain sensitive information, and certain string variables or any string variable in that class may be scrubbed. Object inheritance 304 includes one or more rules that determine that an object may need to be scrubbed based on an inheritance from a class that is flagged for scrubbing. For example, string variables over a certain size that are inherited from a particular class may be scrubbed automatically. Dependencies 306 may include rules that are based on dependencies of an object. For example, if an object is a child of an object that has variables that need to be scrubbed or that is flagged for scrubbing, the object may be considered for scrubbing. Object name 308 includes rules that are based on the name of the object. For example, the user may enter in a set of names of objects that need to be scrubbed. Alternatively or additionally, the system may automatically look for variables and/or objects containing certain strings, such as “credit,” “address,” and “name.” Object type 310 may allow objects to be scrubbed based on the object's type, and may consider an object for scrubbing or allow an object to be scrubbed if the object is of a certain type. Object size 312 may include rules for scrubbing an object based on the object's size. For example, if an object is larger than a certain size, the object may be considered for scrubbing. In an embodiment, the scrubbing may occur automatically, if certain combinations of rules are met. For example, if the object belongs to a certain set of classes flagged as having objects that need scrubbing, inherited from a flagged class, is a child of a certain class, and if the object also is greater than a certain size, has certain strings in the name, is a string variable, or of a certain type the object may be automatically scrubbed. Mapping instructions 314 may include instructions for determining the names of objects and the dependencies of the objects in a heap dump (see the description of
In an embodiment in which an object oriented language is used, class inheritance properties may be used to map dependencies. Inheritance properties may be determined by a heap dump tool based on the manner in which elements of an object are referenced and implemented, as elaborated on in the example below.
The following example is referenced in the discussion examples below. The class User may exist for receiving and storing information associated with a user of a software application. In a particular run of a particular code the class User may be assigned the memory address 0x3384290. The class User may contain the variables userName, UserID, UserPassword, and UserAddress, and the function enrollUser( ). The class Tenant may be created to extend the functionality of the class User to include the enrollment of a distinct category of users defined as “tenants.” In a particular run of a particular code the class Tenant may be assigned the memory address 0x9274365. The Tenant class inherits the fields userName, UserID, UserPassword, and UserAddress, and the function enrollUser( ) from the User class, while adding new variables such as subscribedService and BillingMethod, and new functions such as setBillingInformation( ). In a particular run the variable BillingMethod may have an address 0x30000080 [10000] A for storing a credit card number (e.g., Visa-4605-1978-0928-0000), and the function setBilling information may be assigned a memory address 0x50001010 [6000] G for storing the name of the entity to which invoices are addressed (e.g., MarcoSoft, Inc.) and a memory address 0x500020200 [4000] B for storing the billing address (e.g, 1942 Blitz Way). The class EndUser may be created to extend the functionality of the class User to include the enrollment of a distinct category of users defined as “end users.” In a particular run of a particular code the class EndUser may be assigned the memory address 0x4365927. The EndUser class inherits the variables userName, UserID, UserPassword, and UserAddress, and the function enrollUser( ) from the User class, while adding new variables such as clientofTenant, and new functions such as associateEndUserWithTenant( ). In the above example, any object of the type Tenant or EndUser is also an object of the type User. However, an object of type Tenant is not an object of the type EndUser, and vice versa.
In step 402, the raw heap dump file is loaded (e.g., opened and placed into the working memory) for reading by a heap dump analysis tool (e.g., a tool for reading and analyzing heap dump data), so that the raw data may be worked on further. In step 404, the heap dump analysis tool parses the raw heap dump data by, for example, scanning the raw heap dump for data related to the objects and the objects' dependencies, and loading equivalent objects and information related to the dependencies of the objects into active memory. For example, the raw heap dump data may include lines that list and address of an object of interest and addresses of objects that called or otherwise referenced the object of interest. During the parsing, an object is created that is labeled in order to represent the address of the object of interest, and the addresses of the objects that referenced the object of interest are stored in association with the object created to represent the object of interest. The dependencies are actually determined and mapped in the method of
In step 406, the loaded heap dump objects and other data are formatted for examination and/or modification. In optional step 408, the heap dump data is rendered for navigation via a user interface, such as a web browser. The formatting and rendering of the heap dump data are discussed in greater detail below. Step 408 may be performed when it is desirable for a human to monitor the output of method 400. However, the method of
Formatting Heap Dump Data
The formatting of the heap dump data may include organizing heap dump objects into categories, based on one or more of pieces of information about the objects, such as the class type of the objects, the number of objects of each class, the size of objects of each class, the dependencies of the objects, or other attributes of the objects. In an embodiment, the object type may be the last of a series of strings that are separated by periods in the name of the object. In other embodiments, the objects may be categorized by different object attributes, or the objects may not be categorized at all. In some embodiments, the organizing of the heap dump objects is performed automatically by a heap dump analysis tool. In other embodiments, the organizing of the heap dump objects may be performed based on the input of a user who analyzes a heap dump.
In some embodiments, the heap dump data is organized by running queries on the objects loaded into memory using an object querying language, such as OQL, and organizing the query results in a way that suitably matches the nature of the query. For example, a query may be run for selecting all objects containing strings, and returning the objects in descending order based on the size of the objects. The heap dump analysis tool may consist of a series of OQL queries (or statements in another language) that produce a series of views of the heap dump data.
Rendering Heap Dump Data for Navigation
After the heap dump data has been formatted and categorized as described in step 406, views (e.g., web pages) may be generated by the heap dump analysis tool for presenting heap dump objects by category. Selectors (e.g., links) may also be generated for navigating between the categorized views. Optionally, a selector may automatically issue an OQL query or another type of command that automatically generates a view anew. Examples of categorized views are shown below.
Example of a View Categorizing Objects by Instance Count
Example of a View Categorizing Objects by Address
In the above example views, the class names java.obj.User, java.obj.Tenant and java.obj.EndUser refer to the User, Tenant, and EndUser classes of the example described just prior to the beginning of the discussion of
Detailed views may also be generated for presenting information about individual objects, the detailed views containing details about each object that are not present in the views categorizing the objects. For example, one view may list each object and how many children each object has, and the detailed view may include information about which classes are children of a particular class and how many times the parent was invoked by each child. As described above in reference to the categorized views, selectors may be generated for navigating between the detailed views. An example of a detailed view is described in conjunction with
Overview of Mapping a Heap Dump
Mapping clarifies what is represented by the information in the parsed heap dump. For example, mapping may be used to derive information about the dependencies or inheritance of objects, and use that information to identify the specific relationship that existed between the objects and/or the classes the objects are based on. Mapping may also be used to keep track of the objects containing information that should be scrubbed, and to designate one or more rules that should be used to scrub each of the objects.
A heap dump map may be created by (1) correlating information about classes (such as the structure of the class) with information about objects constructed from the classes, (2) determining how objects derived from different classes relate to one another, (3) identifying objects and object elements that are candidates for containing information that should be scrubbed, and (4) creating a mapping of the information for keeping track of the interrelations between the classes and objects, the attributes of the classes and objects, and the candidates for scrubbing that have been identified. Mapping may be performed because an object's (or an instance dump's) name may not be easily determined from the heap dump due to the format of the object or instance dump (in an embodiment, an instance dump may be the portion of the heap dump having data about the state of objects at the time the objects were in active memory). The object's name (e.g., the name of the class the object is constructed from) is stored in the class dump that an instance dump is linked to. Due to inheritance, an object may share characteristics with another object. In an embodiment, identifying characteristics shared between objects (e.g., by identifying memory addresses that are referenced by two different objects in the instance dump) may be helpful in determining whether the objects are of the same class type, or if the objects are related to one another.
Correlating Class and Instance Dumps
In step 502, the heap dump data for each class is correlated with the heap dump data for all the objects derived from each class. In an embodiment, the correlation of the class dump with the instance dump allows the class type of each object to be determined. In an embodiment, during the correlation step 502 a determination is made as to which objects belong to which class. The relationship between the classes and storing the relationships in an organized manner for later use may be left for steps 504, 506 and 508.
In this specification, the portion of heap dump data related to the classes that define objects is referred to as the class dump, and the portion of heap dump data related to objects derived from classes is referred to as the instance dump. In other words, in an embodiment, the heap dump may include two portions, which will be referred to as the class dump and the instance dump.
Heap dump information may not be easy to understand without first knowing how a class dump corresponds or is correlated to an instance dump. For example, when object data is stored in an instance dump (or other portion of a heap dump) the name of an object's class, and other details about the object, may be unclear or difficult to determine using the object data alone (e.g., what the instance data represents may be unclear until the instance data is correlated—e.g., linked—with the class data). The class data for a particular class may include information relevant to all objects constructed based on the particular class. Correlating the class dump and the instance dump may be performed, because an object's name may not be easily determined from the heap dump due to the format of the object or instance dump. The object's name may be stored in the class dump that an instance dump may be linked with (correlating the class dump and instance dump facilitates associating the objects name with the corresponding instance dump). Additionally, the inheritance properties of objects may not be easy to understand without using the class dump's data for classes to correlate the instance dump's data about objects. Due to inheritance, an object may share characteristics with another object, and it may be desirable to determine how objects are linked by other objects to yet other objects.
A feature of classes (that determines the content of the working memory at the time of the dump that is used in performing the correlation) is that the classes (that are related to the data being scrubbed) may be similar to a blank, un-fillable template that specifies, prior to active use, the structure and functionality of a portion of application code, and the object is comparable to a fillable version, or instance, of the template defined by a class. Consequently, when an attribute of a parent class is assigned a particular value in a child class, the address for that attribute will appear in the class dumps of each the parent and child class. The class may be defined by the class attributes, such as the class name, type and size. In an embodiment, the class dump may be the portion of the heap dump data having data related to a class's attributes and the addresses in memory that were allocated for constructing objects based on the class. The class's attributes may be extracted to facilitate the correlation. In other words, the class dump may be used to identify objects that are created according to the specification of the particular class (e.g., instances of the class) or identify objects having the particular class type, and after the objects have been identified, the information may be used in correlating the class dump and the instance dump.
For example, regarding correlating and determining of the dependencies of the objects of a heap dump and step 502, the instance dump may include a list of objects that happen to be in memory at the time that the memory is dumped (to form the heap dump). Each object in the instance dump may be associated with the address of that object and the addresses of objects that called that object. Optionally, each object in the instance dump may also be stored with the size of the object, the instance number of the object, and/or the name of the object. The class dump may include a list of objects created for the source code to run, an address of the object created and addresses set aside for that object to use. Optionally, in the class dump, each object may also be stored with the amount of memory that is allocated for that object. An example of an instance dump appears below.
An Example of an Instance Dump Prior to Correlation with a Class Dump
An Example of an Instance Dump After Correlation with a Class Dump
In an embodiment, as part of step 502, the name of the object (if not present in the instance dump) and the other objects that a particular object of an instance dump is associated with (e.g., as a result of an inheritance or other dependency) may be determined by searching for the object's address in the class dump, and then locating and reading the name of the object that the address is associated with. The name of the object that the address is associated with may appear next to the address, which may be listed on the same line as the address, above the list of addresses set aside for the that object, or below the list of addresses set aside for that object (depending on the format in which the heap dumps are produced). Determining the location of the class dump and instance dump may depend on the format in which the heap dump is provided (correlating the relationship between objects will be discussed in step 506). The instance dump (e.g., as a result of including an association between the name of the object and addresses associated with the object) may contain data indicative of (1) references made by one object (which will be referred to as object x) to another objects (which is not necessarily used in determining the object name by step 502, but may be used in step 506 in detecting the object relationships), (2) the addresses in the memory of the objects that are referred by object x (which is not necessarily used in determining the object name by step 502, but may be used in step 506 in detecting the object relationships), (3) the number of child objects that exist for object x (which is not necessarily used in determining the object name by step 502, but may be used in step 506 in detecting the object relationships), and (4) additional information about the object x (for example, the object's size in memory). Instance dump data may not necessarily contain an indication of the name or type of class objects were derived from, as shown in the example above, which may need to be determined by comparing the addresses in the instance dump with the addresses of the class dump.
In step 504, the class type of each object is mapped using the identifying information that was obtained in step 502, by storing the object addresses, names, and/or other identifying information in association with one another, for example.
In step 506, the relationships between heap dump objects are detected. In other words, in general, in Object Oriented Languages each object in a heap dump was derived from one or more base objects and/or one or more parent objects, and in step 506 the relationship between the base objects and parent objects (e.g., which object is a parent and/or base object for which other objects) is detected. The class dump data (or other data) within the heap dump may be used for identifying the one or more base object and/or parent object that a specific object was inherited from. Examples of relationships between classes may be the chain of inheritance of a particular class (e.g., whether a given class has a parent class and/or base class), and the dependencies of a particular class on a non-ancestral class (a class that is not a direct parent class).
In this specification, a child class is a class established for extending the functionality of a preexisting class, such as by adding variables and/or methods that are exclusive to the extended class. The new class that is established is a child of the preexisting class, and the child class relies on the structure, data, and methods of the preexisting class for all functionality that is not exclusive to the child class.
As part of the detecting relationships in step 506, class dumps may be used to identify objects in the heap dump data that were constructed from the classes contained in the class dumps (e.g., to determine which class each object belongs to). For example, the data of a class dump may be used to identify a subset of objects within an instance dump as being objects of the class type Tenant, and another subset of objects may be identified within an instance dump as being objects of the class type EndUser. In an embodiment, a link (e.g., a piece of data, such as a shared address, or other data establishing a relationship) between a set of class dump data and a set of instance dump data may exist in the heap dump data. In an embodiment, the link is the memory address of an object.
For example, the class dump may contain the memory addresses allocated for constructing objects of each type of a class that exists in the class dump. The instance dump may contain details about the objects in the instance dump, including the address in memory of the object (as described above). If a match is found between the memory address of an object in an instance dump and one of the memory addresses of the classes associated with a class type that exists in a class dump, the object is identified as an object (e.g., instance) of the class type having the matching address. Alternatively, as a result of being constructed according to the specification defined in a class, the attributes of the object may be compared to the specification of classes contained in the class dump. As a result of the comparison, a determination can be made as to what the class the object is constructed from, and objects may then be compared to one another to determine whether or not the objects were constructed from the same class.
In an embodiment, determining relationship between classes may include one or more of the following, (1) identifying, in the heap dump data, a reference made by an object of a particular class to an element (e.g., a method, function, or field) of an object of another class (which may be performed by identifying memory addresses that are in a portion of the heap dump associated with one of the objects and identifying the same memory address in a portion of the heap dump or another dump associated with the other one of the objects); (2) determining, based on the nature of the reference (e.g., whether the reference appears in both the instance dump and class dump or just in the instance dump or whether the reference appears as an address that called an object or appears as an address that is allocated to the object), a relationship between the class of the object making the reference and the class having the method or field that is being referenced; (3) identifying information about the dependencies of an object on one or more other objects, identifying the whether an object is inherited from one or more other objects, and (4) based on the nature of the dependency or inheritance, determining a relationship between the objects.
In an embodiment, if an object does not have a parent object, does not have a base object, or does not have dependencies on other objects, the object may be identified as a non-related object (e.g., an object having no relationships to other objects). In an alternative embodiment, non-related objects may not be included in the mapping.
In an embodiment, the references made between objects may be contained in the instance dump and the names of the classes that the objects were created from may be contained within the class dump. In an embodiment, the names of the classes may be used as part of identifying relationships between objects. In an alternative embodiment, other data within the heap dump may be used for identifying relationships between objects.
In an alternative embodiment (which may be incorporated within the current embodiment), as part of step 506, dependency information may be automatically gleaned from the name of the object in the class dump (and/or in the instance dump if names of objects appear in the instance dump). For example, the heap dump (or the class dump) may be searched for strings containing one or more periods (“.”), and strings containing periods may be categorized as class method names, and the dependencies of the method may be inferred, by labeling the string before the first period as the parent class, the string between the prior period (the first period) and next periods as a child class of the parent class, and each string between each subsequent prior period and next period as a child class of the prior class until the end of the entire string in which the last string is not followed by a period, depending on the language of the source code. Similarly, the heap dump may be scanned for stings including colons or two consecutive colons, which may indicate other types of dependencies, depending on the language of the source code.
In other embodiments, the information for establishing a relationship between the class dump and the instance dump may be another piece of information related to both objects and classes. In an embodiment, the objects of the instance dump are loaded (e.g., recreated in active memory), as described in step 404, and the loaded objects are the objects used for the correlating described in the current step.
In step 508, the heap dump map is updated to include the relationships between objects that were detected in step 506. The mapping of object relationships is discussed in further detail below.
In an embodiment, objects of two different types may reference an object containing confidential data, and these two objects may also be mapped as potential candidates for having confidential information. The mapping data may be stored for later use (e.g., for rebuilding a heap dump after removing confidential information using only the portion of the heap dump's original data that does not include confidential information).
All of the objects of inherited classes may be mapped to the objects of parent classes so that it can be determined what the heap dump data means and which class the instance dump objects belong to. The mapping process may also be used to determine which classes belong to which threads, how instance dumps are linked to each other, etc. The mapped relationships may be used for heuristic based scrubbing, as described in step 512, as only data belonging to a particular class and linked to a particular object may be scrubbed. Thus in the above example, when values for variables userName, UserID, UserPassword, and UserAddress, and the function enrollUser( ) from the User class appear in the instance dump for the class tenant, a dependence may be inferred to the class user.
When a relationship between objects is detected, the heap dump tool maps the relationship between the objects, based on the nature of the relationship. As each object's relationship to other objects is detected, information about the hierarchy of the objects may also be mapped. Additionally, when objects are mapped, rules may be applied for identifying elements of an object that may contain data that should be scrubbed (e.g., erased or replaced with meaningless or unessential data), as discussed below in step 510.
In step 510, heuristic rules (rules that are heuristic in nature) are applied for determining whether an object being mapped contains data that should be scrubbed (in this specification the term heuristic rules refers to rules that are heuristic in nature). The heuristic rules may give correct results most of the time, but may not necessarily be based on fundamental principles and/or may not necessarily guarantee correct results all of the time. In an embodiment, during the mapping, the data that should be scrubbed may be identified by applying the heuristic rules, and optionally a flag (or other indicator) may be included in the mapping for indicating that the object contains data to be scrubbed and for indicating the element of the object containing the data to be scrubbed is added to the object map.
For example, one rule may identify object elements of the type string that containing 30 characters as having data to be scrubbed. Accordingly, whenever an object in the heap dump containing String elements having more than 30 characters is detected, the mapping data for that object will include an indicator that the object, or an element of the object, is a string that may contain confidential information, and should be considered for scrubbing. In an embodiment, the indicator may include information about which scrubbing rule was applied for each object that was identified as a candidate for scrubbing. Other examples of heuristic rules are, scrub objects containing strings, scrub objects inherited from a particular base class that, sometimes, often, or typically stores confidential information, scrub objects that are child objects of a particular class, scrub objects having a specified name, and scrub object types that are final or static. Heuristic rules were also discussed above in conjunction with
In step 512, the objects and object elements identified within the heap dump mapping as candidates for containing confidential information are scrubbed by implementing code associated with the scrubbing rule designated in step 510. In an embodiment, the scrubbing of an object may be the masking of confidential data contained in the object by replacing the data with zeroes, (or X's, and/or one or more other characters). For example, the heap dump data for an object of the type Tenant may include information related to the string variable BillingMethod, and the value associated with that variable, “Visa-4605-1978-0928-0000.” In the current example, the code for applying a heuristic rule for scrubbing an object of the type String may be
if an object is of the type Tenant
for each variable of type String within the object that has the name BillingMethod
replace value of that variable string with “00000000000.”
In an embodiment, the objects that are scrubbed are the objects loaded into memory in step 404. In alternative embodiments, the scrubbing may be another method of concealing or removing of confidential data contained in an object.
In step 514, the heap dump is rebuilt (e.g., recreated) by using the objects that were loaded, examined, mapped and/or scrubbed in step 404 and steps 502-512. In steps 502-512, the relationship data and or data about the re-loaded objects was mapped, and confidential data was scrubbed (if present). In the current step, the original heap dump file is rebuilt by creating a new data snapshot of the mapped and/or scrubbed objects, and saving the new dump file in manner that allows it to replace the original heap dump file.
Example of Heap Dump Prior to Scrubbing
0x30000080 [10000] G Visa-4605-1978-0928-0000
0x50001010[6000] A MacroSoft, Inc. 0x50002020
0x500020200x50002020[4000] B 1942 Blitz Way 0x10007030 0x50006040 0x50004050
Example of Heap Dump after Scrubbing
0x30000080 [10000] G 00000000000
0x50001010[6000] A 00000000000 0x50002020
The new, scrubbed heap dump created in the current step is safe for debugging and free of sensitive and secure customer information and data.
In an embodiment, each of the steps of method 500 is a distinct step. In another embodiment, although depicted as distinct steps in
In an embodiment, a heap dump consists of data related to the objects of a software application that were present in the memory (e.g., on the heap) of a system running the software at a specific time (e.g., at the time the heap dump was created).
Example 550 is a view of the heap dump displayed by an analysis tool as a categorized readout. In the current embodiment, Example View 550 presents the heap dump information is categorized according to object class type, object instances, object elements, element data types, element values and location of values in memory.
Class name 552 is the name of the class the objects of Example View 550 belong to. In an embodiment, Class name 552 is displayed using a notation that denotes the data type and chain of inheritance of the named class.
Number of instances 554 is the sum of all instantiated objects of the currently displayed class type that are contained within the heap dump. Instance size 556 is the size, in bytes, of an instantiated object of the current class type. Total size of instances 558 is the product of Number of instances 504 and Instance size 506. Instance index header 560 is a header for identifying a list of viewable instances of the currently viewed class type. Instance field header 562 is a header for identifying a category of elements that may be contained within an instantiated object of the currently viewed class type. Variable name index header 564 is a header for indentifying an index of names for variables contained within an instantiated object of the currently viewed class type. Data type index header 556 is a header for indentifying an index of data types associated with the elements (e.g., variables) contained within an instantiated object of the currently viewed class type. Value index header 568 is a header for indentifying an index of values associated with the variables contained within an instantiated object of the currently viewed class type. Address index header 570 is a header for indentifying an index of addresses, in memory, associated with the elements (e.g., variables) contained within an instantiated object of the currently viewed class type. Selected instance indicator 572 indicates which instance within instance index 524a-n (discussed below) is currently displayed. Instance index 574a-n is a list of all instantiated objects of the currently displayed class type. In an embodiment, when a particular instance within instance index 574a-n is selected, Example View 550 is updated to display the information within the heap dump that is associated with the selected instance. Variable name index 576a-m is a list of all the named (e.g., defined) elements of the currently viewed instance. Data type index 578a-1 is a list of the data types associated with each of the items of variable name index 576a-m. Variable value index 580a-o is a list of the values associated with each of the items of variable name index 576a-m. Sensitive values 580b, 580e and 580f are examples of variables that contain sensitive information that needs to be scrubbed. In this example, 580b is the tenant name, 580e is the tenant billing address, and 580f is a tenant credit card number. Address value index 582a-p is a list of the memory addresses associated with each of the items of variable value index 580a-o.
In an embodiment, a map outlining the relationships and hierarchy of the objects within a dump can be constructed from the information shown by the analysis tool. In an embodiment, this map is created automatically by the analysis tool, in other embodiments the map could be generated manually or by a separate program.
In other embodiments, there may be more or less fields displaying different attributes and/or elements of objects within the heap dump. In other embodiments, there might be more information windows within the analysis tool displaying more information specific to the heap dump or the specific object selected.
In an embodiment, the only difference between view 550 and 590 is that view 550 includes the sensitive information and in view 590 the sensitive information has been replaced with dummy values. Consequently, although not labeled, as such, view 590 also contains object class name 552, number of instances 554, instance size 556, total size of instances 558, instance index header 560, instance field header 562, variable name index header 564, data type index header 566, value index header 568, address index header 570, selected instance indicator 572, instance index 574a-n, variable name index 576a-m, data type index 578a-1, variable value index 580a-o, which includes sensitive values 580b, 580e, and 580f, and address value index 582a-p, which were discussed above in conjunction with
Scrubbed variables 590b, 590e and 590f are examples of variables that contain dummy information as a result of being scrubbed. In this example, scrubbed values 590b, 590e and 590f correspond to sensitive values 580b, 580e, and 580f, which prior to scrubbing contained the tenant name, the tenant billing address, and tenant credit card number. Although in the example the tenant names, addresses, and credit card information are shown as being scrubbed, other types of sensitive information may be scrubbed as well, such as the tenant's banking information. In other embodiments, such as for a small company, the tenant's name and address might not be scrubbed, because the tenant's name and address are likely public information and at least in a small company may already be known to the employees.
Method for Creating the Environment (
In step 704, system 16 (
In step 706, user system 12 is communicatively coupled to network 104. In step 708, system 16 is communicatively coupled to network 104 allowing user system 12 and system 16 to communicate with one another (
In another embodiment, although depicted as distinct steps in
Extensions and Alternatives
Regarding determining dependencies of objects in the heap dump, in an embodiment, while in a development environment, for example, the information about the object may be generated as part of a diagnostic routine and placed into the heap dump. For example, the diagnostic routine may cause a label to be generated anytime a particular class, a variable and/or method is invoked. The label may be stored in memory in association with the data generated from the class and/or method invoked. Then when a heap dump is generated the label is dumped with the rest of the data, labeling the data. In another embodiment, as source code is read by a compiler, a map of the objects and the dependency of each object on other objects is constructed and stored, based on the syntax of the source code. The reading of the source code to establish the class dependencies may be performed in addition to or instead of the labeling of the objects of the program. The reading of the source code to establish the class dependencies and/or the labeling of the objects of the program may be performed in addition to or instead of the other methods in this specification.
Each embodiment disclosed herein may be used or otherwise combined with any of the other embodiments disclosed. Any element of any embodiment may be used in any embodiment.
While the invention has been described by way of example and in terms of the specific embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements as would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.
This application claims the benefit of U.S. Provisional Patent Application 61/343,854 entitled METHOD AND SYSTEM FOR SCRUBBING INFORMATION, by Fiaz Hossain et al., filed May 4, 2010, the entire contents of which are incorporated herein by reference. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. The following commonly owned, co-pending or other U.S. patents and patent applications, including the present application, are related to each other. Each of the other patents/applications listed below are incorporated by reference herein in its entirety: U.S. Provisional Patent Application 61/343,854 entitled METHOD AND SYSTEM FOR SCRUBBING INFORMATION, by Fiaz Hossain et al., filed May 4, 2010 and U.S. Utility patent application Ser. No. 13/097,027 entitled METHOD AND SYSTEM FOR SCRUBBING INFORMATION, by Fiaz Hossain et al., filed Apr. 28, 2011.
Number | Name | Date | Kind |
---|---|---|---|
5715450 | Ambrose et al. | Feb 1998 | A |
5821937 | Tonelli et al. | Oct 1998 | A |
5831610 | Tonelli et al. | Nov 1998 | A |
5873096 | Lim et al. | Feb 1999 | A |
5963953 | Cram et al. | Oct 1999 | A |
6092083 | Brodersen et al. | Jul 2000 | A |
6154748 | Gupta et al. | Nov 2000 | A |
6169534 | Raffel et al. | Jan 2001 | B1 |
6178425 | Brodersen et al. | Jan 2001 | B1 |
6189011 | Lim et al. | Feb 2001 | B1 |
6216135 | Brodersen et al. | Apr 2001 | B1 |
6233617 | Rothwein et al. | May 2001 | B1 |
6266669 | Brodersen et al. | Jul 2001 | B1 |
6324568 | Diec | Nov 2001 | B1 |
6324693 | Brodersen et al. | Nov 2001 | B1 |
6336137 | Lee et al. | Jan 2002 | B1 |
D454139 | Feldcamp | Mar 2002 | S |
6367077 | Brodersen et al. | Apr 2002 | B1 |
6393605 | Loomans | May 2002 | B1 |
6405220 | Brodersen et al. | Jun 2002 | B1 |
6434550 | Warner et al. | Aug 2002 | B1 |
6446089 | Brodersen et al. | Sep 2002 | B1 |
6549908 | Loomans | Apr 2003 | B1 |
6553563 | Ambrose et al. | Apr 2003 | B2 |
6574635 | Stauber et al. | Jun 2003 | B2 |
6577726 | Huang et al. | Jun 2003 | B1 |
6604117 | Lim et al. | Aug 2003 | B2 |
6604128 | Diec | Aug 2003 | B2 |
6609150 | Lee et al. | Aug 2003 | B2 |
6665648 | Brodersen et al. | Dec 2003 | B2 |
6665655 | Warner et al. | Dec 2003 | B1 |
6684438 | Brodersen et al. | Feb 2004 | B2 |
6711565 | Subramaniam et al. | Mar 2004 | B1 |
6724399 | Katchour et al. | Apr 2004 | B1 |
6728702 | Subramaniam et al. | Apr 2004 | B1 |
6728960 | Loomans | Apr 2004 | B1 |
6732095 | Warshavsky et al. | May 2004 | B1 |
6732100 | Brodersen et al. | May 2004 | B1 |
6732111 | Brodersen et al. | May 2004 | B2 |
6754681 | Brodersen et al. | Jun 2004 | B2 |
6763351 | Subramaniam et al. | Jul 2004 | B1 |
6768904 | Kim | Jul 2004 | B2 |
6782383 | Subramaniam et al. | Aug 2004 | B2 |
6804330 | Jones et al. | Oct 2004 | B1 |
6826745 | Coker et al. | Nov 2004 | B2 |
6829655 | Huang et al. | Dec 2004 | B1 |
6842748 | Warner et al. | Jan 2005 | B1 |
6850895 | Brodersen et al. | Feb 2005 | B2 |
6850949 | Warner et al. | Feb 2005 | B2 |
7209929 | Dominguez, Jr. et al. | Apr 2007 | B2 |
7721328 | Boulos et al. | May 2010 | B2 |
7802007 | Reese | Sep 2010 | B2 |
8069153 | Chan et al. | Nov 2011 | B2 |
8108919 | Boulos et al. | Jan 2012 | B2 |
8156085 | Dominguez, Jr. et al. | Apr 2012 | B2 |
8453196 | Lerner et al. | May 2013 | B2 |
8453201 | Lerner et al. | May 2013 | B2 |
8453202 | Lerner et al. | May 2013 | B2 |
8453203 | Lerner et al. | May 2013 | B2 |
20010044791 | Richter et al. | Nov 2001 | A1 |
20020022986 | Coker et al. | Feb 2002 | A1 |
20020029161 | Brodersen et al. | Mar 2002 | A1 |
20020029376 | Ambrose et al. | Mar 2002 | A1 |
20020035577 | Brodersen et al. | Mar 2002 | A1 |
20020042264 | Kim | Apr 2002 | A1 |
20020042843 | Diec | Apr 2002 | A1 |
20020072951 | Lee et al. | Jun 2002 | A1 |
20020082892 | Raffel et al. | Jun 2002 | A1 |
20020129352 | Brodersen et al. | Sep 2002 | A1 |
20020140731 | Subramaniam et al. | Oct 2002 | A1 |
20020143997 | Huang et al. | Oct 2002 | A1 |
20020152102 | Brodersen et al. | Oct 2002 | A1 |
20020161734 | Stauber et al. | Oct 2002 | A1 |
20020162090 | Parnell et al. | Oct 2002 | A1 |
20020165742 | Robins | Nov 2002 | A1 |
20030004971 | Gong et al. | Jan 2003 | A1 |
20030018705 | Chen et al. | Jan 2003 | A1 |
20030018830 | Chen et al. | Jan 2003 | A1 |
20030066031 | Laane | Apr 2003 | A1 |
20030066032 | Ramachandran et al. | Apr 2003 | A1 |
20030069936 | Warner et al. | Apr 2003 | A1 |
20030070000 | Coker et al. | Apr 2003 | A1 |
20030070004 | Mukundan et al. | Apr 2003 | A1 |
20030070005 | Mukundan et al. | Apr 2003 | A1 |
20030074418 | Coker | Apr 2003 | A1 |
20030088545 | Subramaniam et al. | May 2003 | A1 |
20030120675 | Stauber et al. | Jun 2003 | A1 |
20030151633 | George et al. | Aug 2003 | A1 |
20030159136 | Huang et al. | Aug 2003 | A1 |
20030187921 | Diec | Oct 2003 | A1 |
20030189600 | Gune et al. | Oct 2003 | A1 |
20030191743 | Brodersen et al. | Oct 2003 | A1 |
20030204427 | Gune et al. | Oct 2003 | A1 |
20030206192 | Chen et al. | Nov 2003 | A1 |
20030225730 | Warner et al. | Dec 2003 | A1 |
20040001092 | Rothwein et al. | Jan 2004 | A1 |
20040010489 | Rio | Jan 2004 | A1 |
20040015981 | Coker et al. | Jan 2004 | A1 |
20040027388 | Berg et al. | Feb 2004 | A1 |
20040128001 | Levin et al. | Jul 2004 | A1 |
20040186860 | Lee et al. | Sep 2004 | A1 |
20040193510 | Catahan, Jr. et al. | Sep 2004 | A1 |
20040199489 | Barnes-Leon et al. | Oct 2004 | A1 |
20040199536 | Barnes Leon et al. | Oct 2004 | A1 |
20040199543 | Braud et al. | Oct 2004 | A1 |
20040210909 | Dominguez, Jr. et al. | Oct 2004 | A1 |
20040249854 | Barnes-Leon et al. | Dec 2004 | A1 |
20040260534 | Pak et al. | Dec 2004 | A1 |
20040260659 | Chan et al. | Dec 2004 | A1 |
20040268299 | Lei et al. | Dec 2004 | A1 |
20050050555 | Exley et al. | Mar 2005 | A1 |
20050091098 | Brodersen et al. | Apr 2005 | A1 |
20050201378 | Ludwig et al. | Sep 2005 | A1 |
20070130130 | Chan et al. | Jun 2007 | A1 |
20070288510 | Dominguez et al. | Dec 2007 | A1 |
20080126301 | Bank et al. | May 2008 | A1 |
20090282036 | Fedtke | Nov 2009 | A1 |
20100077468 | Pragides et al. | Mar 2010 | A1 |
20100192067 | Casalaina | Jul 2010 | A1 |
20100192204 | Boulos et al. | Jul 2010 | A1 |
20100281515 | Lerner et al. | Nov 2010 | A1 |
20110060842 | Reese | Mar 2011 | A1 |
20110083122 | Chen et al. | Apr 2011 | A1 |
20110131314 | Lerner et al. | Jun 2011 | A1 |
20110225232 | Casalaina et al. | Sep 2011 | A1 |
20110225233 | Casalaina et al. | Sep 2011 | A1 |
20110225263 | Sureshchandra et al. | Sep 2011 | A1 |
20110225495 | Casalaina et al. | Sep 2011 | A1 |
20110225500 | Casalaina et al. | Sep 2011 | A1 |
20110225506 | Casalaina et al. | Sep 2011 | A1 |
20110231835 | Smith et al. | Sep 2011 | A1 |
20110238622 | Walters et al. | Sep 2011 | A1 |
20110238760 | Kuruganti et al. | Sep 2011 | A1 |
20110246417 | Maya et al. | Oct 2011 | A1 |
20110247051 | Bulumulla et al. | Oct 2011 | A1 |
20110247066 | Lee | Oct 2011 | A1 |
20110247069 | Slater | Oct 2011 | A1 |
20110258225 | Taylor et al. | Oct 2011 | A1 |
20110258233 | Unger et al. | Oct 2011 | A1 |
20110258628 | Devadhar | Oct 2011 | A1 |
20110264861 | Fee et al. | Oct 2011 | A1 |
20110265066 | Fee et al. | Oct 2011 | A1 |
20110282899 | Mathew et al. | Nov 2011 | A1 |
20110282908 | Fly et al. | Nov 2011 | A1 |
20110283163 | Zheng | Nov 2011 | A1 |
20110283266 | Gallagher et al. | Nov 2011 | A1 |
20110283356 | Fly et al. | Nov 2011 | A1 |
20110289356 | Hossain et al. | Nov 2011 | A1 |
20110302298 | Lawrance et al. | Dec 2011 | A1 |
20110307695 | Slater | Dec 2011 | A1 |
20110321150 | Gluck | Dec 2011 | A1 |
20110321175 | Slater | Dec 2011 | A1 |
20120005537 | Chen et al. | Jan 2012 | A1 |
20120030514 | Lawrance et al. | Feb 2012 | A1 |
20120041921 | Canaday et al. | Feb 2012 | A1 |
20120042383 | Greene et al. | Feb 2012 | A1 |
20120042384 | Badhwar et al. | Feb 2012 | A1 |
20120047489 | Varadharajan | Feb 2012 | A1 |
20120047570 | Chan et al. | Feb 2012 | A1 |
20120059853 | Jagota | Mar 2012 | A1 |
20120059925 | Lerner et al. | Mar 2012 | A1 |
20120059933 | Lerner et al. | Mar 2012 | A1 |
20120060199 | Lerner et al. | Mar 2012 | A1 |
20120060200 | Lerner et al. | Mar 2012 | A1 |
20120066755 | Peddada et al. | Mar 2012 | A1 |
20120072783 | Li et al. | Mar 2012 | A1 |
20120079560 | Lerner et al. | Mar 2012 | A1 |
20120096521 | Peddada | Apr 2012 | A1 |
20120096533 | Boulos et al. | Apr 2012 | A1 |
20120096534 | Boulos et al. | Apr 2012 | A1 |
20120117217 | Lerner et al. | May 2012 | A1 |
20120117613 | Lerner et al. | May 2012 | A1 |
20120130973 | Tamm et al. | May 2012 | A1 |
20120131555 | Hossain et al. | May 2012 | A1 |
20120136602 | Hossain et al. | May 2012 | A1 |
20120140923 | Lee et al. | Jun 2012 | A1 |
20120144374 | Gallagher et al. | Jun 2012 | A1 |
20120173485 | Kothule et al. | Jul 2012 | A1 |
20120239795 | Lerner et al. | Sep 2012 | A1 |
20120240188 | Lerner et al. | Sep 2012 | A1 |
20120240189 | Lerner et al. | Sep 2012 | A1 |
20120240190 | Lerner et al. | Sep 2012 | A1 |
20120259839 | Chan et al. | Oct 2012 | A1 |
20120259894 | Varley et al. | Oct 2012 | A1 |
20120260341 | Chan et al. | Oct 2012 | A1 |
20120330926 | Rajan et al. | Dec 2012 | A1 |
20130007049 | Ziemann | Jan 2013 | A1 |
20130014211 | Boulos et al. | Jan 2013 | A1 |
20130014230 | Boulos et al. | Jan 2013 | A1 |
20130019106 | Fischer | Jan 2013 | A1 |
20130024788 | Olsen et al. | Jan 2013 | A1 |
20130031053 | Tobin et al. | Jan 2013 | A1 |
20130031054 | Tobin et al. | Jan 2013 | A1 |
20130031061 | Jagota | Jan 2013 | A1 |
20130031141 | Tobin et al. | Jan 2013 | A1 |
20130031172 | Olsen et al. | Jan 2013 | A1 |
20130031491 | Tobin et al. | Jan 2013 | A1 |
20130031555 | Tobin et al. | Jan 2013 | A1 |
20130031613 | Shanabrook et al. | Jan 2013 | A1 |
20130036406 | Zheng et al. | Feb 2013 | A1 |
20130036407 | Zheng et al. | Feb 2013 | A1 |
20130054714 | Bedi | Feb 2013 | A1 |
20130054968 | Gupta | Feb 2013 | A1 |
20130060859 | Olsen et al. | Mar 2013 | A1 |
20130097253 | Mencke | Apr 2013 | A1 |
20130097713 | White | Apr 2013 | A1 |
20130117287 | Jagota et al. | May 2013 | A1 |
Entry |
---|
Castro et al, Better Bug Reporting With Better Privacy, Mar. 5, 2008, Microsoft Research Cambridge, pp. 1-10. |
Number | Date | Country | |
---|---|---|---|
20110276610 A1 | Nov 2011 | US |
Number | Date | Country | |
---|---|---|---|
61343854 | May 2010 | US |