Patent Grant
10678757
Certain databases, such as non-relational databases that run on top of distributed file systems, may be rendered unavailable from time to time because of processes such as data splitting, merging, or compaction, or emergency events that disrupt network activity or database components. These databases may rely on unique identifiers to identify stored events. When such databases are unavailable, there may be no capability for generating identifiers for events that are received to be written to the databases. Events awaiting identifiers may be stored in memory; however there may be a high volume of write activity that exceeds memory capacity. As a result, events may be dropped from memory before identifiers for the events can be generated.
The accompanying drawings, which are included to provide a further understanding of the disclosure, are incorporated in and constitute a part of this specification. The drawings also illustrate implementations of the disclosure and together with the detailed description serve to explain the principles of implementations of the disclosure. No attempt is made to show structural details in more detail than may be necessary for a fundamental understanding of the disclosure and various ways in which it may be practiced.
To address issues in distributed database systems such as dropping events before identifiers may be generated, techniques may be provided for reliably generating identifiers for events temporarily stored in low-latency, supplementary data stores. For example, in an implementation of this disclosure, a database system may include a server hosting an application that may receive application data from a client device that is interacting with the application. For instance, an emergency response organization may maintain an application server that provides an application to residents and businesses in a large metropolitan region. The residents and businesses may interact with the application via mobile devices and other computer systems to send data to report emergency circumstances. The application server may generate an event to be written to a database based on the received data. For example, a large earthquake may have hit the region. An event representing a fire at an office complex may be generated based on received data. The received data may include data from three different employees calling the emergency responders via mobile devices running the application, and smoke detector and sprinkler system sensors providing data indicating their sensor measurements to an office computer system running the application. The event may also include the time the data was received, location of the devices that provided the data, or other relevant emergency information. It may be beneficial to store such events in the database because analytical procedures may be performed on data from the database to optimize fire response resources. During emergency events such as earthquakes in densely populated cities, fire response resources may be especially scarce due to the high demand of emergency responders.
A component of the database system, such as an identifier-generating component, may execute on the application server and may attempt to access an event storage database to generate an identifier that uniquely identifies the fire event. This identifier may then be assigned to the event and the event may be written to the distributed data store on which the database is implemented. However, because of the earthquake, the network connection to the data store may be significantly reduced. Moreover, due to the high population of the region, the application server may be inundated with a high volume of emergency data and thus generate a high volume of events to be written to the database. Thus the database may be unresponsive for a certain period of time, for example, at least 16 milliseconds. As a result, after being unresponsive for 16 milliseconds, the database may be deemed unavailable.
A supplementary data store may maintain a low-latency, high throughput, supplementary file system on a data store in communication with the application server. In response to determining the database is unavailable, a batch of events including the fire event may be written as an event file to the supplementary file system. An indirect-identifier-generating component may periodically determine the next event or event file to receive an identifier and/or be written to the database. For example, the indirect-identifier-generating component may periodically access a metadata table stored at the database that maintains an ordered log of events that have not successfully received identifiers or been written to the database. The indirect-identifier-generating component may retrieve attributes from the attribute metadata table that may be used to determine the next event or event file in the supplementary file system to receive an identifier or set of identifiers.
The indirect-identifier-generating component may, based on the retrieved attributes, read the next event file determined to receive identifiers from the supplementary file system and write the event file to a memory component of the application server. The next event file may include the fire event. Once all events from the event file are written to the memory component, the event file may be deleted from the supplementary file system.
It may be beneficial to delete the event file because it may prevent the possibility generating multiple identifiers for the same event, as in the case where the indirect-identifier-generating component is interrupted or halted after it completes processing the event file but before deleting the file from the supplementary file system. The indirect-identifier-generating component may then access the database, generate a unique identifier, and assign the identifier to each event in the event file, including the fire event.
In some circumstances, after assigning the identifier, the database may once again become unavailable. For example, in some implementations, the database system may determine the database is not responding for a certain period of time and as a result determine the database is unavailable, as described above. As a consequence, a batch of events including the fire event with its assigned identifier may be written as a new file to the supplementary file system. An indirect-event-writing component of the database system may periodically attempt to access the database to write events. The indirect-event-writing component may read the fire event from the supplementary file system, and once the database is available, write the event to the database. Once all the events in the new file are written to the database, the indirect-event-writing component may delete the new file from the memory component of the application server and/or the supplementary file system.
Component Environment
Implementations of this disclosure may be executed in a variety of component environments. For example,
Application servers 100, 101 and client devices 130, 131 may be connected through a series of one or more networks such as network 800 described below with respect to
In some embodiments application servers 100, 101 may be multi-tenant application servers designed to provide access to applications for multiple distinct sets of users on one or more client devices, such as client devices 130, 131. A tenant, may be an organization, such as emergency responders or a private business, or portion of an organization, such as the fire fighters or the sales department. More generally, a tenant may be any group of users who share access with specific privileges to a software instance such as the software instance of an application provided by application server 100 or a database. In other embodiments, application servers 100, 101 may be single-tenant application servers designed to service a single set of users on one or more client devices, such as client devices 130, 131. Application servers 100, 101 may provide resources that are made available, continuously or as needed, to client devices 130, 131, such as data processing, data storage, data retrieval, application-specific functionality, and so forth.
Application servers 100, 101 may include any computing devices suitable for the purposes of this disclosure, such as computing device 700 described below with respect to
Client devices 130, 131 may include any computing devices suitable for interacting with applications provided by application servers 100,101, such as computing device 700 described below with respect to
Application servers 100, 101 may include various components. For example, according to an embodiment of the disclosure, application server 100 may include an event-generating component 102; a buffer component 103, which may include a memory component 104 and an event-transfer component 105; event-writing component 106, which may include identifier-generating component 107; indirect-identifier-generating component 108; and indirect-event-writing component 109. However, some or all of these components may be included and execute, in part or in whole, on other components of the database system, such as data store 114 or data store 123.
The event generating component 102 may be a procedure specific to a particular application executing on application server 100 or it may be a service shared across multiple applications executing on application server 102. Generally, the event generating component 102 may receive data from client devices 130,131 and generate events in accordance with conditions, policies, or other data-response procedures specified by the application being provided by application server 100. The events may then eventually be written to a database, such as database 113, for later access and analytics.
An event may be a sequence of one or more values or elements, such as a tuple, which represents something that happened. Events may be standardized across multiple applications executing on an application server, such as application server 100, or customized for a specific application. For example many different applications may have an event that represents a user logging in to the application on a client device; however a sales analysis application may not have an event representing the detection of a fire. An event can be immutable, and may be timestamped or otherwise assigned a sequence indicator as it is generated. An event can refer to data that characterizes an action at a client device, such as client device 130, that causes an interaction with an application being provided by an application server, such as application server 100. Events may be monitored, logged, and recorded for storage and/or retrieval for analysis in near-real time or at a later time.
In an implementation such as described above, the application server 100 may provide an application for emergency responders to client devices associated with an office complex. Application server 100 may receive data from client devices such as data from smart phones of three employees of the office complex indicating a fire, data from a building safety system of the office complex indicating that four smoke alarms are detecting smoke and fifteen fire suppression devices are emitting fire suppression chemicals, and data indicating the location and time of the data collection. Based on this received application data, the event generating component 102 of application server 100 may generate an event that represents a fire has been detected at the office complex.
In another example, events related to user actions may be logged and later analyzed to characterize user behavior. For instance, user actions such as the use of applications, windows opened, system commands executed, check boxes clicked, text entered and/or edited, network locations visited, and so forth may be captured for later analysis.
When an application executing on application server 100 generates an event, it may store the event in memory. In some implementations, events may be received by buffer component 103. Buffer component 103 may include various components such as memory component 104 and event-transfer component 105. As shown in
In some implementations, buffer component 103 may include a buffer service that stores, retrieves, and forwards objects, such as events. The buffer service may be designed to provide a very low latency application program interface (API) to write events to data store 114. For example, the buffer service may write events to database 113 in 0.08 microseconds. The buffer service may keep events in memory until the number of events reaches a configured event number limit and/or for a configured time, for example 10 seconds.
In some implementations, the buffer service can include an event transfer component 105, which may include a capture service that receives events generated by an application executing on application server 100. Buffer component 103 may contain memory component 104, which may include a buffer service that may store captured events in an in-memory bounded buffer. The event transfer component 105 may also include a buffer flush regulator that may manage the transfer of events from the in-memory bounded buffer. The Memory component 104 may include any combination of hardware and software suitable for the purposes of the disclosure, such as for example memory 770 described below with respect to
The in-memory bounded buffer of memory component 104 may support a concurrently writable API, which stores events in memory component 104 of application server 100. The in-memory bounded buffer may include multiple single-threaded segments, and the capture service of event transfer component 105 may be able write events to or read events from each segment in parallel. When a number of events stored in the in-memory bounded buffer reaches a predefined limit, the in-memory bounded buffer may provide a notification to the buffer flush regulator of the event-transfer component 105.
In some implementations, when the buffer flush regulator receives the notification, it may cause the application server 100 to read a set of associated events, such as an event file, from the in-memory bounded buffer and provide the event file to event-writing component 106 of the data store 114 to be written to database 113. A set of items, such as events or a file, may include a single item, multiple items, or no items. Event-writing component 106 may include an executor service that employs a dynamically sized thread pool of execution threads to read the segments of the bounded buffer in parallel to allow the events to be written directly to the data store. Events can be consumed in parallel fashion to allow for high throughput.
The executor service of event-writing component 106 may receive the file and write the events in the file directly to database 113, that is write the events to the database without storing the events in intermediary memory or other storage. In some implementations, the executor service may request a file from the in-memory bounded buffer to be written to database 113. Although,
In some implementations, before an event in an event file may be written to database 113, an identifier may be generated and assigned to the event. For example, in addition to writing events, the executor service of event-writing component 106 may also include identifier-generating component 107. The identifier-generating component may include execution threads that access other components of application server 100 or database 113 to generate and assign identifiers to events. These execution threads may be included in dynamic thread pool of event-writing component 106.
An identifier may enable location, sorting, and retrieval of an event from database 113 because the identifier may be unique amongst all identifiers assigned to events in database 113. A key may be an identifier for an event, and may be generated in any suitable form, such as, for example, an alphanumeric sequence. Portions of the key may provide information about the event. For example, a portion of the key may be a tenant identifier, which may uniquely identify the tenant to whom an event belongs. Other portions of the key may be values that identify, for example, a table number, row, column, or index value of database 113.
In some implementations, an identifier may also include sequence indicator, which may be a number or set of ordered symbols received from a strictly increasing or decreasing counter implemented on application server 100. Each time the counter generates a sequence indicator, the value of the counter may increment such that no identical sequence indicators are generated and each sequence indicator corresponds to the order in which it is generated. A sequence indicator may be a time or approximate the time when an identifier including the sequence indicator is generated and/or assigned to an event. For example as sequence indicator may be a time stamp received from a clock implemented on application server 100. The rate at which the counter increments may correspond to the clock, or values of the counter may be periodically synchronized to values of the clock. In some implementations the database system may access the clock or counter of application server 100 or data store 114 when generating assigning identifiers to events.
In some implementations, the generation of identifiers requires access to database 113. For example, the identifier to be generated may specify particular rows, columns, indexes, or other values of a table associated with a tenant's database 113. Thus, the applicable table of database 113 may need to be analyzed to determine which particular row, column, index, other value, or range of such values, are available within the table to receive an event. As a result, when the database system generates an identifier for an event, identifier-generating component 107 may first access database 113 to determine the needed values for the identifier.
If database 113 is not available, such as during periodic maintenance or when there is network interruption due to an emergency event, it may be prohibitively slow or impossible to generate identifiers. Database 113 may be unavailable for up to an hour or longer, but even a few minutes of unavailability may create significant memory overflow risks in database systems with a high volume of write activity. In implementations where all identifier-generator functionality resides on data store 114, it may not be possible to obtain sequence indicators such as clock or counter values because such data store resources would not be accessible.
In circumstances where events cannot be written to database 113 because identifiers cannot be generated or because database 113 is not available, memory component 104 of the application server 100 may reach its storage capacity. For example, in circumstances, such as the emergency earthquake event described above, there may be an excessive number of events generated by an emergency response application executing on application server 100. The level of event generation may drastically exceed normal operational levels of the database system such that additional storage capacity is required.
In such circumstances, the database system may determine that an identifier has not been generated for an event, and as a result may write the event instead to a supplementary data store. For example, identifier-generating component 107 may fail to generate an identifier for a preconfigured period of time. For instance the fire event referenced above may fail to have an identifier generated for a preconfigured period of time because database 113 is unavailable due to network interruptions caused by the earthquake. As a result, identifier-generating component 107 may send an indicator to event-transport component 105 that represents that the identifier cannot be generated. In response to receiving this indicator, event-transport component 105 may write the event without an identifier to the first storage component 121 of second file system 120 in data store 123.
In an example implementation, event-transfer component 105 may employ a set of execution threads that correspond to each of the threads of the identifier-generating component 107. A thread of the identifier-generating component 107 may send an indicator to the corresponding thread of the event-transfer component 105 indicating that an identifier was not generated for an event. The event may have been read from a particular segment of the bounded buffer of memory component 104. The thread of the event-transfer component 105 may read the particular event from the segment. A batch of events, including the particular event may be read by the event-transfer component 105 and written as an event file to the first storage component 121 of second file system 120.
A supplementary file system, such as second file system 120 implemented on data store 120 may function to relieve memory overflow of application server 100. The supplementary file system may not be backed by a large transactional data store such as data store 113. Rather file system 120 may operate as a separate file storage system that can store unstructured data as files, such as files that contain arbitrary and possibly unrelated data with little or no ordering, or organizing structure required. For example second file system 120 may be a blobstore that stores binary large object (BLOB) values or blobstore values. As a result, in contrast to database 113 implemented on data store 114, second file system 120 may provide much higher throughput and much lower latency. In some embodiments, the latency of writing to second file system 120 may be less than the latency of writing to the first file system, and the latency of reading from second file system 120 may be less than the latency of reading from memory component 104. Further second file system 120 may be available to write regardless of whether the database 113 is in read-only mode or is otherwise unavailable, which increases reliability of the database system. For example second file system 120 may not provide a read-only mode, so events may be written to second file system 120 at any time.
Second file system 120 may include first storage component 121 and second storage component 122, and either component may include directories or other organizations of data. First storage component 121 may be a directory that stores exclusively events without identifiers. For example, the fire event described above may be written to the first storage component because it was not assigned an identifier. Events with identifiers may be written to second file system 120. In such implementations, the events with identifiers may also be written to the second storage component 122. Second storage component 122 may be a directory of second file system 120 that stores exclusively events with identifiers.
The physical storage of data store 123 may include any type of physical storage suitable for the purposes of this disclosure, such as any suitable combination of memory 770 and fixed storage 740 as described with respect to
In implementations of this disclosure, second file system 120 may store an event or event file for a configurable amount of time, after which the database system may delete the event or event file to allow for additional events or event files to be stored. In some implementations this amount of time may be arbitrarily configured. In other implementations, the amount of time may be configured, for example, based on the rate of expected unavailability of database 113, as well as the rate of expected overflow of application server 100. For example, the amount of time may be configured to be 1 hour, 12 hours, 1 day, 7 days, 2 weeks, or any other selected amount of time suitable for the purposes of this disclosure.
Events stored in second file system 120 may ultimately have identifiers generated and assigned and be written to database 113 on data store 114. For example, application server 100 may include an indirect-identifier-generating component 108. Indirect-identifier-generating component 108 may include a service that periodically sends requests to database 113 to determine whether it is available. If indirect-identifier-generating component 108 receives an indicator from database 113, it may determine that database 113 is available. Indirect-identifier-generating component 108 may request and receive an identifier for an event and assign that identifier to the event. In some implementations, event writing functionality of indirect-identifier-generating component 108 may then attempt to write the event along with its accompanying identifier to database 113. In other implementations, other components of the database system, such as indirect-event-writing component 109 may read the event and attempt to write the event to database 113.
In some implementations, events generated at application server 100 may have had identifiers successfully generated by identifier generating component 107. However, the executor service of event-writing component 106 may be unsuccessful when it attempts to write the event to database 113. In such circumstances, event-transfer component 105 may write the event to second storage component 122 of second file system 120. In such implementations indirect-event-writing component 109 may read the event from second storage component 122 and attempt to write the event to database 113.
If the database system either fails to generate an identifier for an event stored in second file system 120 or fails to write the event to database 113, then the respective processes or the threads executing such functionality may periodically repeat until database 113 is available and the process or thread is completed. For example, indirect-identifier-generating component 108 may include a daemon process that repeatedly reads the first storage component 121 of second file system 120 and attempts to generate and assign an identifier to events stored therein. Similarly, indirect-event-writing component 109 may include a daemon process that repeatedly reads the second storage component 122 of second file system 120 and attempts to write events having identifiers to database 113. Further discussion of indirect-identifier-generating component 108 and indirect-event-writing component 109 is included in later portions of this disclosure.
Database 113 may be implemented on a single data store or distributed data store, such as data store 114, having multiple servers and/or server clusters. The physical storage of data store 114 may include any type of physical storage suitable for the purposes of this disclosure, such as any suitable combination of memory 770 and fixed storage 740 as described with respect to
The database system may store events associated with various combinations of tenants, databases, file systems, and data stores. For example, a set of events associated with a set of tenants may be stored on a set of databases associated with the set of tenants. Each of the set of databases may be integrated with a single file system 110 on data store 114, and data store 114 may be distributed. In other implementations, a set of events associated with a set of tenants may be stored on a single database integrated with a single file system 110 on data store 114, and data store 114 may be distributed. In other implementations, data store 114 may not be distributed, for example, where storage devices of the data store 114 are attached to a common processor. Embodiments of this disclosure may include any of the above, as well as any other combination of tenants, databases, file systems, and data stores suitable for the purposes of this disclosure.
Reliable Identifier Generation
Generating identifiers for events stored in a supplementary file system, such as second file system 120 may require the transmission and receipt of various messages amongst components of the database system. For example,
When the particular event file is being stored to first storage component 121, the database system may generate one or more attributes that allow retrieval of the particular event file. For example, attributes may include various types of metadata such as: a cluster name that uniquely identifies a collection of instances that are grouped as a cluster of application servers, an instance name that uniquely identifies a collection of application servers grouped as an instance, an application server name of a particular application server, a group name that uniquely identifies a group of files that were generated during a particular time window, a file name of the particular file, a job started date that specifies when the particular file was created, a number of events that specifies a number of events for this particular event file, a last updated time that indicates when the particular file was last updated, a number of retry attempts that specifies how many times the database system has attempted to process the particular file, a job type that indicates whether the file is associated with regular storage operations or backup operations, and a status that indicates either running or completed. Attributes can also include other fields such as file size information regarding the file size of the particular file, a time-to-live values, and so forth.
After the particular event file is written to second file system 120, at 220 a component of the database system, such as event-transfer component 105, may write attributes that are associated with each event in the particular event file to a metadata table, which may be stored at database 113. Storage of the attribute metadata table at database 113 may be consistent with the data and metadata persistence requirements of database 113. Although the determination that database 113 was unavailable may have resulted in the storage of the event file in second file system 120, at a later time, database 113 may become available and attributes such as the above may be written to the attribute metadata table at database 113. Such attributes may be used by the database system to determine which files to read from second file system 120. For example, the filename associated with the oldest job started date, along with the relevant application server name may be used to query second file system 120 and retrieve a particular file.
The database system may periodically check database 113 to see if it is available to receive requests from or otherwise provide access to various processes and execution threads of components of the database system. For example, at 230, indirect-identifier-generating component 108 may send to database 113 a request to read a metadata table of attributes to determine the next event or event file for which identifiers may be generated and assigned. The metadata table may be accessed and particular attributes may be returned that include attributes associated with the particular event file.
In addition the request may include accessing an arrangement of values associated with particular rows, columns, indexes, other values of a table associated with a particular tenant, or a range of such values of database 113. The request may also retrieve a sequence indicator, such as from a clock or counter of data store 114. The data or combinations of data returned as a result of the request may include a generated identifier or set of identifiers for each event in the particular event file. In other implementations the returned data may be used by a thread or process executing on application server 100 to generate the identifier or set of identifiers.
At 240, the particular attributes and generated identifiers may be returned to a component of the database system, such as indirect-identifier-generating component 108. At 250 the indirect-identifier-generating component 108 may query the first storage component 121 for event files associated with the particular attributes. The particular event file may be returned as a result of the query at 260. Indirect-identifier-generating component 108 may then assign the generated identifiers to each event in the particular event file, including assigning a particular identifier to the particular event.
In other implementations, the database system may first use the retrieved metadata to select the particular event file from second file system 120 and write the particular event file to memory component 104 of the application server. Once the particular event file is written to the memory component, the database system may then determine whether the database system is available, and/or request identifiers for the particular event file. This procedure may also apply to other procedures set forth herein, such as those procedures discussed with respect to
Operations performed in accordance with procedures set forth in this disclosure, such as those shown in
Implementations of this disclosure may execute a variety of procedures when generating identifiers.
At 310, the database system may generate a particular event based the data received from the client device. For example, application server 100 may combine data representing a fire emergency, data representing the time of the report, and data representing the location of client device 130 into an event, such as a fire event, that represents the fire at the office complex. In another example, data representing the name of the customer, data identifying the sales associate, and data representing the request for sales history may be combined into an event, such as a sales inquiry event that represents the sales associate's sales history inquiry. Such an event resulting from data from a tenant's device may be an event associated with the tenant.
At 320 the database system may determine whether an identifier has been generated for the particular event. For example, the application server may determine if the database 113 has been unavailable for threshold period of time. Database 113 may be associated with the tenant. For example, the database may be owned or controlled by the tenant and contain the tenant's data. If database 113 is available, then at 330 the database system may access the database 113 to receive a particular generated identifier and assign the particular identifier to the particular event. If the first file system is still available at 340, then at 350 the database system may write the particular event to the first file system 110. If first file system 110 is not available at 340 then the procedure may continue as set forth below.
At 320 the database system may determine that an identifier has not been generated for the particular event. For example the application server 100 may determine that a threshold period of time has expired and database 113 has not been successfully accessed to generate an identifier for the event. For instance, database 113 and file system 110 on data store 114 may be unavailable due network interruptions or data store destruction due to an earthquake in Los Angeles. In another example, database 113 may be unavailable due to operational procedures such as splitting, merging, or compaction.
In some implementations, database 113 may be determined to be unavailable when a response to a request by the database system has not been received within a threshold period of time. For example, the database system may send a request to database 113 to return a particular identifier or data representing a particular identifier for the particular record. The particular identifier or data representing a particular identifier may not be received from database 113 within a threshold period of time. At 360 in response to threshold period of time expiring, the database system may write the particular event or a particular event file containing the particular event to a secondary file system, such as second file system 120. Thus for example, the fire event or sales inquiry event may be written to second file system 120.
The particular event may be without an identifier may be written to a specific storage component of second file system 120, such as first storage component 121. First storage component 121 may be, for example, a specific directory of second file system 120. First storage component 121 may be configured to store no events other than events not having assigned identifiers. The directory may have a specific naming pattern that identifies that the directory contains only files which contain events that have not been assigned identifiers and have not been written to database 113. For instance the directory may be named EventsNoID-MMddyyyyHHmm. Second file system 120 may be configured to temporarily store events or event files for subsequent generation and assignment of identifiers.
Actions taken by the database system with respect to the secondary file system, such as second file system 120, may be logged or otherwise recorded in a component of the database system. For example a metadata table, such as described above with respect to
A component of the database system may have access to an indicator of the next event or event file to receive an identifier or be written to database 113. This indicator may be derived from the attribute metadata table. For example, indirect-identifier-generating component 108 may periodically request such an indicator from the attribute metadata table. Such data may also be represented as a sequence of attributes or other indicators of the order events or event files to be addressed by the database system. In other implementations, data indicating the next event or event file to be accessed may be stored in other components of the database system, such as application server 100, or data store 114. In some implementations, the next event or event file to be written to the database 113 may be the next event or event file to receive an identifier or set of identifiers.
Continuing the example, the database system, such as via the indirect-identifier-generating component 108, may access an indicator of the next event or event file to receive an identifier. The indicator may be an attribute associated with the particular event or particular event file. The database system may periodically access second file system 120 and read the next event or next event file based on the attribute retrieved from the attribute metadata table. For example, at a time after the fire event or sales history event had been written to second file system 120, the database system via indirect-identifier-generating component 108 may access the attribute metadata table and determine that the next event to receive an identifier is the fire event or the sales history inquiry event. For instance, the indirect-identifier-generating component 108 may query the attribute metadata table to return the event stored in first component 121 of second file system 120 having the longest duration being stored in second file system 120 or the greatest number of attempts at generating an identifier for the event or writing the event to database 113. Indicators of the fire event or sales history inquiry event, such as their file names, may be returned based on such criteria.
At 370, the indirect-identifier-generating component 108 may read the particular event or particular event file from the first storage component 121 of the second file system 120. For example, the indirect-identifier-generating component 108 may query first storage component 121 for events having the filename of the fire event or the sales history inquiry event, and such events may be returned.
At 380 the particular event or event file may be written to the application server 100, such as to memory component 104. In addition, the attribute metadata table may be updated to indicate the particular event or event file had been read from second file system 120 and was being processed. Thus, for example, the fire event or sales history inquiry event may be written to memory component 104 and the attribute metadata table may be updated accordingly.
A component of the database system, such as the indirect-identifier-generating component 108, may periodically check to see if the database is available to generate an identifier or access data that may serve as a basis for generating the identifier for an event. Thus the indirect-identifier-generating component 108 may send a request to database 113 to provide an identifier or set of identifiers for the particular event or event file. In other implementations, the database system may attempt to generate an identifier for the particular event or event file in response to the particular event or event file being written to memory component 104, without first requesting confirmation that database 113 is available.
If, at 390 database 113 is available, then the identifier may be generated such as above at 330. Thus for example, an identifier may be generated for and assigned to the fire event or the sales history inquiry event. If, at 390, the database system determines an identifier has not been generated for the particular event, then at 395, the request to generate an identifier may be repeated, for example, in accordance with a scheduled retry policy. Thus, for example, if an identifier is not generated and/or assigned to the fire event or the sales history inquiry event, then the database system may repeatedly attempt to generate and assign identifiers to such events.
A variety of techniques may be employed when determining that an identifier has not been generated for an event, the event has not been written to the database, and/or that the database is not available, as is described throughout this disclosure. For example, a threshold period of time may be preconfigured. In another example, a pendency value may be equal to a period of time or may be preconfigured. If a period of time equal to the pendency value or the threshold period of time expires, then the database system may determine that the database is unavailable, that the identifier has otherwise not been generated, and/or an event has not been written to the database.
In other implementations, a pendency value may be based on one or more other threshold periods of time that may be preconfigured or dynamically generated. For example, the database system may attempt to generate a set of identifiers for an event file. A pendency value may still serve as the basis for determining whether the identifier has been generated, but the pendency value may be determined based on other values such as the number of events in the event file, a first threshold period of time that may be maximum amount of time per event, and a second threshold period of time that may be a minimum amount of time per event file. For example, the pendency value may be determined by the database system by selecting the maximum value from amongst the product of the number of events in the event file and the first threshold period of time, and the second threshold period of time. The first threshold period of time may be, for example 16 milliseconds, the number of events in the event file may be, for example, 1000, and the second threshold period of time may be 7000 milliseconds. Such a determination may be represented by the formula:
MAX(MaxWritePerRecordWaitTime*NumberOfEvents,MinWriteWaitTime)
In the above formula, MinWriteWaitTime may be the second threshold period of time, NumberOfEvents may be the number of events in the event file, MaxWritePerRecordWaitTime may be the first threshold period of time, and MAX may be a function that calculates the maximum of the two values.
In other implementations, determinations that an identifier has not been generated for an event, that the event has not been written to a database, and/or that the database is not available may be based on a number of unsuccessful attempts at generating the identifier or set of identifiers or a number of unsuccessful attempts at writing the event or event file to the database. For example, the database system may track the number of unsuccessful attempts at generating an identifier or set of identifiers and compare this number to a threshold value. If the threshold value is exceeded, then the database system may determine that the identifier and/or set of identifiers has not been generated and/or that the database is not available. In another example, the database system may track the number of unsuccessful attempts at writing an event or event file to the database and compare this number to a threshold value. If the threshold value is exceeded, then the database system may determine that the event and/or event file has not been written to the database and/or that the database is not available.
As discussed above, it may be beneficial to prevent the possibility generating multiple identifiers for the same event, such as in the case where the indirect-identifier-generating component 108 is interrupted or halted after it completes processing an event file but before deleting the event file from second file system 120. In such a circumstance, the indirect-identifier-generating component 108 may successfully generate identifiers for each event in a given event file, and after the indirect-identifier-generating component 108 recovers from being interrupted, it may erroneously address the same event file again and generate a second set of identifiers for the same event file.
Reliable Event Writing
In some circumstances, a particular event may have successfully been assigned an identifier but database 113 may be determined to be unavailable when the database system attempts to write the particular event to database 113. For example, identifier-generating component 107 may successfully access database 113 to generate and assign an identifier to an event, but database 113 may become unavailable before event-writing component 106 can write the event to database 113. In such circumstances, implementations of the disclosure may write the event having the identifier to a supplementary file system, for example second storage component 122 of second file system 120 on data store 123.
Writing events from a supplementary file, such as second file system 120, to a database, such as database 113, may require the transmission and receipt of various messages amongst components of the database system. For example,
Instances of second storage component 122 and/or first storage component 121 may be periodically generated by the database system. For example, second storage component 122 and first storage component 121 may each be directories of second file system 120 and new version of each directory may be generated every ten minutes. Repeated generation of new directories may be beneficial to maintain adequate capacity in second file system 120 for receiving and tracking events from application server 100 that result from database 113 being unavailable.
After the event file is written to second file system 120, at 520 a component of the database system, such as event-transfer component 105 or indirect-event-writing component 109, may write attributes that are associated with each event in the particular event file to an attribute metadata table, which may be stored at database 113 on data store 114. Storage of the attribute metadata table at database 113 may be consistent with the data and metadata persistence requirements of database 113. Although the determination that database 113 was unavailable may have resulted in the storage of the particular event file in second file system 120, at a later time, database 113 may become available and attributes such as the above may be written to the attribute metadata table. Such attributes may be used by the database system to determine which files to read from second file system 120. For example, the filename associated with the oldest job started date, along with the relevant application server name may be used to query second file system 120 and retrieve a particular event file.
The database system may periodically check database 113 to see if it is available to receive or otherwise provide access to various processes and execution threads of components of the database system. For example, at 530, indirect-event-writing component 108 may send to database 113 a request to read a metadata table of attributes to determine the next event or event file to be written to database 113. The metadata table may be accessed and particular attributes may be returned that include attributes associated with the particular event file.
At 540, the particular attributes and generated identifiers may be returned to a component of the database system, such as indirect-event-writing component 109. At 550 the indirect-event-writing component 109 may query the second storage component 122 for event files associated with the particular attributes. At 560, an event, or event file including the particular event may be returned and written to memory, such as to memory component 104 of application server 100. At 570, the database system may write the particular event to database 113.
As described above with respect to
A component of the database system may have access to a record of the activity of second file system 120, and may update such a record. For example, indirect-event-writing component 109 may update an attribute metadata table, such as is described with respect to FIG. 1, to indicate that the particular event or event file has been written to second storage component 122. At a later time, the database system may access the attribute metadata table to determine the next event or event file to attempt to write to database 113. For example, indirect-event-writing component 109 may periodically query the attribute metadata table and return an indicator of the next event or event file to attempt to write to database 113. Such access, updating, querying, and other suitable actions with respect indicators of the next event or event file to be written to database 113 may be performed as is elsewhere described with respect to the attribute metadata table or as otherwise is suitable for the purposes of this disclosure.
The indirect-event-writing component 109 may access an indicator of the next event or event file to be written. The indicator may be an attribute associated with the particular event file. At 610, the indirect-event-writing component 109 may read the particular event file from the second storage component 122 of the second file system 120. After reading the particular event or event file from the second storage component, the attribute metadata table may be updated to indicate the particular event or particular event file had been read and is being processed by the database system. At 620 the particular event file may be written to the application server 100, such as to memory component 104.
At 630, a component of the database system, such as the indirect-event-writing component 109, may periodically check to see if the database is available. Thus the indirect-event-writing component 109 may send a request to database 113 to confirm that database 113 is available. Database 113 may send a response indicating it is available, and at 640, indirect-event-writing component 109 may write the particular event file to database 113. In some implementations, the database system may attempt to write the particular event file to database 113 periodically without first requesting confirmation that database 113 is available.
In some circumstances, the database system may determine that database 113 is not available, such not receiving confirmation on availability or that the particular event has been written within a threshold period of time. In such circumstances, the indirect-event-writing component 109 may retry writing the particular event file at 650, for example, in accordance with a scheduled retry policy.
In some circumstances database 113 may have been available the first time the database system attempted to generate an identifier for an event, but may become unavailable when the database system attempts to write the event to database 113. Implementations of this disclosure may process such events having identifiers the same as events that required multiple attempts before receiving identifiers, such as events that had been stored in second file system 120.
Aspects of implementations of this disclosure may make determinations of database unavailability and may execute retry processes. Generally, the techniques described with respect to making any specific determination of database unavailability or executing any specific retry process may be employed with respect to any other determination of database unavailability or execution of any other retry process. In addition, the attribute metadata table described herein may be updated at times and under conditions in addition to those expressed explicitly herein. For example, each time an event or event file is being read from or written to a secondary file system, such as second file system 120, the attribute metadata table may be updated, for example to indicate the current location of the event or event file, status of the event or event file, or to indicate the event or event file is being processed. In general, any procedure or process executed in a specific implementation described in this disclosure may also be executed in other implementations of this disclosure.
The database system may determine whether all events of a set of associated events such as an event file have been written to a database, such as database 113. If all the events in the event file have not been written to the database, then at the next event in the event file may be read from the memory component. For example, the event file may contain an ordered set of event such that the next event is always determined. If all the events in the event file have been written to the database 113, then database system may delete the event file. For example indirect-event-writing component 109 may delete the event file from memory component 104 and/or from second storage component 122. Furthermore, the database system may retrieve the next event or set of associated events. For example, the indirect-event-writing component 109 may read the attribute metadata able, such described above with respect to
Other Components
Implementations of the disclosure may be implemented in and used with a variety of component and network architectures.
The bus 710 allows data communication between the central processor 780 and the memory 770, which may include read-only memory (ROM) or flash memory (neither shown), and random access memory (RAM) (not shown), as previously noted. The RAM is generally the main memory into which the operating system and application programs are loaded. The ROM or flash memory can contain, among other code, the Basic Input-Output system (BIOS) which controls basic hardware operation such as the interaction with peripheral components. Applications resident with the computing device 700 are generally stored on and accessed via a computer-readable medium, such as a hard disk drive (e.g., fixed storage 740), an optical drive, floppy disk, or other storage medium.
The fixed storage 730 may be integral with the computing device 700 or may be separate and accessed through other interfaces. A network interface 790 may provide a direct connection to a remote server via a telephone link, to the Internet via an internet service provider (ISP), or a direct connection to a remote server via a direct network link to the Internet via a POP (point of presence) or other technique. The network interface 790 may provide such connection using wireless techniques, including digital cellular telephone connection, Cellular Digital Packet Data (CDPD) connection, digital satellite data connection or the like. For example, the network interface 790 may allow the computing device to communicate with other computing devices via one or more local, wide-area, or other networks, as shown in
Many other devices or components (not shown) may be connected in a similar manner (e.g., document scanners, digital cameras and so on). Conversely, all of the components shown in
More generally, various implementations of this disclosure may include or be implemented in the form of computer-implemented procedures or processes and apparatuses for practicing those processes. Implementations also may be implemented in the form of a computing device program product having instructions or computer program code containing instructions implemented in non-transitory and/or tangible media, such as floppy diskettes, CD-ROMs, hard drives, USB (universal serial bus) drives, or any other machine readable storage medium, wherein, when the computing device program code is loaded into and executed by a computing device, the computing device becomes an apparatus for practicing implementations of the disclosure.
Implementations also may be implemented in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computing device, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein when the computer program code is loaded into and executed by a computing device, the computing device becomes an apparatus for practicing implementations of the disclosure. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits. In some configurations, a set of computer-readable instructions stored on a computer-readable storage medium may be implemented by a general-purpose processor, which may transform the general-purpose processor or a device containing the general-purpose processor into a special-purpose device configured to implement or carry out the instructions. Implementations may be implemented using hardware that may include a processor, such as a general purpose microprocessor and/or an Application Specific Integrated Circuit (ASIC) that implements all or part of the techniques according to implementations of the disclosure in hardware and/or firmware. The processor may be coupled to memory, such as RAM, ROM, flash memory, a hard disk or any other device capable of storing electronic information. The memory may store instructions adapted to be executed by the processor to perform the techniques according to implementations of the disclosure.
The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit implementations of the disclosure to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The implementations were chosen and described in order to explain the principles of implementations of the disclosure and their practical applications, to thereby enable others skilled in the art to utilize those implementations as well as various implementations with various modifications as may be suited to the particular use contemplated.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3637077 | Cowan | Jan 1972 | A |
| 5577188 | Zhu | Nov 1996 | A |
| 5608872 | Schwartz et al. | Mar 1997 | A |
| 5649104 | Carleton et al. | Jul 1997 | A |
| 5715450 | Ambrose et al. | Feb 1998 | A |
| 5761419 | Schwartz et al. | Jun 1998 | A |
| 5819038 | Carleton et al. | Oct 1998 | A |
| 5821937 | Tonelli et al. | Oct 1998 | A |
| 5831610 | Tonelli et al. | Nov 1998 | A |
| 5873096 | Lim et al. | Feb 1999 | A |
| 5918159 | Fomukong et al. | Jun 1999 | A |
| 5963953 | Cram et al. | Oct 1999 | A |
| 5983227 | Nazem et al. | Nov 1999 | A |
| 6092083 | Brodersen et al. | Jul 2000 | A |
| 6161149 | Achacoso et al. | Dec 2000 | A |
| 6169534 | Raffel et al. | Jan 2001 | B1 |
| 6178425 | Brodersen et al. | Jan 2001 | B1 |
| 6189011 | Lim et al. | Feb 2001 | B1 |
| 6216133 | Masthoff et al. | Apr 2001 | B1 |
| 6216135 | Brodersen et al. | Apr 2001 | B1 |
| 6233617 | Rothwein et al. | May 2001 | B1 |
| 6236978 | Tuzhilin et al. | May 2001 | B1 |
| 6266669 | Brodersen et al. | Jul 2001 | B1 |
| 6288717 | Dunkle et al. | Sep 2001 | B1 |
| 6295530 | Ritchie et al. | Sep 2001 | B1 |
| 6324568 | Diec | Nov 2001 | B1 |
| 6324693 | Brodersen et al. | Nov 2001 | B1 |
| 6336137 | Lee et al. | Jan 2002 | B1 |
| D454139 | Feldcamp | Mar 2002 | S |
| 6367077 | Brodersen et al. | Apr 2002 | B1 |
| 6393605 | Loomans | May 2002 | B1 |
| 6405220 | Brodersen et al. | Jun 2002 | B1 |
| 6411949 | Schaffer et al. | Jun 2002 | B1 |
| 6434550 | Warner et al. | Aug 2002 | B1 |
| 6446089 | Brodersen et al. | Sep 2002 | B1 |
| 6535909 | Rust | Mar 2003 | B1 |
| 6549908 | Loomans | Apr 2003 | B1 |
| 6553563 | Ambrose et al. | Apr 2003 | B2 |
| 6560461 | Fomukong et al. | May 2003 | B1 |
| 6574635 | Stauber et al. | Jun 2003 | B2 |
| 6577726 | Huang et al. | Jun 2003 | B1 |
| 6601087 | Zhu et al. | Jul 2003 | B1 |
| 6604117 | Lim et al. | Aug 2003 | B2 |
| 6604128 | Diec | Aug 2003 | B2 |
| 6609150 | Lee et al. | Aug 2003 | B2 |
| 6621834 | Scherpbier et al. | Sep 2003 | B1 |
| 6654032 | Zhu et al. | Nov 2003 | B1 |
| 6665648 | Brodersen et al. | Dec 2003 | B2 |
| 6665655 | Warner et al. | Dec 2003 | B1 |
| 6684438 | Brodersen et al. | Feb 2004 | B2 |
| 6711565 | Subramaniam et al. | Mar 2004 | B1 |
| 6724399 | Katchour et al. | Apr 2004 | B1 |
| 6728702 | Subramaniam et al. | Apr 2004 | B1 |
| 6728960 | Loomans | Apr 2004 | B1 |
| 6732095 | Warshavsky et al. | May 2004 | B1 |
| 6732100 | Brodersen et al. | May 2004 | B1 |
| 6732111 | Brodersen et al. | May 2004 | B2 |
| 6754681 | Brodersen et al. | Jun 2004 | B2 |
| 6763351 | Subramaniam et al. | Jul 2004 | B1 |
| 6763501 | Zhu et al. | Jul 2004 | B1 |
| 6768904 | Kim | Jul 2004 | B2 |
| 6772229 | Achacoso et al. | Aug 2004 | B1 |
| 6782383 | Subramaniam et al. | Aug 2004 | B2 |
| 6804330 | Jones et al. | Oct 2004 | B1 |
| 6826565 | Ritchie et al. | Nov 2004 | B2 |
| 6826582 | Chatterjee et al. | Nov 2004 | B1 |
| 6826745 | Coker et al. | Nov 2004 | B2 |
| 6829655 | Huang et al. | Dec 2004 | B1 |
| 6842748 | Warner et al. | Jan 2005 | B1 |
| 6850895 | Brodersen et al. | Feb 2005 | B2 |
| 6850949 | Warner et al. | Feb 2005 | B2 |
| 6907566 | Mineiro et al. | Jun 2005 | B1 |
| 7062502 | Kesler et al. | Jun 2006 | B1 |
| 7069231 | Cinarkaya et al. | Jun 2006 | B1 |
| 7069497 | Desai et al. | Jun 2006 | B1 |
| 7100111 | Mineiro et al. | Aug 2006 | B2 |
| 7181758 | Chan et al. | Feb 2007 | B1 |
| 7269590 | Hull et al. | Sep 2007 | B2 |
| 7289976 | Kihneman et al. | Oct 2007 | B2 |
| 7340411 | Cook | Mar 2008 | B2 |
| 7356482 | Frankland et al. | Apr 2008 | B2 |
| 7373599 | Mineiro et al. | May 2008 | B2 |
| 7401094 | Kesler et al. | Jul 2008 | B1 |
| 7406501 | Szeto et al. | Jul 2008 | B2 |
| 7412455 | Dillon et al. | Aug 2008 | B2 |
| 7454509 | Veilleux et al. | Nov 2008 | B2 |
| 7508789 | Chan et al. | Mar 2009 | B2 |
| 7599935 | La Rotonda et al. | Oct 2009 | B2 |
| 7603331 | Tuzhilin et al. | Oct 2009 | B2 |
| 7603483 | Psounis et al. | Oct 2009 | B2 |
| 7620655 | Larsson et al. | Nov 2009 | B2 |
| 7644122 | Weyer et al. | Jan 2010 | B2 |
| 7668861 | Doliov et al. | Feb 2010 | B2 |
| 7698160 | Beaven et al. | Apr 2010 | B2 |
| 7730478 | Weissman | Jun 2010 | B2 |
| 7747648 | Kraft et al. | Jun 2010 | B1 |
| 7779039 | Weissman et al. | Aug 2010 | B2 |
| 7779475 | Jakobson et al. | Aug 2010 | B2 |
| 7827208 | D'Angelo et al. | Nov 2010 | B2 |
| 7853881 | Aly Assal et al. | Dec 2010 | B1 |
| 7945653 | Zuckerberg et al. | May 2011 | B2 |
| 8005896 | Cheah | Aug 2011 | B2 |
| 8014943 | Jakobson et al. | Sep 2011 | B2 |
| 8015495 | Achacoso et al. | Sep 2011 | B2 |
| 8032297 | Jakobson et al. | Oct 2011 | B2 |
| 8073850 | Steven et al. | Dec 2011 | B1 |
| 8082301 | Ahlgren et al. | Dec 2011 | B2 |
| 8095413 | Beaven et al. | Jan 2012 | B1 |
| 8095531 | Weissman et al. | Jan 2012 | B2 |
| 8095594 | Beaven et al. | Jan 2012 | B2 |
| 8103611 | Tuzhilin et al. | Jan 2012 | B2 |
| 8150913 | Cheah | Apr 2012 | B2 |
| 8185356 | Mukaigawa | May 2012 | B2 |
| 8209308 | Rueben et al. | Jun 2012 | B2 |
| 8209333 | Hubbard et al. | Jun 2012 | B2 |
| 8275836 | Douglas et al. | Sep 2012 | B2 |
| 8457545 | Chan | Jun 2013 | B2 |
| 8484111 | Sturgeon et al. | Jul 2013 | B2 |
| 8490025 | Jakobson et al. | Jul 2013 | B2 |
| 8504945 | Jakobson et al. | Aug 2013 | B2 |
| 8510045 | Rueben et al. | Aug 2013 | B2 |
| 8510664 | Rueben et al. | Aug 2013 | B2 |
| 8566301 | Rueben et al. | Oct 2013 | B2 |
| 8646103 | Jakobson et al. | Feb 2014 | B2 |
| 20010044791 | Richter et al. | Nov 2001 | A1 |
| 20020072951 | Lee et al. | Jun 2002 | A1 |
| 20020082892 | Raffel et al. | Jun 2002 | A1 |
| 20020129352 | Brodersen et al. | Sep 2002 | A1 |
| 20020140731 | Subramaniam et al. | Oct 2002 | A1 |
| 20020143997 | Huang et al. | Oct 2002 | A1 |
| 20020162090 | Parnell et al. | Oct 2002 | A1 |
| 20020165742 | Robins et al. | Nov 2002 | A1 |
| 20030004971 | Gong et al. | Jan 2003 | A1 |
| 20030018705 | Chen et al. | Jan 2003 | A1 |
| 20030018830 | Chen et al. | Jan 2003 | A1 |
| 20030066031 | Laane et al. | Apr 2003 | A1 |
| 20030066032 | Ramachandran et al. | Apr 2003 | A1 |
| 20030069936 | Warner et al. | Apr 2003 | A1 |
| 20030070000 | Coker et al. | Apr 2003 | A1 |
| 20030070004 | Mukundan et al. | Apr 2003 | A1 |
| 20030070005 | Mukundan et al. | Apr 2003 | A1 |
| 20030074418 | Coker et al. | Apr 2003 | A1 |
| 20030120675 | Stauber et al. | Jun 2003 | A1 |
| 20030151633 | George et al. | Aug 2003 | A1 |
| 20030159136 | Huang et al. | Aug 2003 | A1 |
| 20030187921 | Diec | Oct 2003 | A1 |
| 20030189600 | Gune et al. | Oct 2003 | A1 |
| 20030204427 | Gune et al. | Oct 2003 | A1 |
| 20030206192 | Chen et al. | Nov 2003 | A1 |
| 20030225730 | Warner et al. | Dec 2003 | A1 |
| 20040001092 | Rothwein et al. | Jan 2004 | A1 |
| 20040010489 | Rio | Jan 2004 | A1 |
| 20040015981 | Coker et al. | Jan 2004 | A1 |
| 20040027388 | Berg et al. | Feb 2004 | A1 |
| 20040128001 | Levin et al. | Jul 2004 | A1 |
| 20040186860 | Lee et al. | Sep 2004 | A1 |
| 20040193510 | Catahan, Jr. et al. | Sep 2004 | A1 |
| 20040199489 | Barnes-Leon et al. | Oct 2004 | A1 |
| 20040199536 | Barnes Leon et al. | Oct 2004 | A1 |
| 20040199543 | Braud et al. | Oct 2004 | A1 |
| 20040249854 | Barnes-Leon et al. | Dec 2004 | A1 |
| 20040260534 | Pak et al. | Dec 2004 | A1 |
| 20040260659 | Chan et al. | Dec 2004 | A1 |
| 20040268299 | Lei et al. | Dec 2004 | A1 |
| 20050050555 | Exley et al. | Mar 2005 | A1 |
| 20050091098 | Brodersen et al. | Apr 2005 | A1 |
| 20060021019 | Hinton et al. | Jan 2006 | A1 |
| 20080249972 | Dillon et al. | Oct 2008 | A1 |
| 20090063414 | White et al. | Mar 2009 | A1 |
| 20090063415 | Chatfield et al. | Mar 2009 | A1 |
| 20090100342 | Jakobson et al. | Apr 2009 | A1 |
| 20090177744 | Marlow et al. | Jul 2009 | A1 |
| 20110060939 | Singhal et al. | Mar 2011 | A1 |
| 20110247051 | Bulumulla et al. | Oct 2011 | A1 |
| 20110320409 | Iyengar | Dec 2011 | A1 |
| 20120042218 | Cinarkaya et al. | Feb 2012 | A1 |
| 20120218958 | Rangaiah et al. | Aug 2012 | A1 |
| 20120233137 | Jakobson et al. | Sep 2012 | A1 |
| 20120290407 | Hubbard et al. | Nov 2012 | A1 |
| 20130212497 | Zelenko et al. | Aug 2013 | A1 |
| 20130218948 | Jakobson et al. | Aug 2013 | A1 |
| 20130218949 | Jakobson et al. | Aug 2013 | A1 |
| 20130218966 | Jakobson et al. | Aug 2013 | A1 |
| 20130247216 | Cinarkaya et al. | Sep 2013 | A1 |
| 20140359537 | Jackobson et al. | Dec 2014 | A1 |
| 20150215389 | Spencer | Jul 2015 | A1 |
| 20170220596 | Smith | Aug 2017 | A1 |
| 20170279901 | Carricarte | Sep 2017 | A1 |
| Entry |
|---|
| Grover, Mark; “Processing frameworks for Hadoop—How to decide which framework is best for your particular use case” dated Feb. 16, 2015; retrieved from the internet at <http://radar.oreilly.com/2015/02/processingframeworks-for-hadoop.html>; 10 pages. |
| Anonymous: “Multi Tenant Architecture”, Salesforce.com, Mar. 31, 2016 (Mar. 31, 2016), pp. 1-23, XP055373747, Retreived from the internet: URL:https://developer.salesforce.com/. |
| Extended European Search Report dated Jun. 9, 2017 as received in Application No. 17164448.7. |
| European Office Action issued in EP Patent Application No. 17164448.7, dated Aug. 8, 2018, 9 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20170293632 A1 | Oct 2017 | US |
