Patent Application
20090024558
The present disclosure generally relates to the field of data processing and to methods and systems for storing and retrieving rejected data. More specifically, the disclosure relates to methods and systems for storing and retrieving data objects that are rejected after being received from an asynchronous user system, the storing and retrieving being accomplished by, for example, relating stored rejected data objects to a current object format for use in a data system.
The complete documentation of electronic business activities is an important part of many business processes. For some businesses, auditing may be essential or even legally required, and all activities related to data manipulation must be documented. In addition to tracking business activities, documented data can also be used to analyze trends and locate inefficiencies and other problems within the business process.
Data processing and storage can be complex in a backend system that receives data from one or more user systems that are not perfectly synchronized with the backend system. When perfect synchrony between all system elements is not possible, errors and data conflicts can arise.
In general, systems document electronic data after first validating the data. If validation fails, that invalid data is not documented. For example, a second system may receive data from a first system via a user interface or a system interface and, after assessing the validity of the data and/or authorizing the first system, the second system may further process and store the valid data, and document these activities. However, if the data is invalid, out of date, or unauthorized, it may be rejected, preventing adequate documentation. If the first system is connected asynchronously to the second system, the submitted data may be lost, and documentation of the submission, rejection, and reasons for the rejection may not occur. However, such information is necessary for complete audits.
Usually, the second system includes a backend that stores data and performs business logic that is hidden from a front-end user of a user system (e.g., “first system”). In an asynchronous environment, the user system may accept a user's changes, and later submit the data object(s) for storage in the backend. The user system can include one or more client devices. For example, the user system may include a network of computers at a local office branch, or alternatively may comprise a single computer or handheld device, such as a phone or PDA. Because the user system may not be synchronized with the backend system, a data object accepted by the user system may be later rejected before or after being received at the backend.
The user system may access the backend, for example, through a middleware layer that coordinates activities between the front end and the backend. The middleware layer may contain a conflict checking service that analyzes the submitted data object(s) for conflicts with existing backend data before relaying the data object(s) to the backend system. In some cases, the middleware is part of the backend system. The data objects sent through the middleware may define requests for data, modifications to existing data, or submissions of entirely new data, such as a document.
Rejected data objects may not be stored in the backend repository. In some systems, an error message may be sent to the client device to alert the user that the data was not successfully stored. However, the activity embodied in the rejected data object may not be tracked, and therefore will not be revealed in an audit. Additionally, if the user forgets to resubmit the data, or changes the data before resubmitting, the rejected data object itself may be lost forever.
Data object rejections can occur for many reasons, but the two most common reasons are an error in validation and a conflict created by outdated data. A validation error can occur, for example, if an equipment number is misspelled, the data object is missing a required component, or data is corrupted during transmission from the client to the backend. On the other hand, a data conflict can occur when a client manipulates data that has already been fundamentally changed on the backend. For example, in a system utilizing middleware, the data sent from the middleware to the client device may change on the backend, causing the data submitted by the client to be out dated. As a result, when the client device modifies the old data object, a conflict occurs and the submitted data may be rejected instead of stored. In that case, the data object (and/or the documentation regarding the rejected modification of the data object) may be lost.
Without a full picture of the many front-end-to-backend transactions, it may be impossible to fully audit the business processes or determine the cause of lost data. For businesses that are legally required to track particular activities, such as aircraft maintenance, the problem may be even more critical. For example, if a user system accepts a user's changes to a maintenance document but the change is not stored in the backend, the maintenance activity may go undetected. Therefore, the business process is incomplete and the audit will fail.
In accordance with some embodiments presented herein, a method, computer-readable medium, and system are proposed for storing, retrieving, and displaying a rejected data object. Specifically, an embodiment is configured to receive a data object sent from an asynchronous user system and collect the data object if it is rejected from storage on a backend data repository (where accepted data is otherwise stored). Instead of merely sending an error message and ignoring the rejected data object, the rejected data object is documented by converting the rejected data object into a storage format and storing the data on a storage medium. Information relating the rejected data object in the storage format to a first data model is also stored. This relational information allows the rejected data object to be used by the system in the future, by providing an avenue for translating the rejected data object from the storage format into a first object format consistent with the system's data model at that time.
In certain embodiments, the translation of the rejected data object may occur in response to receiving a request for a first data value included in the rejected data object. The request may include information that is utilized to locate the rejected data object. Then, the rejected data object is translated from a storage format into the first object format by using the stored relational information. A value from the translated rejected data object can then be displayed contemporaneously with a second data value from an accepted data object. In one embodiment, the rejected and accepted data is displayed in chronological order. This may facilitate an audit by allowing a user to see transactions and data that might otherwise be lost. This may also permit a system analyst to correct system problems when abnormal trends in the rejected data are observed.
Additional objects and advantages will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the embodiments. The objects and advantages will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several embodiments and, together with the description, serve to explain the principles included herein.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Unless otherwise specified, a “data object,” for the purposes of this disclosure, is any data conforming to a particular format. Examples of data objects include an instance of a class of data, one or more records in a database, and/or at least one data value. The “backend” portion of a system is that part of the system responsible for business logic and data storage, and may or may not include the middleware portion of the system. A “data model” may be thought of as a blueprint that specifies the structure of data objects used by the backend or middleware. Therefore, the format of a useable data object may be dictated by the data model in one embodiment.
As already stated, a user system may be any device or satellite system utilized to send data to and/or from the middleware and/or backend system. For example, a user system may comprise a network of computers that is not directly synchronized with the backend system (for example, when either system is not online) such as a network at a local office branch. In this example, work done on the user system is saved as a data object on the user system. Later, when connectivity to the middleware and/or backend system is established, the user system may send the data object for storage to the middleware and/or backend for storage. In addition, the user system may include an automated component that manipulates data rather than receiving direct input from a user. For example, the automated component may create and submit electronic forms without input from the user.
Past systems may reject data objects based on a detected error or conflict, but do not store or retrieve the rejected data in a format useable by a system like the present embodiments. As a result, past systems cannot display the history of interactions related to a rejected data object, and the lost data (and/or the source of the data loss) may go undetected.
Accordingly, in one embodiment, a conflict resolution service, validation service, and/or documentation engine (“rejection services”) may be provided for collecting rejected data and converting the rejected data into a storage format. For example, the storage format may be generic in nature so that rejected data of nearly any type can be stored by using the format. Along with the rejected data, the rejection services may contemporaneously store information that can be later used for relating specific data values and/or object-oriented relationships within the rejected data object in the storage format to a first object format currently being used on the system. This may allow a user to later perform an audit by retrieving rejected data objects and display them contemporaneously with successfully-stored data objects. Contemporaneously displaying accepted and rejected data objects relating to a particular document can also reveal business information that otherwise would be lost, such as the source of a missing edit, or an undocumented broken part. In addition, the rejected data object can be reconstructed and displayed even if the system now uses new hardware, business logic, and/or a new data model.
Turning now to
The user system 128 may connect to the middleware server 135 over a network, such as the Internet, intranet, local area network, and/or Ethernet. Security measures, such as a virtual private network or encryption, may be used to ensure that the remote transaction is secure. In addition, more than one user system 128 may communicate with middleware server 135 in an embodiment.
The user system 128 may modify data objects stored in the backend 138, such as in backend data repository 175. In some embodiments, the modification is performed locally on the user system 128. For example, the user 105 may open and modify a text document that is copied locally to user system 128 from the backend 138. When the user saves the document, the data object may be accepted by the user system 128. Thereafter, the user system 128 may send the data object to the middleware server 135 to store the change.
The middleware server 135 may be comprised of one or more servers, and includes a processor and a memory. The memory contains instructions that, when executed by the processor, cause the middleware server to operate a conflict resolution service. As used herein, the processor may include multiple processors across one or more servers. The conflict resolution service effectively shields the backend 138 from data objects that conflict with data within backend data repository 175. For example, the conflict resolution service may compare a received data object (such as a document) with an existing version of that data object to determine whether a conflict exists before sending the received data object on to other system services. In addition to the conflict resolution service, the middleware server 135 may be used to periodically re-synchronize data between storages in the backend 138 and user system 128. However, the user system 128 remains asynchronous if it locally persists data.
As shown in
If no rejection occurs, the middleware server 135 passes the data object to the backend 138. In the exemplary system illustrated in
Continuing with
The validity service of validation server 140 may also report the data object to the documentation server 160, which runs a documentation service. The documentation service stores data object information relevant for auditing the system, such as who changed a data object, when the change occurred, what was changed, etc. In one embodiment, the documentation service stores information regarding an accepted data object in a first documentation repository 120.
The first documentation repository 120 may include one or more databases, such as an object-oriented database, a relational database, and/or a flat file, for storing information regarding activities performed on accepted data objects during the normal course of business operations. In addition, the first documentation repository 120 may include one or more servers. The first documentation repository 120 may be included within the backend 138. Alternatively, the first documentation repository may be part of the middleware 132 (or within the middleware server 135), as illustrated in
The first documentation repository 120 may, for example, store only activity information regarding data objects accepted for storage in backend data repository 175. However, in another embodiment (such as in
In some embodiments, the first documentation repository may be part of backend data repository 175. For example, the accepted data object may consist of more than just the file or document to which it pertains. The document may be represented as a data value within a data object, or the data object may contain a pointer to the document. In any cases, the data object may contain other fields or values for auditing purposes, such as file version, a user associated with the file, a timestamp, the type of action last taken on the file, and other information. By tracking this information in the first documentation repository 120, the system can recreate histories of activities and transactions involving accepted data objects.
If, on the other hand, the data object is rejected by the conflict resolution service or the validation service, the rejected data object may be collected and sent to the documentation service for further processing. In one embodiment, the collection occurs on the middleware server 135. In another embodiment, when a backend 138 process such as the validation service is responsible for the rejection, a backend component such as validation server 140 may collect the data object. Alternatively (or in addition), the backend 138 process may report the rejection to the middleware server 135, for example, by sending an error message. The middleware server 135 may then collect the rejected data object. In yet another embodiment, some other server or service within the backend 138 may collect the rejected data. In some embodiments, the middleware server 132 will also relay an error message to the client device 110 or 150 that submitted the data object.
After collecting the rejected data object, the rejected data object is stored on the second documentation repository 130 with information that can later be used to relate the rejected data object in the storage format to a data model used in the backend 138 by backend data repository 175. This may be accomplished in one embodiment by translating the rejected data object into a format recognizable by the documentation service and sending the rejected data object to the documentation service. The documentation service (or some other service) then converts the rejected data object into the storage format and sends the rejected data object to the second documentation repository 130 for storage. The specific storage format can vary between embodiments. However, the middleware server 135 may store information that allows the middleware server 135 to convert the rejected data object from the storage format to a format consistent with a current data model. Although the processing device responsible for this conversion may be outside of the middleware server 135, the outside processing device should still be considered part of the middleware server 135 for the purposes of this disclosure.
In the example of
In this example, the second documentation repository 130 stores rejected data objects in a storage format. The storage format could be a generic flat-file format that allows for storing multiple types of rejected data objects in a consistent manner. However, the storage format may also be object-oriented or relational in nature, depending on the embodiment. The second documentation repository may also contemporaneously store information that allows the middleware server 135 to later construct the data object from the storage format.
By storing the rejected data object in a storage format on the second documentation repository 130, a user 105 may retrieve the rejected data at a later time. In some embodiments, this allows the user 105 to audit the business process by reconstructing a full picture of business activities, including both rejected data from the second documentation repository 130 and accepted data from the first documentation repository 120 and backend data repository 175. The translation process is described more thoroughly with respect to
At some later time, the middleware server 135 may receive a request from a user system 128 for a data value contained in the rejected data object. The middleware server then locates the rejected data object and coverts the rejected data object from the storage format into a format consistent with the current data model. The rejected data object (or value within the data object) is then sent to the client device 110 for display to the user 105.
Each row of data 202-209 in
At 202, a user 228 named “LarryB” created a document 226 called “Plane.doc.” The next day, at 204, a user identified as “Melvin” performed an attribute change activity 224 of the document 226, changing the name of the document to “BigPlane.doc.” Thereafter, at 206, LarryB attempted a revision on the original document, “Plane.doc.” Because the system detected a conflict 230, the data object containing changes by LarryB was rejected 232. In this example, the conflict may have been detected when the middleware server detected that the document (on the backend) associated with the data object submitted by LarryB was a more recent version of the document than was received from the user system. A rejection can occur for a multitude of other reasons as well. In another instance, the data object may be missing a data value required for storage on the backend server. The extent of error detection and description varies depending upon the particular system using the embodiment.
Displaying rejected data objects contemporaneously with accepted data objects, as shown in
Because the changes made by LarryB at 206 were rejected, the changes were not stored in the backend data repository. In past systems, the changes made by LarryB would remain undocumented, and the changes (and even evidence of the changes) may be lost forever. In this example, LarryB did not resubmit the changes before leaving work, perhaps because the rejection occurred toward the end of his work day. The next morning, Melvin modified the document at 206 but did not incorporate LarryB's rejected changes. As a result, an audit of the document may not disclose LarryB's potentially-critical activities. For instance, the document may not show that LarryB replaced a component within the airplane, which could lead to incorrect presumptions about this component or the airplane in general. Even if LarryB resubmits the changes in an accepted data object, a prior system may no reveal the actual maintenance chronology (e.g., LarryB made changes before Melvin). Therefore, the audit would fail.
However, by using an embodiment to store and display the rejected data objects, as in
Similarly, at 208, a user 228 named Belinda submitted a revision 224 that was rejected based on a validation error 230. According to the error description 232, Belinda did not have permission to modify the document. However, if Belinda's changes are not implemented by another user, important-business data may be lost. By storing and displaying values from the rejected business object, the source of missing business data and, in some embodiments, even the rejected data object can be retrieved and displayed.
At step 310, the rejected data object can be converted into a storage format. The storage format may be a generic format that allows the middleware server to reconstruct the documented data object in a format consistent with the backend data model (e.g., a first data model) at a later time. In general, the storage format may include the data value(s) of the rejected object, and data related to the reason for rejection.
Converting the rejected data object to the storage format may require mapping fields from the rejected data object to the storage format. This process is explained more fully below with respect to, for example,
At step 320, the middleware server stores the rejected data object in the storage format on a storage medium. The storage medium may be part of (or include) the documentation repository in some embodiments, and may be part of the backend portion of the system. Possible storage mediums include one or more servers, disks, hard drives, flash drives, memory (such as RAM or ROM), and other media capable of holding electronic data. The data may be stored in an organized fashion, such as with one or more indexes, to facilitate locating the documented rejected data object.
At step 330, the middleware server stores information for relating the rejected data object in the storage format to a first data model. The first data model includes the format with which files are recognized within the business logic in the backend of the system. In some embodiments, the data model also includes a database schema for relating objects in the backend. This schema may also designate tables for holding particular field values in a relational manner.
The relational information varies depending upon the embodiment. In one embodiment, the relational information includes a pointer to metadata that describes the relationships between entities in the backend system to data objects used by the system. This can enable the middleware server to access domain definitions and texts needed to display, visualize, or otherwise use the data in the future. It may also allow for translating the values in the data object into various available logon languages in some embodiments.
In one embodiment, the relational information includes the metadata itself. For example, the metadata may describe the data object that was rejected. The metadata may also describe the mapping of the rejected data object to the backend data model. The middleware server can access the metadata when attempting to translate the rejected data object. Alternatively, the metadata may identify a service within the backend for converting the rejected data object to a currently used format.
The metadata may also include a version identification in some embodiments. The version identification may enable the middleware server to process documented data objects long after the data object format, data fields, and/or service signatures have changed on the backend system. For example, the exact set of metadata used to convert or reformat the stored data object may be selected based on the version identifier.
The request may include one or more values that are utilized for locating the rejected data object. For example, a particular device ID may be included in the request for retrieving rejected data related to that device ID. In some embodiments, the request may not specify a particular data object, and may instead apply to a group of more than one data objects. The scope of allowable queries depends on the embodiment.
At step 420, the middleware server locates the rejected data object. This may done by searching the documentation repository using information included in the request. In some embodiments, the middleware server may submit a query to the documentation repository where the rejected data is stored. The documentation service may locate the rejected data object. In another embodiment, some other service performs the search.
At step 430, the rejected data object may be translated from the storage format into a first object format consistent with a current data model used by the system. For example, the storage format of the rejected data may not be usable within the system without first applying the translation.
In some embodiments, the middleware server can utilize metadata or other relational information to perform the translation. The rejected data object may include a pointer to the appropriate metadata that defines relationships between values of the rejected data object in storage format to the first data format used by the current system. In this way, the metadata may serve as a relational map between data object formats. The relational map can correlate fields, values, and/or objects, depending on the embodiment.
In another instance, the documented rejected data object includes a version identifier that can be used to identify the correct version of metadata to utilize. Once the middleware server establishes the relationships specified in the metadata or other relational information, the middleware server can rearrange and translate the rejected data object values and relationships into the first object format. Example illustrations of this step are explained below with respect to, for example,
At step 440, after the rejected data object has been formatted according to the first data object format, a data value belonging to the rejected data object can be displayed contemporaneously with another data value from an accepted data object. Exemplary illustrations are provided in
First, submitted data 512 is received from a user system by the middleware server in a submission format 514. For example, if a user edits a file on the user system, the submission format may include the file itself, a value indicating a revision was made, a user identification number, and/or a time stamp. These values may be submitted as part of, for example, a data object that is recognized and useable by the middleware server. Alternatively, the middleware server may convert the data object into a first object format before using the data object. In either case, the data object may be submitted when the user system gains connects to the middleware server. Thereafter, a system component may reject the data object for any of the various reasons already discussed.
Once the data object has been rejected, a process begins for collecting the rejected data 520. The process may depend on what part of the system does the rejection. In addition, at 522, a rejection message may be sent to the client device. The rejection message might also be used in the collection process, as previously explained.
The middleware server then converts the collected rejected data values 530 into a storage format 532. The storage format may contain more data than the submission format. For example, the storage format may contain a reason for the rejection, and a generic identifier to distinguish the stored object from other rejected objects stored in the repository. In addition, the storage format may include metadata and/or a pointer to metadata for use in reconstructing the documented data object 530 into a new data format 562 and/or 572 at a later time.
Besides additional data values, the storage format 532 may also be structurally different than the submission format 514. For example, the data may be formatted to fit within a specific database, such as an object oriented datable, a relational database, and/or a flat file. Additionally, the data may be compressed to save storage space, or encrypted differently than the submission data to allow only certain users or processes to access the stored rejection data object. In one aspect, the storage format for a rejected data object may differ from the storage format of audit information for an accepted data object.
The data retrieval 550 process may begin with the rejected data object in 530 in the storage format 532. The rejected data object may be translated into a first object format 562 consistent with a first data model 563 by using metadata or some other relational information. More specifically, the metadata may allow the middleware server to map any field and its data value in the stored data object to the first object format 562 for the same type of data object as would be successfully stored in a backend data repository (for accepted data objects). In one embodiment, the stored data object is related to one or more object types in the backend service, from there to a first object format 562 including tables and/or object relationships, and from there to a rejected data object 560 that conforms with the first object format 562. In this way, the rejected data object 530 can be manipulated as a first format data object 560, even though the successfully stored data objects may rely on a different data object definition 562, different technical implementation, and/or different execution system context than when the submitted data 512 was initially rejected.
In one embodiment, the data retrieval process 550 may also include translating the rejected data object 530 into a second object format 572, consistent with a second data model 573. For example, the second data model 573 may be a later version of the first data model 563. When the first data model is updated to improve system performance or add new features to the system, the documented rejected data object 530 does not become obsolete and/or unusable. So long as the metadata or other relational information is updated to relate the rejected data object to the new (i.e., second) data model, the rejected data object may still be translatable into the second object format 572.
In one embodiment, the rejected data object 530 is translated directly to the second object format 572. In another embodiment, the rejected data object 530 is initially translated into the first object format 562 as described above, and then translated into the second object format 572 from the first data object format 562. For example, the backend system may include additional metadata defining the relationships between the first data model 563 (e.g., a previous data model) and the second data model 573 (e.g., the current data model).
The data may then be translated into a display format 582, such as is illustrated in the examples of
Turning to
To translate the rejected data object 610 into storage format 640, the data value 612 field is mapped to the generic value 628 field, and the reason rejected field 614 is similarly mapped. In addition, the data object 620 is given a generic identifier 622 for identifying the rejected data object 620 amongst other documented data objects. This relational information may be defined by metadata, or may be pre-programmed into the middleware server.
A generic type 624 may also be defined in some embodiments that relates the documented rejected data object 620 to one or more entities within the first data model. Alternatively or in addition, the generic type 624 may contain a pointer to metadata for establishing the essential relationships. The generic type 624 can also define how to extract multiple values and/or pointers the generic value 628 in creating a data object consistent with the first object model.
In some embodiments, a generic version identifier 626 is used to identify the correct metadata to use in reconstructing the documented rejected data object 620. The version identifier 626 may be useful when a change is made to the data object format of either the client (i.e., front end) data objects, the documented data objects, or the data objects used within the first data model. For example, the same data object type may require different relationships to accommodate changes made to any of the data formats or data models.
A generic description 630 may also be included to allow for manual identification of the documented data object if the automated process fails.
Turning to
In this example, the rejected data object is broken down into individual fields (e.g., 660 and 680) and values (e.g., 658 and 678). By breaking the rejected data object down into a generic and abstract form for storage, the data object may be easily rebuilt to conform with a first data model at some later date. For example, the generic type 674 may be used to identify the type of object that the rejected data object represents. In this example, the object is an order record. Using metadata that establishes relationships to the current data model, each field identified in the data description can be mapped to the appropriate field(s) for an order within the current data model. The metadata may relate a data value 658, based on the data description 660, to an entity used in a successfully-stored data object of similar type. The model version 656 can be used in this example to identify the correct metadata to use in establishing the relationships.
In one embodiment, there are two separate version identification numbers for the metadata. The first version number may indicate a storage format version used to store the rejected data object. The second version number may indicate a first object format to which the rejected data object should be translated. In this embodiment, the second version identification may be specified by the middleware server or by a client device at the time that the rejected data object is requested, depending on the embodiment. In this way, the same rejected data object may be displayed, for example, in different languages at different client devices.
The metadata is also used to form the necessary relationships between records 730, 732, and 734, which are included in the rejected data object in first object format 750. In this example, the records are relationally linked by defining the same user value 741 of record 730 as value 742 of record 732, and the same note value 743 as note value 744.
After being translated to the first object format 750, the rejected data object can be used by the system to make internal calculations and/or display 790 one or more values from the rejected data object.
The rejected data object 752 may contain an object identifier 754 so that the rejected data object 752 can be located amongst other rejected data objects. In addition, the version identifier 756 can indicate the correct metadata to use in translating the rejected data object 752. In this example, the rejected data object 752 contains a pointer to metadata so that the metadata can be located efficiently.
The data value 760 can contain data for populating the various fields of the rejected data object 754 in first object format 750. In this example, the different fields can be mapped from the data value 760 by using metadata that describes which portions of data value 760 correspond to the various fields of object 754. This may also be useful, for example, if the rejected data object 752 is stored in a flat file, the metadata may define how to parse data from the flat file that correlates to each data field of the rejected data object 754 in first object format. These definitions may also be stored as part of the rejected data object 752 in storage format 640, such as by defining the metadata object 764 and/or metadata mapping 766.
The rejected data object 754 in first object format 750 may also contain a file, such as “Plane.doc” 756. This allows the complete reconstruction of a data object, including modifications that might otherwise be lost forever. The file may be stored in a data repository and/or as part of the data value 760. In one embodiment, a user can access the file after the rejected data object has been displayed.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the exemplary embodiments of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
