Data extensibility using external database tables

Abstract

In one embodiment, a data processing system includes a first database 304 maintained by a first party (such as a customer); a second database 324, 328 maintained by a second party (such as a vendor) different from the first party; a work queue 316 to specify data to be added to the second database; an extract, transform and load module 308 to write data from the first database to the work queue; and a data import module operable 320 to import data from the work queue to the second database. In the system, the first party is not privileged to write the data directly to the second database but is privileged to write the data to the work queue. The import module transforms the data written to the work queue to a form semantically compatible with the second database.

Description

FIELD OF THE INVENTION

The present invention is directed generally to event processing and specifically to monitoring agent performance.

BACKGROUND OF THE INVENTION

Contact centers, such as Automatic Call Distribution or ACD systems, are employed by many enterprises to service customer contacts. A typical contact center includes a switch and/or server to receive and route incoming packet-switched and/or circuit-switched contacts and one or more resources, such as human agents and automated resources (e.g., Interactive Voice Response (IVR) units), to service the incoming contacts. Contact centers distribute contacts, whether inbound or outbound, for servicing to any suitable resource according to predefined criteria. In many existing systems, the criteria for servicing the contact from the moment that the contact center becomes aware of the contact until the contact is connected to an agent are customer-specifiable (i.e., programmable by the operator of the contact center), via a capability called vectoring. Normally in present-day ACDs when the ACD system's controller detects that an agent has become available to handle a contact, the controller identifies all predefined contact-handling skills of the agent (usually in some order of priority) and delivers to the agent the highest-priority oldest contact that matches the agent's highest-priority skill. Generally, the only condition that results in a contact not being delivered to an available agent is that there are no contacts waiting to be handled.

The primary objective of contact center management, including call-distribution algorithms, is to ultimately maximize contact center performance and profitability. An ongoing challenge in contact center administration is monitoring of agent behaviors to optimize the use of contact center resources and maximize agent performance and profitably. Current products for monitoring and reporting on contact center performance, such as Call Management System or CMS™ by Avaya, Inc., are configured as data warehouses that extract data from multiple sources, transform the data into a normalized form, and load the data into the data warehouse database, typically on a batch schedule. Additional calculations and reporting are performed after the batch load.

A common type of data warehouse is based on dimensional modeling. Dimensional modeling is a data model that divides the world into measurements and context. Measurements are usually numeric and taken repeatedly. Numeric measurements are facts. Facts are surrounded by textual context in existence when the fact is recorded. Context is often subdivided into dimensions. Fact tables are used in dimensional modeling to logically model measurements with multiple foreign keys referring to the contextual entities. The contextual entities each have an associated primary key. A “key” is a data element (e.g., attribute or column) that identifies an instance of an entity or record in a collection of data, such as a table. A “primary key” is a column or combination of columns whose values uniquely identify a row in a table or is the attribute or group of attributes selected from the candidate keys as the most suitable to uniquely identify each instance of an entity. A “foreign key” refers to a column or combination of columns whose values are required to match a primary key in another table or is a primary key of a parent entity that contributes to a child entity across a relationship. Types of primary keys include a natural key, or a key having a meaning to users, and a surrogate key, or a key that is artificially or synthetically established, meaningless to users, and used as a substitute for a natural key.

If the same entity (e.g., agent) is represented on multiple data sources (e.g., inbound call system and outbound call system) by different natural keys, a traditional data warehouse generates and assigns a surrogate key to identify the entity. The surrogate key is an internal identifier managed by the data warehouse. For example, in a contact center an agent may handle inbound calls from one system and outbound calls from another system, with different identities on each system. Data warehouses commonly process each data source independently, performing data correlation across sources at a later time.

Some data models specify a behavior known as a type 2 slowly changing dimension. A type 2 dimension tracks the history of changes to an entity over time. When an attribute of an entity is changed, such as when a contact center agent changes their skill set or group membership, a new surrogate key for that entity is generated, and a new row inserted into the database. Fact data associated with the entity can now be tracked separately for activities that occurred before versus after the change by referencing the appropriate surrogate key.

An ongoing issue confronting vendors of contact center software products, particularly products including dimensional modeling software, is integration of the enterprise database applications (or application software), typically purchased from other vendors, with the contact center software. The vendor of the contact center software desires ease of integration with the existing enterprise database application software but must be careful that the integration of the software does not lead to semantical inconsistencies and other conflicts that can result in reporting inaccuracies or, even worse, software malfunctions.

A number of approaches have been employed to address this issue.

In one approach, the database provides a limited ability to control the types of extensions a particular database user can make. For example, a database table owner disallows column deletion to protect existing data. The table owner may also wish to allow columns to be added to the table and can configure the database to allow a table user to add a fixed set of columns. However, if the existing columns have constraints on their values beyond the conventional relational model, there is no way to ensure that data in the new columns adheres to those constraints. For example in dimensional modeling, dimension tables with type-2 keys have particular rules about when a new key must be generated. Basically, if the value of any of the non-type-2 key columns changes, an entire new row with a new type-2 key and a complete copy of all up-to-date data are inserted to the table. This arrangement can preserve the historical information associated with the dimensional table. However, allowing an arbitrary third party to create such columns requires the third party to participate in the type-2 key semantics. Unfortunately, databases do not provide a way to guarantee the type-2 semantics. For example, the customer can undo a type-2 key change done by the contact center database software or vice versa. Moreover, this approach has also suffered from race conditions that could cause loss of updates.

In another approach, database triggers on the table are used to watch for third party changes, which, when identified, are followed by an application applying appropriate type-2 semantics. Unfortunately, such an approach frequently leads to race conditions between the application and the third party that prevent proper logical semantics from being maintained. Simply put, there is no way to control the third party's timing associated with changes to the extended column set. Often times, the application is unwilling to allow changes to a table for some reason while an Extract, Transform, and Load or ETL module is computing summaries from the table, for example.

Another approach uses transactions or “database locking” in which change requests (transactions) are delayed for a determined period of time. Database locking may help but can be difficult to use for a complete solution. For instance, if the transaction hold time is long-lived, the third party's application or the contact center application may be bogged down.

SUMMARY OF THE INVENTION

These and other needs are addressed by the various embodiments and configurations of the present invention. The present invention is directed generally to an extensible data model.

In one embodiment or the present invention, a data processing system includes:

(a) a first database maintained or managed by a first party;

(b) a second database maintained or managed by a second party different from the first party;

(c) a work queue to specify data to be added to the second database;

(d) an extract, transform and load module operable to write data from the first database to the work queue; and

(e) a data import module operable to import data from the work queue to the second database.

The first party's privileges with respect to the second database are limited. For example, the first party is preferably not privileged to write (insert, delete, and/or update) the data directly to the second database or to modify (insert, delete, and/or update) the data model defining the second database. This is so because the second party is responsible for maintaining and modifying the second database, including customizing the second database to receive custom data from the first database. Typically, the first and second parties are different unrelated business entities that are not under common control and one does not have a controlling interest in the other.

To protect the proprietary schema of the second database from the first party, the second database permissions for the first party are preferably set to read-only for first party access. Preferably, the first party is prohibited from viewing selected parts of the schema. This is done by permitting access only through the database views and not the physical tables. Generally, the first party is permitted to write data directly only to work queue tables and, possibly, specified custom tables.

In a second embodiment, a method is provided that includes the steps of:

(a) receiving from the first party a set of specifications defining an extension to the second database maintained by the second party;

(b) modifying the second database as set forth in the set of specifications, the modification including a custom table, a custom row in a table, and/or a custom column in a table;

(c) creating a work queue corresponding to the modification;

(d) the work queue receiving first data from a first database;

(e) normalizing the first data to form second data; and

(f) writing the second data to the custom table, custom row, and/or custom column.

The method effectively performs mapping of data from the first to the second database. The first data is semantically compatible with the first database and semantically incompatible with the second database, and the second data is semantically incompatible with the first database and semantically compatible with the second database. As used herein, “semantics” refers to the relationships of symbols, characters, or groups of symbols or characters to their meanings and/or to one another. The relationships are typically independent of the manner of their interpretation and use. Two symbols or characters are “compatible” when they may be used in a common database without first requiring semantical changes to the database to accommodate their concurrent use.

The above embodiments can allow unprivileged parties (potentially not trusted) to make extensions to a database model while providing protection for logical semantics needed by the database model. The unprivileged party knows the intricacies of the first but not the second database while the second party knows the intricacies of the second but not the first database. The above embodiments can effectively blend the knowledge of the two parties to combine the first and second databases while effectively inhibiting data conflicts, collisions, and other inconsistencies from conflicting modifications to the second database. It therefore relieves the first party from having to know and implement complicated semantics, particularly type-2 semantics, of the second database model. For example, the present invention can provide a mechanism to guarantee proper type-2 dimensional keys if the database model uses dimensional modeling.

In one configuration, the invention uses external tables (in the work queues) to protect access to the application's database tables (referred to as “protected tables”). The external tables include a “sandbox” where the enterprise can insert, delete, and update. Database triggers or similar change notification mechanism on the external table provide notification to application software that then makes appropriate changes to the corresponding protected table. The changes can include enforcement of type-2 dimension semantics for the protected table.

In another configuration, the enterprise may only insert into the external table. In this case, the triggers work as before, causing appropriate changes to take place in the protected table. Once those changes are complete, the corresponding row is deleted from the external table. In this way, the external table implements a work request queue for changes to the protected table.

The external tables may not be set up by the unprivileged party. The protected table owner may set the external tables up based on information given by the unprivileged party. For example, the information can include what protected table the external table relates to, what external table columns are, whether the columns should be created inside the protected table or as a separate protected table, etc.

In both configurations, the protected tables cannot be modified by the unprivileged party. The triggering mechanisms enforce the additional semantics (e.g., type-2 semantics) so that the protected table always maintains proper semantics. The application may suspend triggering operations during periods where the protected table undergoes processing (e.g., forming summary data from it). In that case, the unprivileged party can track which changes have occurred by examining the contents of the external table.

Other means of protecting table semantics allow access (e.g., by Structured Query Language or SQL) but require the otherwise unprivileged party to participate in the protected table semantics. The present invention, in contrast, provides the external work queues accessible to the unprivileged party using SQL but preferably uses triggers to ensure proper semantics of the protected table. The invention can thus avoid the need to protect tables using non-SQL third party access.

For the reasons noted above, the invention can provide an interface that blends or multiplexes, in real time or near real time, two different data from two different sources. One source is the enterprise data collection and processing software external to the contact center data collection and processing software and the other source is the contact center software itself.

These and other advantages will be apparent from the disclosure of the invention(s) contained herein.

As used herein, “at least one”, “one or more”, and “and/or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and/or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.

The above-described embodiments and configurations are neither complete nor exhaustive. As will be appreciated, other embodiments of the invention are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram depicting a contact center according to an embodiment of the present invention;

FIG. 2 is a block diagram of a server according to an embodiment of the present invention;

FIG. 3 is a block diagram depicting an embodiment of the event processing module;

FIG. 4 is a flow chart depicting a first operational embodiment of the event processing module; and

FIG. 5 is a flow chart depicting a second operational embodiment of the event processing module.

DETAILED DESCRIPTION

The invention will be illustrated below in conjunction with an exemplary communication system. Although well suited for use with, e.g., a system having an ACD or other similar contact processing switch, the invention is not limited to use with any particular type of communication system switch or configuration of system elements. Those skilled in the art will recognize that the disclosed techniques may be used in any application in which it is desirable to provide improved event (e.g., contact) processing.

FIG. 1 shows an illustrative embodiment of the present invention. A contact center 100 comprises a central server 110, a set of data stores or databases 114 containing contact or customer related information and other information that can enhance the value and efficiency of the contact processing, and a plurality of servers, namely a voice mail server 118, an Interactive Voice Response unit or IVR 122, and other servers 126 (such as a predictive dialer), a switch 130, a plurality of working agents operating packet-switched (first) telecommunication devices 134-1 to N (such as computer work stations or personal computers), and/or circuit-switched (second) telecommunication devices 138-1 to M, all interconnected by a local area network LAN (or wide area network WAN) 142. The servers can be connected via optional communication lines 146 to the switch 130. As will be appreciated, the other servers 126 can also include a scanner (which is normally not connected to the switch 130 or Web server), VoIP software, video call software, voice messaging software, an IP voice server, a fax server, a web server, and an email server and the like. The switch 130 is connected via a plurality of trunks 150 to the Public Switch Telecommunication Network or PSTN 154 and via link(s) 152 to the second telecommunication devices 138-1 to M. A gateway 158 is positioned between the server 110 and the packet-switched network 162 to process communications passing between the server 110 and the network 162.

The term “switch” or “server” as used herein should be understood to include a PBX, an ACD, an enterprise switch, or other type of telecommunications system switch or server, as well as other types of processor-based communication control devices such as media servers, computers, adjuncts, etc.

The switch 130 and/or server 110 can be any architecture for directing contacts to one or more telecommunication devices. Illustratively, the switch and/or server can be a modified form of Avaya Inc.'s Definity™ Private-Branch Exchange (PBX)-based ACD system; MultiVantage™ PBX, CRM Central 2000 Server™, Communication Manager™, S8300™ media server, and/or Avaya Interaction Center™. Typically, the switch/server is a stored-program-controlled system that conventionally includes interfaces to external communication links, a communications switching fabric, service circuits (e.g., tone generators, announcement circuits, etc.), memory for storing control programs and data, and a processor (i.e., a computer) for executing the stored control programs to control the interfaces and the fabric and to provide automatic contact-distribution functionality. The switch and/or server typically include a network interface card (not shown) to provide services to the serviced telecommunication devices. Other types of known switches and servers are well known in the art and therefore not described in detail herein.

The first telecommunication devices 134-1, . . . 134-N are packet-switched and can include, for example, IP hardphones such as the Avaya Inc.'s, 4600 Series IP Phones™, IP softphones such as Avaya Inc.'s, IP Softphone™, Personal Digital Assistants or PDAs, Personal Computers or PCs, laptops, packet-based H.320 video phones and conferencing units, packet-based voice messaging and response units, packet-based traditional computer telephony adjuncts, and any other communication device.

The second telecommunication devices 138-1, . . . 138-M are circuit-switched and can include, for example, wired and wireless telephones, PDAs, H.320 video phones and conferencing units, voice messaging and response units, traditional computer telephony adjuncts, and any other communication device.

The packet-switched network 162 can be any data and/or distributed processing network, such as the Internet.

The packet-switched network 162 is in communication with an external first telecommunication device 174 via a gateway 178, and the circuit-switched network 154 with an external second telecommunication device 180.

It should be emphasized that the configuration of the switch, server, user telecommunication devices, and other elements as shown in FIG. 1 is for purposes of illustration only and should not be construed as limiting the invention to any particular arrangement of elements.

Referring to FIG. 2, one possible configuration of the server 110 is depicted. The server 110 is in communication with a plurality of customer communication lines 200a-y (which can be one or more trunks, phone lines, etc.) and agent communication line 204 (which can be a voice-and-data transmission line such as LAN 142 and/or a circuit switched voice line 140). The server 110 can include an event processing module 228, such as a modified form of Basic Call Management System™ or BCMS, Call Management System™, and/or Operational Analyst™ by Avaya, Inc., that gathers call records and contact-center statistics for use in generating contact-center reports.

Included among the data stored in the server 110 is a set of contact queues 208a-n and a separate set of agent queues 212a-n. Each contact queue 208a-n corresponds to a different set of agent skills, as does each agent queue 212a-n. Conventionally, contacts are prioritized and either are enqueued in individual ones of the contact queues 208a-n in their order of priority or are enqueued in different ones of a plurality of contact queues that correspond to a different priority. Likewise, each agent's skills are prioritized according to his or her level of expertise in that skill, and either agents are enqueued in individual ones of agent queues 212a-n in their order of expertise level or are enqueued in different ones of a plurality of agent queues 212a-n that correspond to a skill and each one of which corresponds to a different expertise level. Included among the control programs in the server 110 is a contact vector 216. Contacts incoming to the contact center are assigned by contact vector 216 to different contact queues 208a-n based upon a number of predetermined criteria, including customer identity, customer needs, contact center needs, current contact center queue lengths, customer value, and the agent skill that is required for the proper handling of the contact. Agents who are available for handling contacts are assigned to agent queues 212a-n based upon the skills and/or skill levels that they possess.

FIG. 3 depicts an embodiment of the event processing module 228. The module 228 comprises a first set 300 of devices, namely an enterprise database 304 and an Extract, Transform and Load or ETL module 308, that are controlled by the enterprise and a second set 312 of devices, namely the enterprise-controlled specification(s) 318, data model modifier 322, work queues 316a-n, data import module 320, vendor database 324, extension table 328, semantics module 332, key authority 336, and identity resolution service 340, all of which are controlled by a third party, such as the manufacturer, vendor, or servicing or managing entity of the contact center software. In other words, the enterprise and third party have first and second sets of privileges respecting modification of the contact center database software, with the first set of privileges not including the ability to modify the contact center database software and/or data model and the second set of privileges including that ability. “Modifying the database software and/or data model” includes, for example, the ability to insert, delete and/or update dimensions, keys, or attributes, and rows and/or columns of tables.

The enterprise database 304 and ETL module 308 are conventional. As will be appreciated, the ETL module 308 reads data from its source, cleans it up and formats it uniformly, and then writes it to the target repository, which is depicted as a work queue 316. By way of example, the ETL module 308 performs any key/identifier mapping from external sources to natural keys recognized by the third party controlled system. The ETL module 308 further ensures that the correct data is loaded. This may include performing calculations or other transformations on the data and associating it with the correct entities. Examples of suitable enterprise database 304 software include Oracle™ database products, and Microsoft™ SQL server database products and customer relationship management software by Siebel™ and of suitable ETL modules include ETL systems offered by Microsoft™, Oracle™, IBM™, Ascential Software Corporation™, and Hummingbird Ltd™.

The enterprise controlled specification(s) 318 and data model modifier 322 collectively provide an extensible data model that may be modified by the third party as requested by the enterprise. The specification(s) are preferably an enterprise metadata specification. The specification is an eXtended Markup Language or XML file that defines the tables and columns to be inserted in, deleted from, or updated in the data model. The specification is preferably a set of XML tags and their relationships to one another. The enterprise edits the specification(s) manually and/or via a user interface. The user interface, for example, would ask the user a series of queries regarding the proposed modifications and would write the responses into an XML file in the specification(s). To make this possible, the enterprise, in one configuration, can view (without editing privileges) selected portions of the data model and/or physical tables for the database 324. The data model modifier 322 implements the modification set forth in the specification(s) 318. The data model can be modified by the data modifier 322 in at least the following ways:

1. custom facts can be added to an existing fact table;

2. custom facts can be added to a new table that parallels an existing fact table;

3. custom attributes can be added to an existing dimension table;

4. custom attributes can be added to a new table that parallels an existing dimension table; and

5. custom tables can be added.

With this in mind, the specification(s) typically include specifications regarding the data to be received from the ETL module. Examples of such specifications include dimension identities and relationships to other objects (i.e., dimensions, keys, and attributes) in the data model 324, new rows and column identities and relationships to other objects in the data model, key identities and types and their relationships to other objects in the data model, attribute identities and relationships to other objects in the data model, the identities of columns requiring type 1 or 2 changes, and the like.

When an existing fact or dimension table is extended, the columns specified in the specification(s) are added to the existing table by the modifier 322. This provides for simpler and better performing report queries but may result in wide tables. This approach is, of course, limited if a particular database implementation restricts the number of columns permitted in a table.

To get around wide table restrictions, the data model modifier 322 can create a new table that is parallel to the existing table. The new table contains the custom columns and a key that relates to the existing table. This approach is arbitrarily extensible but requires a join to access the enterprise columns. New tables created through this approach are managed by the third party in the same way as standard tables.

The modifier 322 can also create custom tables managed entirely by the enterprise.

The new tables and enterprise tables are collectively included in the extension tables 328.

For each table with custom columns and for each extension table 328, the modifier 322 creates a corresponding work queue 316a-n. Each work queue contains the custom columns and the natural key and receives data from the ETL module 308 for import into the respective custom column in vendor database 324 and/or into the corresponding custom (extension) table 328. Preferably, the only tables that the enterprise can write to are the work queue tables and any enterprise-controlled custom tables.

The data import module 320 copies the data from the work queues 316a-n to the third party database 324 and extension tables 328, as appropriate. The module 320 acts as an input adapter that translates records in the work queue to normalized events and makes dimension updates as needed. An “event” is an occurrence or happening and a “normalized event” is an expression of an event that is consistent with the data structures in the third party data model. Normalized events are created for each update and injected into the system 312 for processing. For example, the module 320 converts natural keys into entity identifiers, or keys internal to the contact center database 324 using the (optional) Identity Resolution Service (IRS) 340 and key authority 336.

The data import module can convert custom events into normalized events in a variety of ways. For example, an XML schema for custom events can be defined, and a web service published that translates the XML into normalized events. An alternative approach is to implement a custom input translator that receives events from the ETL module 308 and translates them into normalized events. Additionally, a standard event source may have its own customizable event capabilities.

The data import module can be scheduled to run at an enterprise-specified interval (e.g., daily, hourly, every 5 minutes, etc.), invoked manually after the ETL module has run, or invoked by a database trigger.

The identity resolution service 340 maintains a table of entity identifiers, hints, and corresponding natural keys. A “hint” is a form of instruction indicating when a surrogate key is valid for use. Examples of hints include a yes/no flag regarding availability of the surrogate key for current use, a date and/or time after which the surrogate key is valid, a signature indicating that the key is valid, an identified event which must occur before the key is valid, a time stamp (such as of the Entity IDentifier or EID request), the surrogate key itself, a sequence number and the like. In one embodiment, the hint is used by the key authority to map from an EID to a corresponding (valid) surrogate key.

The key authority 336 manages surrogate keys. When an administration change causes a new surrogate key to be generated, the key authority 324 sends the entity identifier to the identity resolution service 340 along with the hint that a new surrogate key is available. The configurations and operations of the IRS 340 and key authority 336 are described in detail in copending U.S. application Ser. No. 11/199,828, filed Aug. 8, 2005, entitled “Deferred Control of Surrogate Key Generation in a Distributed Processing Architecture” to Bland, et al., which is incorporated herein by this reference.

The (optional) semantics module 332 presents an object model for updating dimension and fact data, hides from the enterprise the details of the physical database implementation, including the physical schema, third party database abstraction, and type 2 dimension semantics, manages type 2 dimension semantics (i.e., handles type 2 dimension changes including the generation of surrogate keys), and, in cooperation with the data import module 320, writes dimension and fact data to the databases.

The third party, extension table, and enterprise databases 304, 324, and 328 collectively form the database 114 of FIG. 1. The databases include facts and measures and related semantics regarding a number of enterprise objects, such as customers, agents, contacts or work items, and agent and contact queues 208 and 212.

The operation of the data model modifier 322 will now be discussed with reference to FIG. 4.

The process is instantiated in step 400 when an entry is written by the enterprise into the specification(s) 318.

In step 404, the modifier 322 detects the entry by a suitable technique. The technique may be, for example, a polling mechanism or by directly launching the modifier 322 after the entry is made.

In step 408, the modifier 322 retrieves a next entry in the specification(s).

In step 412, the modifier 322 modifies the data model as requested by the entry.

In step 416, the modifier 322 deletes the processed entry from the specification(s).

In decision diamond 420, the modifier 322 determines whether there is a next entry in the specification(s). If not, the modifier 322 returns to step 400. If so, the modifier 322 returns to and repeats step 408 with respect to the next entry.

An example will illustrate the operations of the modifier 322. Assume that the enterprise desires to add an agent identifier to the data model because the enterprise has an internal agent identifier for each of its agents in the contact center. The enterprise therefore desires to add to the agent dimension of the third party data model. The enterprise creates an XML specification indicating that a column (or type 2 dimension) called “customer ID” is to be added to the agent dimension table. The type of the column will be called “string”. As will be appreciated, other potential types of columns include numeric currency, integer, float and date/time. The XML specification is as follows:

<table>
<name>agent dimension</name>
<column>
<colname>customerID</name>
<coltype>string</coltype>
<type2dim/>
</column>
</table>

In response, the modifier creates, in the data model, an extension table to the agent dimension table containing the customerID column (as a foreign key in the third party data model for database 324) and adds a work queue table associated with the extension table. After data is written to the work queue for a customer identifier, it is written by the data import module 320 to the corresponding extension table. As will be appreciated, rather than creating a separate extension table the new column could be added directly to the data model for database 324.

The operations of the data import module 320, identity resolution service 340, key authority 336, and semantics module 332 will now be discussed with reference to FIG. 5.

In step 500, the ETL module 320 writes record(s) to one or more work queues (e.g., work queue tables).

In step 504, the data import module 320 detects the new record(s) in the work queue(s). The detection is done by any suitable technique, such as by a polling mechanism (in which the data import module periodically (at defined intervals) polls the work queues for information about unprocessed entries), a database trigger (which fires software in the work queue; that is, database code in the work queues invokes application code when the database code determines that a new row in the work queue table has been written to), or to the enterprise directly launching an import application after the queue table has been written to.

In step 508, the data import module 320 retrieves the next (entered) record from the work queue.

In step 512, the data import module 320 performs any necessary transformations on the retrieved record to normalize the data. For example, enterprise (external) natural keys are converted to database model (internal) keys (such as entity identifiers).

In step 516, the data import module 320 reads the most recent existing record from the database 114 for the specified internal key.

In step 520, the module 320 compares the attributes of the existing database record with the new attributes in the retrieved record.

In decision diamond 524, the module 320 determines whether any attributes have changed. In other words, the module determines whether the attributes in the retrieved record differ from the attributes in the database 114 for specified internal key.

When there is no change, the module 320, in step 528, does not make any change to the database 114 for the retrieved record.

When there is a change, the module 320, in decision diamond 536, determines whether any type 2 attributes has changed.

When no type 2 attribute has changed, the module 320, in step 532, executes a type 1 update. In one configuration, when a type 1 dimension is configured, the entity identifier is used as the key and an update in step 532 affects the existing row in the table.

When a type 2 attribute has changed, the semantics module, in step 540, generates a new surrogate key and, in step 544, the data import module 320 inserts a new record containing the new surrogate key and new attribute value(s).

In one configuration when a type 2 dimension is configured and the update is to a dimension, the modules 320 and 332 select the current dimension data from the database (the selection being based on the entity identifier or use of the key authority 336 to map the entity identifier to a corresponding surrogate key) (step 516); compare the current data to the new data requested for update (if no columns were changed no current update is necessary) (step 520); determine which columns are type 2 (step 536); when all changes are limited to type 1 columns generate an update statement to update the current row in the database (step 532); and if any type 2 column was changed, request a new surrogate key from the key authority (step 540) and generate an insert statement to insert a new row in the database with the new surrogate key (step 544).

If a type 2 column is configured and the update is to a fact, the modules 320 and 322 update the fact by requesting a surrogate key from the key authority using the entity identifier and a hint (as noted the hint informs the key authority which surrogate key applies to the particular fact update and is generated by the IRS, data import module or an event processor and provided to the key authority) (step 540), and generate an appropriate Structured Query Language (SQL) statement to insert or update the fact data using the surrogate key provided by the key authority (step 544).

An illustration of the operation of the data input and semantics modules 320 and 332 will be discussed with reference to the example above. Assume that an agent “Bob Smith” having a natural key “100” in the database 324 has a corresponding customer ID “12345”. In the corresponding work queue, the ETL module writes a foreign key, namely “100” (corresponding to the third party's natural key) and the enterprise's corresponding natural key, or customer ID, namely “12345”. The data import module detects the entry in the work table and inserts a corresponding record, namely a new instance of “Bob Smith” having a new key “101” because the change is specified as a type 2 change. The record is a new row in the agent dimension table that is identified by the key “101”. The entry is then deleted from the table in the work queue.

A number of variations and modifications of the invention can be used. It would be possible to provide for some features of the invention without providing others.

For example in one alternative embodiment, the invention is not limited to type 2 dimensional models but to any dimensional model or other type of relational (nondimensional) database model. The invention is particularly applicable to any application in which there are complicated semantics between the application's data model and the extensions that a customer might want to include. Although type 2 semantics are a good example of such complexity, there could be other applications involving only type 1 dimensional or nondimensional databases having complicated semantics.

In another alternative embodiment, the server and/or switch can be a software-controlled system including a processing unit (CPU), microprocessor, or other type of digital data processor executing software or an Application-Specific Integrated Circuit (ASIC) as well as various portions or combinations of such elements. The memory may be a random access memory (RAM), a read-only memory (ROM), or combinations of these and other types of electronic memory devices.

The present invention, in various embodiments, includes components, methods, processes, systems and/or apparatus substantially as depicted and described herein, including various embodiments, subcombinations, and subsets thereof. Those of skill in the art will understand how to make and use the present invention after understanding the present disclosure. The present invention, in various embodiments, includes providing devices and processes in the absence of items not depicted and/or described herein or in various embodiments hereof, including in the absence of such items as may have been used in previous devices or processes, e.g., for improving performance, achieving ease and/or reducing cost of implementation.

The foregoing discussion of the invention has been presented for purposes of illustration and description. The foregoing is not intended to limit the invention to the form or forms disclosed herein. In the foregoing Detailed Description for example, various features of the invention are grouped together in one or more embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the invention.

Moreover, though the description of the invention has included description of one or more embodiments and certain variations and modifications, other variations and modifications are within the scope of the invention, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative embodiments to the extent permitted, including alternate, interchangeable and/or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and/or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.

Claims

1. A data processing system, comprising: a first database maintained by a first party;a second database maintained by a second party different from and unrelated to the first party, wherein the second party is at least one of a manufacturer or vendor of a software product comprising an extensible data model defining the second database and wherein the first party is a purchaser of the software product;a work queue operable to specify data to be added to the second database;an extract, transform, and load module operable to write data from the first database to the work queue;a data import module operable to import data from the work queue to the second database, wherein the first party is not privileged to write the data directly to the second database, wherein the data in the first database is semantically incompatible with the second database, and wherein the data import module transforms the data written to the work queue to a semantically compatible form for the second database;a specification to extend a second database maintained by a second party, the specification being provided by the first party and defining tables and columns to be inserted in, deleted from, and/or updated in a data model defining to the second database;a modifier operable to modify the data model to reflect the specification and to create the work queue corresponding to the modification;an identity resolution service operable to maintain a table of entity identifiers, hints, and corresponding natural keys, a hint indicating when a surrogate key is valid for use; anda key identifier operable to generate new surrogate keys, send new surrogate keys to the identity resolution service along with a corresponding hint that a new surrogate key is available, and use the hint to map from an entity identifier to a corresponding surrogate key.

2. The system of claim 1, wherein the first party is not privileged to modify directly the semantics of the second database, wherein the first party is not privileged to delete pre-existing data from and update pre-existing data in the second database, and wherein the second party is not privileged to modify directly the semantics of the first database and further comprising: a modifier to modify the second database as set forth in a set of specifications, wherein the modification comprises at least one of a custom table and a custom column in a table.

3. The system of claim 1, wherein the extract, transform and load module maps keys and/or identifiers from an external source to natural keys recognized by the first database and further comprising: a semantics module operable to effect updates to type 2 dimension semantics in the second database to accommodate the imported data.

4. The system of claim 3, wherein the work queue comprises a natural key to the first database, the natural key being external to the second database, wherein the semantics module presents an object model for updating dimension and fact data, hides from the first party details of the second database, and, in cooperation with the data import module, writes dimension and fact data to the second database, and wherein the data import module converts the natural key to a key internal to the second database and external to the first database.

5. The system of claim 1 wherein the first party is privileged to view at least selected parts of the second database, wherein the modifier creates a new custom table that is parallel to an existing table in the data model, the custom table containing a custom column and a key that relates to the existing table, wherein, for each existing table with a custom column and for each custom table the modifier creates a corresponding work queue, wherein each work queue comprises at least one custom column and a natural key relating to an existing table and receives data from the extract, transform, and load module for import into a respective custom column in the second database and/or into a corresponding custom table, and wherein the first party is privileged to write data to the work queue.

6. The system of claim 1, wherein the data import module translates data in the work queue to normalized events and makes dimension updates as needed and wherein the modification is at least one of the following: (i) addition of custom facts to an existing fact table in the second database;(ii) addition of custom facts to a new table that parallels an existing fact table in the second database;(iii) addition of custom attributes to existing dimension table in the second database;(iv) addition of custom attributes to a new table that parallels an existing dimension table in the second database; and(v) addition of a custom table.

7. A method, comprising: (a) receiving from a first party a set of specifications defining an extension to a second database maintained by a second party;(b) modifying, by a modifier, the second database as set forth in the set of specifications, wherein the modification comprises an addition of a custom table and further comprising;(b1) providing a work queue corresponding to the modification;(b2) receiving, by the work queue, first data from a first database;(b3) normalizing, by a semantics module, the first data to form second data, wherein the first data is semantically compatible with the first database and semantically incompatible with the second database and wherein the second data is semantically incompatible with the first database and semantically compatible with the second database;(b4) writing, by a data import module, the second data to the at least one of a custom table, and a custom column in a table;(b5) maintaining, by an identity resolution service, a table of entity identifiers, hints, and corresponding natural keys, a hint indicating when a surrogate key is valid for use; and(b6) generating, by a key identifier, new surrogate keys;(b7) sending, by the key identifier, new surrogate keys to the identity resolution service along with a corresponding hint that a new surrogate key is available; and(b8) using the hint to map from an entity identifier to a corresponding surrogate key.

8. The method of claim 7, wherein a work queue is created for each of a plurality of sets of specifications, wherein, in the receiving step, keys and/or identifiers from an external source are mapped to natural keys recognized by the first database.

9. The method of claim 7, wherein the first party is a customer and the second party is at least one of a manufacturer and vendor of a software product comprising an extensible data model defining the second database wherein the first party is a purchaser of the software product, wherein the first party is not privileged to modify directly the semantics of the second database, wherein the first party is not privileged to write directly to the second database, wherein a set of specifications defines at least one of tables and columns to be inserted in, deleted from, and/or updated in a data model defining the second database, and wherein the second party is not privileged to modify directly the semantics of the first database.

10. The method of claim 7, wherein a modifier creates a new custom table that is parallel to an existing table in a data model defining the second database, the custom table containing a custom column and a key that relates to the existing table, wherein, for each existing table with a custom column and for each custom table the modifier creates a corresponding work queue, wherein each work queue comprises at least one custom column and a natural key relating to an existing table and receives data from an extract, transform, and load module for import into a respective custom column in the second database and/or into a corresponding custom table, and wherein the first party is privileged to write data to the work queue and further comprising: (g) permitting the first party to view only selected parts of the second database.

11. The method of claim 7, wherein the modification is at least one of the following (i) (iv): (i) addition of custom facts to an existing fact table in the second database;(ii) addition of custom facts to a new table that parallels an existing fact table in the second database;(iii) addition of custom attributes to existing dimension table in the second database; and(iv) addition of custom attributes to a new table that parallels an existing dimension table in the second database.

12. The method of claim 11, wherein the at least one of the following (i)-(iv) is (i) addition of custom facts to an existing fact table in the second database.

13. The method of claim 11, wherein the at least one of the following (i)-(iv) is (ii) addition of custom facts to a new table that parallels an existing fact table in the second database.

14. The method of claim 11, wherein the at least one of the following (i)-(iv) is (iii) addition of custom attributes to existing dimension table in the second database.

15. The method of claim 11, wherein the at least one of the following (i)-(iv) is (iv) addition of custom attributes to a new table that parallels an existing dimension table in the second database.

16. The method of claim 7, wherein a semantics module presents an object model for updating dimension and fact data, hides from the first party details of the second database, and, in cooperation with a data import module, writes dimension and fact data to the second database, and further comprising: (g) after step (b), deleting the processed set of specifications.

17. The method of claim 7, wherein, in step (b3), the work queue comprises a natural key to the first database, the natural key being external to the second database, and wherein the data import module converts the natural key to a key internal to the second database and external to the first database.

18. The method of claim 7, wherein the writing step comprises: (b4a) comparing a first set of attributes in the second database with a corresponding preexisting second set of attributes in the second database;(b4b) when the first and second set of attributes are different, determining whether a type 2 attribute has changed;(b4c) when a type 2 attribute has not changed, executing a type 1 update to the second database; and(b4d) when a type 2 attribute has changed, generating a new surrogate key and inserting into the second database a new record containing the new surrogate key and the first set of attributes.

19. A computer readable medium comprising processor executable instructions stored on a tangible medium, the instructions to perform the following steps: (a) receiving from a first party a set of specifications defining an extension to a second database maintained by a second party;(b) modifying, by a modifier, a data model defining the second database as set forth in the sets of specifications, wherein the modifications comprise a plurality of the following: (b1) addition of custom facts to an existing fact table in the second database;(b2) addition of custom facts to a new table that parallels an existing fact table in the second database;(b3) addition of custom attributes to existing dimension table in the second database;(b4) addition of custom attributes to a new table that parallels an existing dimension table in the second database; and(b5) addition of a custom table;(c) providing a respective work queue for each of the modifications, each work queue comprising a different key associated with the corresponding modification(d) receiving, by a selected work queue, first data from a first database;(e) normalizing, by a semantics module, the first data to form second data wherein the second data is associated with a selected key, wherein the first data is semantically compatible with the first database and semantically incompatible with the second database, and wherein the second data is semantically incompatible with the first database and semantically compatible with the second database;(f) determining, from the selected key and by a data import module, an existing record in the second database;(g) comparing, by the data import module, an attribute of the existing record with an attribute of the second data;(h) determining, by the data import module, whether the attribute has changed;(i) in response to the determining step (h), applying, by the data import module, the following rules: (i1) when there is no attribute change, not making any change to the second database;(i2) when there is an attribute change and the changed attribute is a type 2 attribute, generating a surrogate key and corresponding hint for the second data and inserting a new record containing the generated surrogate key and new attribute value into the second database; and(i3) when there is an attribute change and the changed attribute is a type 1 attribute, updating an existing row of an existing table.

20. A method, comprising: (a) receiving from a first party a plurality of sets of specifications defining an extension to a second database maintained by a second party;(b) modifying, by a modifier, a data model defining the second database as set forth in the sets of specifications, wherein the modifications comprise a plurality of the following: (b1) addition of custom facts to an existing fact table in the second database;(b2) addition of custom facts to a new table that parallels an existing fact table in the second database;(b3) addition of custom attributes to existing dimension table in the second database;(b4) addition of custom attributes to a new table that parallels an existing dimension table in the second database; and(b5) addition of a custom table;(c) providing a respective work queue for each of the modifications, each work queue comprising a different key associated with the corresponding modification;(d) receiving, by a selected work queue, first data from a first database;(e) normalizing, by a semantics module, the first data to form second data wherein the second data is associated with a selected key, wherein the first data is semantically compatible with the first database and semantically incompatible with the second database, and wherein the second data is semantically incompatible with the first database and semantically compatible with the second database;(f) determining, from the selected key and by a data import module, an existing record in the second database;(g) comparing, by the data import module, an attribute of the existing record with an attribute of the second data;(h) determining, by the data import module, whether the attribute has changed;(i) in response to the determining step (h), applying, by the data import module, the following rules: (i1) when there is no attribute change, not making any change to the second database;(i2) when there is an attribute change and the changed attribute is a type 2 attribute, generating a surrogate key and corresponding hint for the second data and inserting a new record containing the generated surrogate key and new attribute value into the second database; and(i3) when there is an attribute change and the changed attribute is a type 1 attribute, updating an existing row of an existing table.