METHOD AND APPARATUS FOR COMPARING PROCESS DESIGNS

Abstract

This invention relates to a data comparison tool, method and computer program product for analysing a set of data, the set of data being derived from an original set of data, comprising: comparing the derived set of data with the original set of data; generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item; providing at least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item; whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets. An editor agent is provided for allowing the user to confirm or make further changes to the data item. An undo agent is provided for undoing the difference wherein a changed item is changed back, an additional item is removed and a missing item is replaced.

Description

BACKGROUND

IBM WebSphere Data Integration Suite comprises a job design tool that is used to design data flow between stages, known within the product as “Jobs” using a graphical user interface (GUI). An example GUI window 10 produced by the job design tool is shown in FIG. 1. GUI window 10 presents stages 12 and links 14 between the stages. Stages 12 represent data sources or targets, or data processing components, all collectively known as “stages”. The links 14 connect pairs of stages and represent the (one-way) flow of data between them. A link 14 has a stage 12 at either end; these ends are known as either “inputs” or “outputs” of the stage, depending on whether the stage is at the target or source end of a link 14. Data is deemed to flow along a link 14 as records, whose structure is declared via a set of “column” definitions attached to the link or, in some cases, the stage being traversed. Stages 12 and links 14 have user-assigned names, and types that are indicated via different icons.

Jobs, stages 12, links 14 and columns all have “Properties” that further define their behaviour. The job design tool allows the user to drag and drop stages 12 and links 14 onto a “canvas” that represents the overall job design; then to navigate the canvas, select a stage 12 by pointing at it, and open a properties editor that dives down into a stage's link-level inputs and outputs, and columns, to edit the various properties. In this way very complicated data flow graphs can be built up, containing several levels with large amounts of metadata.

A perennial issue for designers has been how to compare versions of a job design in a genuinely useful way. The current approach is to export a job's overall metadata as an XML representation, and use a standard XML-oriented diff tool to compare two XML documents generated from two copies of the job. The problem with this approach is that, except in trivial cases, there is insufficient context for the designer to compare the designs, and be able to distinguish between differences in structure or properties. Also, there is often unnecessary detail shown in terms of what has not been changed.

SUMMARY OF INVENTION

According to a first aspect of the present invention there is provided a method of analyzing a set of data, the set of data being derived from an original set of data, comprising: comparing the derived set of data with the original set of data; generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item; providing at least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item; whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets.

According to a second aspect of the invention there is provided a data comparison tool for analyzing a set of data, the set of data being derived from an original set of data, comprising: a comparator for comparing the derived set of data with the original set of data; a link generator for generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item; and at least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item; whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets.

A comparison tool has been implemented that has knowledge of the major structural components of a job design and extracts the differences between two designs as hierarchical information so that it can be presented as an expandable tree in the context of the original jobs being compared.

The differences between two designs are presented hierarchically for ease of exploration, and as the user selects items in the comparison window the relevant part of one or both job design canvases is highlighted. The nodes of the tree represent the basic structure that the user navigates to set properties (Job>Stage>Input/Output>Column). A node represents either a repeating group (e.g. stages, inputs) or a specific item at that level (i.e. name of the stage or link involved). Leaves of the tree represent property or name differences, or the addition or removal of a node. Nodes are only present if there is a change to some leaf below it—so only the parts of the jobs that differ will appear in the tree.

If the user selects a node in the comparison window that represents a stage, the stage or stages involved will be highlighted in the job design windows. If the node represents a changed stage, both stage icons are highlighted on both canvases. If it represents an added or removed stage, the stage icon is highlighted on only one canvas—that of the job where the stage exists. Furthermore, leaf items that describe any change are presented in a “hyperlink” style, so that clicking on the appropriate part of the item takes you directly to the job editor dialog that contains the property in question. The dialog may pertain to either of the two jobs being compared, depending on which hyperlink was selected. This makes it easy to look at a changed property in the context it is being used, either in the “before” version of the job, or the “after” version.

A tool to allow a user to save the comparison tree as an HTML file has also been implemented. This tool uses dynamic HTML to allow a user to expand and contract the difference tree in the same way as when looking at it via the comparison tool in the context of the job design component. The layout and appearance of the HTML is the same in both the report and the tool.

DESCRIPTION OF DRAWINGS

Embodiments of the invention will now be described, by means of example only, with reference to the accompanying drawings in which:

FIG. 1 is a design diagram schematic;

FIG. 2 is a schematic of the system of the present embodiment;

FIG. 3 is a schematic of the method of the present embodiment;

FIG. 4 is a comparison Window with a stage added;

FIG. 5 is a comparison Window with a stage changed; and

FIG. 6 is a comparison Report as HTML.

DESCRIPTION OF THE EMBODIMENTS

A data integration system 20 of the preferred embodiment is shown in FIG. 2. The data integration system comprises a data flow editor 22; a user interface 23; a storage for a data sets 24A and 24B; and data comparison tool 26.

The data flow editor displays a selected data set and provides editing functions for the data. For instance, a user selected an original data set 24A, edits data within and saves a new derived data set 24B.

The user interface 23 provides a viewer for the data set and for comparison results of the data sets. In this embodiment the user interface uses operating system windows to display the data and comparisons.

An original data set can be created using the data flow editor and saved in memory 24A. A derived data set can be edited using data flow editor 22 and save in memory 24B.

The data comparison tool 26 comprises: a comparator 28; a link generator 30; an editor agent 32; a undo agent 34; and storage 36 for comparison results. The data comparison tool 26 is controlled by method 300 shown in FIG. 3.

The comparator 28 compares a derived set of data with an original set of data.

The link generator 30 generates a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item. A list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets. The hyperlink can link to an item in the derived data set which is different to a corresponding item in the original data set. The hyperlink can also link to an item in the derived data set which was added to the original data set. The hyperlink can also link to an item in the original data set that was removed in the derived data set.

The editor agent 34 allows the user to confirm or make further changes to the data item. The editor agent can be a simple text editor or a special editor for that particular data item.

The undo agent 36 undoes the difference wherein a changed item is changed back, an additional item is removed and a missing item is replaced.

The comparison tool method 300 defines a sequence of steps that the tool takes once operated.

The first step 302 the comparator is prompted to compare the derived set of data with the original set of data.

The next step 304, the link generator is prompted to generate a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item.

In step 306, the list of hyperlinks is saved in the memory 36.

In step 308, the list is displayed using user interface 23.

In step 310 an agent is activated on selection of a hyperlink to operate on the changed, additional or missing data item. The editor agent can be activated to allow the user to confirm or make further changes to the data item. The undo agent can be activated to undoing the difference wherein a changed item is changed back, an additional item is removed and an missing item is replaced.

Some examples of how the comparison tool works follow. FIG. 4 shows three windows; a job design (“Parallel”) after the difference was applied; the job design before the difference was applied; and the comparison results from the tool. Using the job design from FIG. 1 as the “before” version of a job, the designer has replaced the rightmost stage, named “Sequential_File_—10”, with one of a different type, named “Data_Set_—13”. The designer has also made various lower-level changes to properties that do not appear on the canvas—to see them one would need to open up stage editor dialogs on the various stages whose names and types match between the two jobs, and try to spot the differences.

The comparison results window in FIG. 4 shows the result of running the comparison tool. The comparison tool identified which stages have been changed and added detail about each change. Note that the stages named “Row_Generator_—0” and “Transformer_—1” do not appear in the example comparison results window because they have identical properties and sub-structure. Expanding the node “Data_Set_—13” reveals a highlighted hypertext statement with text indicating that it has been added to job “After_Diff”. The hypertext “Data_Set_—13” is linked to the Data_Set-13 object and clicking on the link brings up the stage editor for that stage in the job named “After_Diff”.

The next screenshot in FIG. 5 shows what happens when you expand the node labelled “Lookup_File_Set_—5”. This shows that there have been changes to an output of that stage, named “DSLink6”. There is both a property change, the property seen by the user as hypertext “Lookup File Set” has changed its value, and a structural change, in that the column named “ColB” was added to the link. In this case the icons representing the stage “Lookup File Set” are highlighted in both canvases. Two markup statements have been added to the comparison results. A first markup statement indicates that the change is from one value to another value and comprises a hyperlink to original property field of the object in the original job; a hyperlink to the changed property field of the object in the changed job; and a hyperlink to an undo agent. Clicking on the hypertext “MyFirstLUFS” will open up the stage editor for that stage in the context of the “Before_Diff” job, since “MyFirstLUFS” was the value of the property in that job. Clicking on “MySecondLUFS” will conversely open up the stage editor in the “After_Diff” job. Clicking on “Changed” will execute the undo agent which prompts the user to undo the change in the changed job. A second markup statement indicates that the change was an addition of something and comprises a hyperlink to the added data (“ColB”) and a hyperlink to an undo agent (“Added”). Clicking on “ColB” takes you into the “After_Diff” job, since that is where column “ColB” exists. Clicking on “Added” will execute the undo agent which prompts the user to undo the change in the changed job.

The next screenshot in FIG. 6 shows the complete output of the comparison tool for this example, saved as an HTML file and viewed in Internet Explorer, with all nodes expanded.

It will be clear to one skilled in the art that the method of the present invention may suitably be embodied in a logic apparatus comprising logic means to perform the steps of the method, and that such logic means may comprise hardware components or firmware components.

It will be equally clear to one skilled in the art that the logic arrangement of the present invention may suitably be embodied in a logic apparatus comprising logic means to perform the steps of the method, and that such logic means may comprise components such as logic gates in, for example, a programmable logic array. Such a logic arrangement may further be embodied in enabling means for temporarily or permanently establishing logical structures in such an array using, for example, a virtual hardware descriptor language, which may be stored using fixed or transmittable carrier media.

It will be appreciated that the method described above may also suitably be carried out fully or partially in software running on one or more processors (not shown), and that the software may be provided as a computer program element carried on any suitable data carrier (also not shown) such as a magnetic or optical computer disc. The channels for the transmission of data likewise may include storage media of all descriptions as well as signal carrying media, such as wired or wireless signal media.

The present invention may suitably be embodied as a computer program product for use with a computer system. Such an implementation may comprise a series of computer readable instructions either fixed on a tangible medium, such as a computer readable medium, for example, diskette, CD-ROM, ROM, or hard disk, or transmittable to a computer system, via a modem or other interface device, over either a tangible medium, including but not limited to optical or analogue communications lines, or intangibly using wireless techniques, including but not limited to microwave, infrared or other transmission techniques. The series of computer readable instructions embodies all or part of the functionality previously described herein.

Those skilled in the art will appreciate that such computer readable instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Further, such instructions may be stored using any memory technology, present or future, including but not limited to, semiconductor, magnetic, or optical, or transmitted using any communications technology, present or future, including but not limited to optical, infrared, or microwave. It is contemplated that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation, for example, shrink-wrapped software, pre-loaded with a computer system, for example, on a system ROM or fixed disk, or distributed from a server or electronic bulletin board over a network, for example, the Internet or World Wide Web.

It will be further appreciated that embodiments of the present invention may be provided in the form of a service deployed on behalf of a customer to offer service on demand.

It will also be appreciated that various further modifications to the preferred embodiment described above will be apparent to a person of ordinary skill in the art.

Claims

1. A method of analysing a set of data, the set of data being derived from an original set of data, comprising: comparing the derived set of data with the original set of data; generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item;providing at least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item;whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets.

2. A method according to claim 1 further comprising providing an editor agent for allowing the user to confirm or make further changes to the data item.

3. A method according to claim 1 further comprising providing an undo agent for undoing the difference wherein a changed item is changed back, an additional item is removed and an missing item is replaced.

4. A method according to claim 1 wherein the hyperlink links to an item in the derived data set which is different to a corresponding item in the original data set.

5. A method according to claim 1 wherein the hyperlink links to an item in the derived data set which was added to the original data set.

6. A method according to claim 1 wherein the hyperlink links to an item in the original data set that was removed in the derived data set.

7. A data comparison tool for analysing a set of data, the set of data being derived from an original set of data, comprising: a comparator for comparing the derived set of data with the original set of data;a link generator for generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item; andat least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item;whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets.

8. A tool according to claim 7 further comprising an editor agent for allowing the user to confirm or make further changes to the data item.

9. A tool according to claim 7 further comprising an undo agent for undoing the difference wherein a changed item is changed back, an additional item is removed and an missing item is replaced.

10. A tool according to claim 7 wherein the hyperlink links to an item in the derived data set which is different to a corresponding item in the original data set.

11. A tool according to claim 7 wherein the hyperlink links to an item in the derived data set which was added to the original data set.

12. A tool according to claim 7 wherein the hyperlink links to an item in the original data set that was removed in the derived data set.

13. A computer program product comprising computer readable recording medium having computer readable code stored thereon for analysing a set of data, the set of data being derived from an original set of data, said computer readable code which when loaded onto a computer system and executed performs the following steps: comparing the derived set of data with the original set of data;generating a hyperlink to represent each difference between the derived and original data set, each difference being a changed item, an additional item, or a missing data item;providing at least one agent that is activated on selection of the hyperlink to operate on the changed, additional or missing data item;whereby a list of hyperlinks is generated to represent all the differences of the derived and original data set and selecting one of the hyperlinks will execute an agent to operate on a single data item in one of the data sets.

14. A computer program product according to claim 13 further comprising providing an editor agent for allowing the user to confirm or make further changes to the data item.

15. A computer program product according to claim 13 further comprising providing an undo agent for undoing the difference wherein a changed item is changed back, an additional item is removed and an missing item is replaced.

16. A computer program product according to claim 13 wherein the hyperlink links to an item in the derived data set which is different to a corresponding item in the original data set.

17. A computer program product according to claim 13 wherein the hyperlink links to an item in the derived data set which was added to the original data set.

18. A computer program product according to claim 13 wherein the hyperlink links to an item in the original data set that was removed in the derived data set.