The present disclosure generally relates to analyzing relationships between data. Particular implementations relate to automatically implementing rules using a collection of relationships determined using machine learning techniques.
As computers become more pervasive, opportunities exist for determining relationships between data that may be generated or acquired. For example, relationships between data, which can be expressed as rules, can be used to determine whether particular types of data are associated with each other. These rules can be used for a variety of purposes, including optimizing various processes, or to obtain insights that might be exploited in other ways.
However, problems can arise in trying to apply rules, particularly collections of rules, to a data set. For example, individuals who may understand the practical implications of rules may find it difficult to develop a technical implementation of the rule. In addition, errors can arise if rules are not correctly implemented, including accounting for possible overlap between rules. Accordingly, room for improvement exists.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Technologies are provided for automatically implementing composite data rules, where a composite data rule includes a plurality of data rules. From the plurality of data rules, rule antecedents and rule consequents are used to automatically generate one or more computing artifacts for evaluating data for compliance with a composite data rule. Computing artifacts can include a scope decision table, which includes rule antecedents of association rules in a composite data rule, and a condition decision table, which includes rule consequents of individual data rules in a composite data rule. Scope and condition expressions can be used with the scope decision table and the condition decision table, respectively, to generate a result indicating whether given data is in scope or whether the data item satisfied consequents in an individual data rule of the composite data rule if the composite data rule is in scope for the data.
In one aspect, a method is provided for automatically generating at least one collective data rule artifact that can be used in evaluating data for compliance with a collective data rule. A first plurality of individual data rules is received. An individual data rule includes one or more antecedents and one or more consequents.
A selection of a second plurality of individual data rules of the first plurality of individual data rules is received, where the second plurality of individual data rules are to be associated with a collective data rule. At least one collective data rule artifact is automatically generated at least in part from at least a portion of the antecedents, consequents, or a combination thereof, of individual data rules of the second plurality of individual data rules.
In another aspect, a method is provided for automatically generating a condition table (i.e., a condition decision table) that can be used in analyzing data items for compliance with a collective data rule. A collective data rule is received that includes a plurality of individual data rules. An individual data rule includes one or more antecedent fields and corresponding antecedent field values and one or more consequent fields and corresponding consequent field values. A condition table is automatically generated. The condition table includes a plurality of rows, where a row corresponds to an individual data rule of the plurality of individual data rules and includes the consequent field values of the respective individual data rule.
In a further aspect, a method is provided for automatically generating a plurality of computing artifacts that can be used in evaluating whether data items (such as data from one or more database tables, including data from a single row of a single database table) complies with a collective data rule. A plurality of data rules (e.g., individual data rules) are received. A data rule includes one or more database fields and corresponding field values corresponding to rule antecedents and one or more database fields and corresponding field values corresponding to rule consequents. One or more data definition language statements are automatically executed to generate a first table. The first table has a plurality of rows. A given row corresponds to a data rule of the plurality of data rules and includes rule antecedent field values for the given data rule.
One or more data definition language statements are automatically executed to generate a second table. The second table has a plurality of rows. A given row corresponds to a data rule of the plurality of data rules and includes rule antecedent field values and rule consequent values for the given data rule. A first condition expression is automatically generated. The first condition expression is configured to return a first value if a data item corresponds to a row of the first table corresponding to a data rule and a second value otherwise. A second condition expression is automatically generated. The second condition expression is configured to return the first value if a data item corresponds to a row of the second table corresponding to a data rule and the second value otherwise.
The present disclosure also includes computing systems and tangible, non-transitory computer readable storage media configured to carry out, or including instructions for carrying out, an above-described method. As described herein, a variety of other features and advantages can be incorporated into the technologies as desired.
As computers become more pervasive, opportunities exist for determining relationships between data that may be generated or acquired. For example, relationships between data, which can be expressed as rules, can be used to determine whether particular types of data are associated with each other. These rules can be used for a variety of purposes, including optimizing various processes, or to obtain insights that might be exploited in other ways.
For example, a user may analyze rules to determine that they provide an indication or test of data quality (e.g., the rules can be used to define validation checks), for predictive purposes, and to uncover relationships that may be used for a variety of purposes, including improving performance and efficiency. In a particular aspect, relationships between particular attributes, or attribute values, of one or more relational database tables can be used to optimize database operations, such as by partitioning data to optimize database operations, such as select and join operations (e.g., reducing a number of multi-node selects or joins, or determine optimized paths between relations) or simplifying database transaction management (e.g., by reducing a number of two-phase commit operations).
However, problems can arise in trying to apply rules, particularly collections of rules, to a data set. For example, individuals who may understand the practical implications of rules may find it difficult to develop a technical implementation of the rule. In addition, errors can arise if rules are not correctly implemented, including accounting for possible overlap between rules. Accordingly, room for improvement exists.
The present disclosure provides technologies that can be used to implement collective (or composite) data rules (for example, a data quality rule, such as for use in evaluating master data of an entity), where a collective data rule includes a plurality of individual data rules. An individual data rule (such as an association rule) can be generally of the form:
A rule can be considered to be in scope if the antecedents for the rule are satisfied. A rule can be considered to be satisfied, or valid, if the rule is in scope and all expected consequents of the rule are satisfied. In some cases, rather than being used to determine whether particular data is valid (e.g., satisfies the rule), a rule can be used to set values.
For example, consider a rule: Field_A=X, Field_B=1→Field_C=XYZ. If particular data being analyzed has Field A=X and Field B=1, then the rule is in scope. If a value has been assigned for Field C, the rule can be determined as satisfied if the value is “XYZ,” and not satisfied otherwise. Or, if no value has been assigned to Field C, but the antecedents of the rule are satisfied, “XYZ” can be assigned to Field C. In some cases, if a rule is not satisfied for particular data, the data can be updated so that is has values corresponding to the consequents for the rule.
Typically, the individual data rules will have some relationship with each other. In some cases, two (or more) rules can be considered to be related if there is an overlap between rule consequents, rule antecedents, or both. Rules can be considered related for other reasons, such as an observed correlation between rules (e.g., statistically, it is observed that if Rule A is satisfied, that Rule B is satisfied, at least to a threshold amount, or that the rules are negatively correlated, such that if Rule A is satisfied, then Rule B is not satisfied, at least to a threshold amount), based on user input, or based on a data model. For example, if a data model represents a particular analog world object, it may be known that two attributes have a semantic relationship, even if those two attributes do not show up in the same rule.
Once individual data rules have been defined or obtained, such as using a machine learning technique, such as association rule mining, a plurality of semantically related rules can be selected for inclusion as a collective data rule. In some cases, individual data rules in a collective data rule can represent various possible value combinations for fields that are included as antecedents of one or more rules. For example, assume two fields are to be included as rule antecedents in individual data rules, and each field has five possible values.
In this case, there are twenty-five possible combinations for the values of A and B, and therefore a maximum of twenty-five possible rules. However, in practice, there could be a smaller number of rules, as, for example, some values of A (or B) maybe not be correlated with a value of B (or A), or, some combinations of A and B may not occur, such as not representing combinations that are found in the analog world.
Note that in some cases, rules can be combined, such as if the value of a possible antecedent does not really matter in terms of consequents that might be associated with a given data item. For instance, in the example above, it could be that a particular value of A determines whether the rule is in scope (and thus the one or more consequents associated with the rule) no matter the value of B. In this case, the rule could be solely expressed in terms of A. In any event, all or a portion of rules involving A, B, or a combination of A and B may be considered for a collective data rule.
One issue that can arise from rule discovery procedures is that manual implementation of trends identified can be overly general. For example, a user may specify a consequent portion of a rule to be implemented on a data set without checking to see whether individual members of the data set should have that rule in scope. If the rule should not be in scope for a particular data item, a value of that data item may be incorrectly modified, or identified as being erroneous even though it is not.
Particularly if a large number of individual data rules or collective data rules exist, or if a collective data rules includes a large number of data rules, issues can arise in implementing data rules. Rules may be implemented in a manner that incorrectly identifies errors, or results in values being changed incorrectly. For example, if a more general rule is set to take effect before a more specific rule, a value for the general rule might be assigned to a data item that also meets the more specific rule, and where the more specific rule assigns a different value. As described above, in any event, implementing collective data rules can generally be time consuming and error prone for other reasons, and rules simply may not be implemented at all if the users who find and understand the rules are not technically capable of implementing the rules.
Disclosed technologies can provide advantages by separately determining whether a rule is in scope and determining whether the rule is satisfied. A report can be provided that indicates, for a collective data rule, for what amount (e.g., percentage) of the data set the rule was in scope, for what amount the rule was satisfied, and for what amount the rule was not satisfied. Similar information can optionally be provided regarding individual data rules in a collective data rule. Understanding the relevance of data rules, individual or collective, to a data set can help a user in evaluating what rules should be applied to give data set. Among other things, removing or disabling irrelevant rules can save computing resources.
In particular examples, the disclosed technologies include generating one or more artifacts, such as data objects useable by a computer, useable in implementing a collective data rule. Generally, these artifacts can be classified as relating to the scope of a collective data rule or a condition (or validity, or assignment of value) of a data quality rule. Scope artifacts can include a scope table (or another data structure that can store information, and be evaluated, in a similar manner as a table) where rows represent individual data rules in the collective data rule and have values (including NULL or wildcard values, in at least some implementations) for rule antecedents. Each row can include a value that is assigned during data evaluation if the antecedents for that row are satisfied. The value can be a Boolean value that indicates whether that particular individual data rule is in scope for that data item. In at least some cases, the Boolean value can be assigned to an indicator for the collective data rule instead of, or in addition to, assigning the value for an individual data rule. That is, typically a collective data rule will be considered to be in scope if at least one individual data rule in the collective data rule is in scope.
The Boolean value associated with the scope table artifacts can be used in a scope expression. The scope expression can be used in determining whether the collective data rule is in scope for a data item. Assume that the scope table determines a value of a Boolean scope variable SCOPE. A Boolean expression can have the form of IF SCOPE==TRUE, THEN CollectiveDataRule.Scope=TRUE.
Similar artifacts, a table artifact and an expression artifact, can be generated for conditions (or consequents) associated with a collective data rule. The condition table can be structured in a similar manner as the scope table. However, the condition table includes one or more columns for values that are expected if the rule antecedents are satisfied (e.g., if the particular rule is in scope). Individual data rules, corresponding to rows of the condition table, can also be associated with a Boolean value, which can be used by the condition expression to provide an indication as to whether the collective data rule (or optionally an individual data rule in the collective data rule) is satisfied by the data element. In some cases, the Boolean variable can be set to FALSE, such that FALSE is the value returned unless it is changed to TRUE as a result of satisfying a row of the condition table.
These relationships can be mined using a rule mining algorithm 112, which can be an algorithm, such as a machine learning algorithm, associated with an analytics library 108. Suitable algorithms include Apriori, ELCAT, and FP-growth. However, other algorithms can be used for identifying relationships between attributes and attribute values. Executing the rule mining algorithm on at least a portion of the tables 104 can provide one or more mined individual data rules 116. As explained in Example 1, individual data rules typically have one more antecedents 118 and one or more consequents 120.
Although
In at least some cases, individual data rules 116 can be associated with result statistics 122. Result statistics 122 can provide information about the accuracy of an individual data rule 116, such as in the tables 104 used for rule mining, or in a test or sample set of tables to which the rules will be applied or another sample set. The result statistics 122 can include a value 124 indicating an amount (e.g., a percentage) of data that does not satisfy the rule, a value 126 indicating an amount (e.g., a percentage) of data that satisfies the rule, and a value 128 indicating an amount (e.g., a percentage) of data for which the rule is not in scope. In some cases, a user interface display can render the results statistics 122 in a stacked bar or column format, including, in some cases, color coding or otherwise visually distinguishing the values 124, 126, 128.
One or more, typically a plurality, of individual data rules 116 can be selected for inclusion in one or more collective data rules 138. The collective data rules 138 can include, or include references to, their constituent individual data rules 116. In some implementations, a user can manually select individual data rules 116 to be included in a collective data rule 138. In other cases, all or a portion of the individual data rules 116 for a collective data rule 138 can be automatically selected for inclusion in the collective data rule, at least initially. That is, a user may review, and optionally modify or delete, any putative collective data rules 138 that might have been automatically been selected or generated.
In cases where all of a portion of one or more collective data rules 138 are determined automatically, construction of the collective data rules can be carried out by a rule generation engine 142. The rule generation engine 142 can construct collective data rules 138 using one or both of rule criteria 146 and a data model 148. Rule criteria 146 can include templates or criteria for selecting individual data rules 116 to be included in a collective rule 138. Criteria can include, for example, criteria for determining when two individual rules 116 are sufficiently related to be included in a collective data rule 138, such as having a threshold number (e.g., one, or a plurality) of antecedents in common, having a threshold number (e.g., one, or a plurality) of consequents in common, being from the same table or related tables etc.
Relatedness of tables, or of attributes between or within tables, can be determined with respect to the data model 148. The data model 148 can be a representation of relationships between the tables 104, such as showing relationships based on foreign keys, alternate keys, or associations between tables, or using information such as database triggers or views to determine how tables and their attributes are related. The data model 148 can be, or can be based at least in part, a data dictionary or information schema associated with a database system.
In further implementations, the data model 148 can include information regarding computing objects (e.g., abstract data types) associated with the tables 104, such as data objects associated with the tables via object relational mapping. For example, if a given table 104 includes 20 attributes, and 5 are included in a particular data object (e.g., representing a product being produced via a production process), that information can be used in determining whether individual data rules 116 that do or do not include those 5 attributes should be considered for inclusion in the same collective data rule 138. Or, relationships between such data objects can be used to determine whether relationships should be inferred between the attributes used in such related data objects (e.g., a product table is known to be related to a material table, which may in turn be related to a supplier table).
The antecedents 118 and consequents 120 of the individual data rules 116 in a collective rule 138 are typically associated with information 156 regarding their source and which can be used to identify them. The information 156 can include a table identifier 158, identifying the table 104 associated with the antecedent 118 or consequent 120. The information 156 can also include an identifier 160 for a field associated with the antecedent 118 or consequent 120, and a value (or optionally, plural values) 162 associated with each field 160.
The information 156 for the individual data rules 116 in a collective data rule 138 can be used to construct various rule artifacts 166. The rule artifacts 166 can include a scope expression 170, a scope decision table 172, a condition expression 174, and a condition decision table 176. The scope decision table 172 and the condition decision table 176 are typically created for a particular collective data rule 138. Correspondingly, the scope expression 170 and the condition expression 174 are typically specified with respect to the corresponding scope decision table 172 and the condition decision table 176, respectively.
The scope expression 170 and the condition expression 174 are also evaluated with respect to their respective scope decision table 172 and condition decision table 176. That is, the scope expression 170 and the condition expression 174 can be constructed as conditional statements that evaluate to TRUE or FALSE depending on analysis results of the corresponding decision table 172 or 176.
The tables 172, 176 can be generated, in some examples, by populating the antecedents 118 or the consequents 120 of individual data rules 116 in a collective data rule 138 into a suitable programming language, such as SQL. For example, a program can be written having a command such as:
After the collective rules 138 are defined, and the corresponding rule artifacts 166 are created, one or more of the collective rules can be used to evaluate a data set, such as all or a portion of the database tables 104. Evaluation results 184 can be provided in an evaluation report 180. The evaluation results 184 can include one or both of result statistics 188 for a given collective data rule 138 and result statistics 192 for given individual data rules 116 in the given collective data rule, where the result statistics 188, 192 can be at least generally similar to the result statistics 120.
The antecedents 214 correspond to different values of the MTART attribute of the MARA table. A given individual data rule 212 is in scope if the value of MTART for a particular data item is equal to the antecedent 214 for that rule. In a given individual data rule 212, the antecedent 216 provides the value expected for a particular data item. As shown, the antecedents 216 correspond to different values of the MATKL attribute.
Note that, in the collective data rule 208, the collective data rule is in scope as long as the antecedent 214 of any individual data rule 212 is satisfied. Also, the individual data rules 212 can be considered to be non-overlapping, in that only a single individual data rule will be in scope at a time, and thus only a single consequent 216 will be active/possible if the collective data rule 208 is in scope. However, even though the antecedents 214 of the individual data rules 212 are unique, the consequents 216 of the individual data rules are not unique.
The antecedents 214 of the individual data rules 212 in the collective data rule 208 can be automatically extracted and used to populate a scope decision table 224. For example, the individual data rules 212 can be parsed, and each field that serves as an antecedent 214 in an individual data rule can be used in a data definition statement (e.g., in SQL) used to create or modify the scope decision table 224. Although in the illustrated example, the individual data rules 212 only include a single antecedent 214, the same procedure can be applied when the individual data rules have multiple antecedents, including when antecedents differ between different individual data rules.
The antecedents 214 corresponding to each individual data rule 212 can be inserted as rows 226 in the scope decision table 224, with the value of the antecedent inserted as the value for the column corresponding to the antecedent. In the event that not all individual data rules 212 have values for all antecedents 214 in the scope decision table 224, the corresponding cell(s) can be left empty, or a NULL value or similar value provided in the cell to indicate that a particular antecedent is not used for a particular individual data rule.
The scope decision table 224 can include a column 230a that is not correlated with a particular antecedent 214, but provides a value that can be used when the scope decision table is evaluated, such as a value that can be assigned to a variable that represents an outcome of the scope decision table, indicating whether the collective data rule 208 is in scope.
The scope decision table 224 can include a row 226a that corresponds to a default result that will apply if a particular data item does not match any other row in the scope decision table (e.g., the values of the data item do not match the antecedents 214 for any of the individual data rules 212 in the collective data rule 208). In some cases, the value in row 226a of the column 230a can be a value indicating that the collective data rule 208 is not in scope, including a value (including a lack of a value) that does not change a default value of a variable that is associated with a result indicating whether the collective data rule is in scope. For example, such a variable may initially have a value of FALSE, and if the row 226a applies, the value remains FALSE, and if another row 226 is satisfied, the value is changed to TRUE.
A scope expression, such as scope expression 240a or scope expression 240b, can be configured to evaluate the scope decision table 224. Scope expression 240a is configured to evaluate rows 226 of the scope decision table 224, and the scope expression 240a will in turn provide an evaluation result based on the evaluation of the scope decision table. That is, typically a scope expression, such as scope expression 240a, will be configured to return a Boolean value, or assign a Boolean value to a variable, based on the evaluation of the scope decision table 224. In a particular example, rows 226 of the scope decision table 224 are evaluated, such as being sequentially evaluated. However, if desired, multiple rows 226 can be concurrently evaluated.
Evaluation of a row can include evaluating values for columns of the scope decision table 224 corresponding to operands (i.e., values for antecedents 214 for particular individual data rules 212 corresponding to rows of the scope decision table). If the data item being evaluated matches all of the operand values for a given row 226, the value provided in the column 230a can be provided as an evaluation result for evaluation of the scope decision table 224. However, other implementations are possible, such as omitting the column 230a and using program logic to assign (or not assign) a value based on the evaluation of a row 226.
In some cases, evaluation of rows 226 continues until a matching row is found, which can include the row 226a. In other implementations, the row 226a is omitted, and, if no match is found, a FALSE value is assigned or maintained to indicate that the collective data rule 208 is not in scope for the particular data item (or collection of data items, such as a collection of data items having common values for the antecedents 214 of the individual data rules 212 in the scope decision table 224).
In the case that evaluation of the scope decision table 224 is only used to determine whether a collective data rule is in scope, evaluation of the scope decision table can terminate once a row 226 matching values for a data item being evaluated has been identified. Scope expression 240a represents this scenario, as only an overall value for the scope decision table 224 is used. In another implementation, the scope expression 240b causes information to be stored regarding which individual data rule was in scope in the scope decision table 224. This information can be useful, such in specifying an individual data rule 212 to be evaluated for correctness (e.g., if the consequents of the rule are consistent with a particular data item being evaluated), or for tracking statistics as to which individual data rules are in scope for a given data set or collective data rule 208.
In some cases, different individual data rules 212 can have different specificities. That is, consider two rules, one having a single antecedent 214 of FIRST=X, and a second rule having the single antecedent of FIRST=X and a second antecedent of SECOND=Y. A data item having FIRST=X and SECOND=Y will satisfy both rules. In some cases, the scope decision table 224 can be structured such that rules having overlapped antecedent values are ordered in a particular way, such as having narrower rules before broader rules, or having broader rules before narrower rules.
If the scope decision table 224 is being evaluated simply to determine if any rule is in scope, then it can be beneficial to include broader rules before narrower rules, as processing of the scope decision table 224 can be faster and more efficient. If the scope decision table 224 is being used to identify which individual data rule should be evaluated for consequent correctness, then it may be beneficial to include narrower rules before broader rules.
Continuing the above example, the first rule with the single antecedent 214 may have a consequent of THIRD=A, while the second rule with two antecedents may have a consequent of THIRD=B. Even though the example data item would satisfy both individual data rules 212, typically the values associated with the most specific individual data rule met by a given data item are used for evaluation/assignment.
It may be useful to structure the scope decision table 224 in a manner other than that which produces higher processing efficiencies. For example, artifacts generated for a collective data rule 208 can include a condition decision table 250. The condition decision table 250 is typically structured having narrower rules before (e.g., higher in the table) broader rules, to help ensure that the consequents 216 of the most specific rule are used for evaluation/assignment for a given data item.
It can be useful to structure the scope decision table 224 in the same manner as the condition decision table 250, so there is a correspondence between the tables and the tables are more intuitive for a user to understand. However, in some cases, the scope decision table 224 can be displayed to the user in narrowest to broadest format, like the condition decision table 250, but the version of the scope decision table used for evaluation can be structured in broadest to narrowest format. Note that when a row representing a default outcome is provided, such as row 226a, that row is typically included in the scope decision table 224 as the last row, or is otherwise evaluated last.
In cases where multiple individual data rules 212 may be simultaneously in scope, either because they have overlapping antecedents 214 or because they include different antecedents (e.g., one rule uses FIRST and another uses SECOND), evaluation of the scope decision table 224 can include tracking individual data rules that are in scope (e.g., rows whose operands are matched). For instance, each row 226 (other than row 226a, in some cases) can be associated with a value, such as a Boolean value for a Boolean variable, indicating whether the associated individual data rule 212 is in scope for the particular data item being evaluated. Information about individual data rules 212 and collective data rules 208 that are in scope can be presented in a report provided regarding the analysis of a data set using one or more collective data rules.
The condition decision table 250 can be also be structured with rows 254 corresponding to individual data rules 212 in the collective data rule 208. If desired, some of the information in the scope decision table 224 can be omitted from the condition decision table 250. For example, the scope decision table 224 is shown as having a column 230b that can be used to reflect a data selection condition that is not necessarily included in an individual data rule 212 as an antecedent 214.
In at least some cases, an equivalent column to the column 230b is not included in the condition decision table 250. In particular, such a column can be omitted because the condition decision table 250 is typically only evaluated for data items that are already known to be in scope, including satisfying any data selection conditions. That is, the condition decision table 250 may only be evaluated for those data items where a positive result was returned during evaluation of the scope decision table 224 with respect to that data item.
Otherwise, the condition decision table 250 typically includes columns for each antecedent 214 in an individual data rule 212 of the collective data rule 208, or at least those antecedents whose value makes a difference in a value of a consequent 216 that is to be assigned or checked for consistency with an individual data rule. As illustrated, the individual data rules 212 include a single antecedent 214, and so the condition decision table 250 includes a column 258a for this antecedent, where each row 254 of the condition decision table includes different values for this antecedent (and more generally, for tables with columns for multiple antecedents, a value of at least one antecedent differs between any two rows of the condition decision table).
While the scope decision table 224 included a column 230a having a value indicating whether an individual data rule 212 associated with a particular row 226 was in scope (i.e., its antecedents 214 were met), the condition decision table 250 can include a column 258b providing values for a consequent 216 (or multiple consequents) associated with the corresponding individual data rule. In the case where one or more individual data rules 212 in the collective data rule 208 are associated with multiple consequents, the condition decision table can include a column (e.g., similar to column 254b) for each consequent. In some implementations, a condition decision table 250 can include a column analogous to column 230 of the scope decision table 224.
A condition expression, such as condition expression 260a, 260b, or condition 260c, can be evaluated by evaluating the condition decision table 250. Evaluation of the condition expression 260a, 260b, or 260c can include evaluation to determine whether values of a data item being evaluated are consistent with expected values based on the individual data rule 212 satisfied by the data item, or to assign values to a data item based on the individual data rule satisfied by the data item. Although typically collective data rules 208 are structured so that only a single individual data rule 212 of a collective data rule will be satisfied by any given data item, in some cases a collective data rule can include multiple individual data rules 212 that can independently and simultaneously be satisfied by a data item. In such cases, a given data item can be checked for consistency with multiple rows 254 of the condition decision table, or values optionally assigned or suggested based on values of the consequent columns, such as column 250b.
In a similar manner as the scope expressions 240a, 240b, the condition expressions 260a, 260b, 260c can analyze and return different information. Condition expression 260a is shown as returning a value of TRUE if the values for the antecedent column 258a and the consequent column 258b are met. Condition expression 260b includes the same information as condition expression 260a, but also returns an identifier for the particular individual data rule 212 that was satisfied by the data item being evaluated. Condition expression 260c includes the same information as condition expression 260b, but also includes the expected consequent value associated with the corresponding individual data rule 212. In the case where multiple individual data rules 212 can simultaneously be met, condition expressions 260b, 260c can include information for such multiple individual data rules.
Typically, rows of the condition expression table 250 are evaluated until a matching row 254 is identified. In other cases, such as if multiple rows 254 might match a given data item, all rows of the condition expression table can be evaluated. In a yet further example, one or more rows of the scope expression table 224 that are satisfied can include identifiers for rows of the condition decision table 250 associated with the corresponding individual data rule 212, and the relevant rows of the condition decision table evaluated.
If the scope expression table 224 is used in conjunction with the condition expression table 250, it is typically known that at least one row 254 of the condition expression table 250 is satisfied, and thus a row corresponding to the row 226a of the scope expression table 224 need not be included. However, in other cases, a row similar to row 226a can be included in the condition expression table 250, including to account for situations where scope expression table 224 or the condition expression table 250 were configured incorrectly.
As describe above, the condition expression table 250 is typically structured, or at least evaluated, such that narrower individual data rules 212 are evaluated before broader data rules.
Although described as implemented in two tables, in some cases functionality of the scope decision table 224 and the condition decision table 250 can be combined in a single table. Typically, such a table includes columns for antecedents and consequents in individual data rules in a collective data rule. Program logic can, among other things, assign values for data item being in scope, and whether the conditions of a relevant individual data rule are satisfied for the data item.
The data rule listing can also include descriptions 316 for individual data rules 312. The descriptions 316 can list antecedents and consequents in the rule, and a relationship between the antecedents and consequents, if appropriate. Typically, the relationship is an if/then type relationship. An indication of the focus area 318 of individual data rules 312 can also be provided, where the focus area can represent conditions, such as filter conditions, under which the rules have been found, which in turn can relate to validity statistics for the rule. That is, data may have been filtered by the focus area 318 when rule mining was conducted, and so the existence of the rule, as well as features such as support and confidence, and a proportion of data to which the rule applies, may be valid so long as data is consistent with the focus area. In some cases, a given individual data rule 312 may be valid outside of the focus area 318, but the validity outside of the focus area may need to be separately established
In addition to helping a user understand when individual data rules 312 may be valid, providing an indication of the focus area 318 can assist a user in identifying individual data rules that might have some interrelationship such that they should be considered for inclusion in the same collective data rule (or instead should be included in different collective data rules, or not included in any collective data rules). That is, it may be useful to include multiple data individual data rules that have the same, or overlapping, focus areas 318 in the same collective data rule.
The listing of individual data rules 312 can include summary display elements 320 that provide information regarding the applicability of individual data rules. For example, the display elements 320 may provide a visual (e.g., a graphical, such as in a stacked column/bar graph) indication of applicability statistics for a given individual data rule. The applicability statistics can include one or more of a proportion of a data set for which the individual data rule 312 was found to be in scope, a proportion of the data set for which the individual data rule was found not to be in scope, a proportion of the data set for which the individual data rule was in scope and valid, or a proportion of the data set for which the individual data rule was in scope and was not valid. In particular examples, the visual display elements 320 display a portion of a data set for which the given individual data rule 312 was in scope and a proportion of the data set for which the individual data rule was in scope and valid, or displays this information and additionally displays a proportion of the data set for which the individual data rule was in scope and not valid. Providing the display element 320 can help a user determine the usefulness of a particular individual data rule 312, and whether such rule should be included in a collective data rule for a given data set.
Information provided in the screen 300 for individual data rules 312 can include at least one checked field 322 and checked field value 324 for the rule; that is, one or more fields serving as antecedents and the value of that antecedent in the given rule. The information can further include an indicator 326 indicating whether the given rule has been accepted or not (e.g., a rule might be proposed through automated rule mining, but a user or another process may need to determine whether the rule is actually meaningful/should be made available for use). An indicator 328 can be provided, indicating whether the individual data rule 312 is associated with/assigned to one or more collective data rules.
Each individual data rule 312 can be associated with a selection box 332, which can be used to select a rule to have various actions taken with respect to the rule. A selection box 334 can be provided to select all, or all displayed, individual data rules. Actions to be taken can be triggered via respective user interface controls to accept a rule 338, reject a rule 340, marked a rule for later review 342, to set a rule to an initial state 344 (e.g., such as to clear or reset any previously assigned status, such as accepted, rejected, or marked for review), to link 346 an individual data rule 312 to a collective data rule (either an existing collective data rule, or a collective data rule to be generated), or to delete 348 an individual data rule.
The screen 400 can provide navigation icons 416 for navigating to various areas of the screen, such as an icon 418 to navigate to a general information area, an icon 420 to navigate to an area that provides usage information, an icon 422 to navigate to an area that provides implementation information (as described below, scope and condition expression artifacts), an icon 424 to navigate to an area that describes dimensions associated with the collective data rule, an icon 426 to navigate to an area that provides rule mining implementation details (as described below, scope and condition decision table artifacts), an icon 428 to navigate to an area that provides information regarding individual data rules from rule mining operations (such as those that are currently included in the collective data rule), and an icon 430 to navigate to an area that includes administrative data.
Dimensions associated with the icon 424 can represent different subcategories used in assessing collective data rules. That is, different collective data rules may be associated with different assessments of data (e.g., completeness, accuracy) or categories of data (e.g., material data, supplier data). In some cases, different dimensions for a different category can be associated with different weightings towards an overall score, such as weighting collective data rules associated with “material data” higher than collective data rules associated with the “supplier data” dimension. In further implementations, analogous weightings can be applied to categories of individual data rules.
A general information area 434 can provide additional basic information about a collective data rule, such as information providing addition description regarding the collective data rule (e.g., a summary of what the collection of individual data rules in the collective data rule is intended to probe), a reason for the collective data rule (e.g., what is indicated by information indicating whether a data item does or does not comply with the collective data rule, or a degree of compliance for a collection of data items in a data set), information regarding the scope of the collective data rule (e.g., for the one or more base tables 404, what fields and field values cause the collective data rule to be in scope), and a link to where additional details regarding the collective data rule may be viewed. The general information area 434 can also provide information 436 for contacts associated with the rule, such as an owner designated for the rule, an individual designated to attend to issues with implementing the collective data rule, a general contact, and an identifier associated with an individual responsible for data being evaluated by the rule.
A usage area 438 can provide information and actions related to use of the collective data rule. In some cases, a collective data rule can be used for multiple purposes. Identifiers 440 can be provided for each such purpose, as well as a selection control 442 that can be used to indicate whether the collective data rule is currently selected for, or active for, such use. Each purpose (e.g., corresponding to an identifier 440) can be associated with a status 444, such as whether the collective data rule is available or ready to be used for the associated purpose.
UI controls for available actions for a given purpose can be presented in the usage area 438. For example, a control 446 can allow a user to prepare a collective data rule for a particular use. Preparing a collective data rule for use can include automatically generating computing artifacts to be used in implementing the collective data rule for the use, such as generating one or more of a scope decision table, a scope expression, a condition decision table, or a condition expression.
An implementation area 450 can provide details regarding artifacts used in implementing a collective data rule, including an identifier for the condition expression and an identifier for the scope expression, as well as a status associated with such expressions. That is, for example, a status may be used to indicate whether a given expression has been generated.
The user interface screen 400 can include controls for causing a collective data rule to be activated, such as to cause a data set to be evaluated by the collective data rule. An approve control 452 can allow a user to indicate that the collective data rule has been approved, and is ready to be executed. A send for implementation control 454 can allow a user to send the collective data rule to a collective data rule execution system, such as a system where collective data rules can be scheduled for execution against a data set or manually set for execution.
That is, the rule mining area 470 provides an identifier 472 for a scope decision table associated with the collective data rule and an identifier 474 for a condition decision table associated with the collective data rule. Status information, such as active, not ready, inactive, and the like, can be provided for the scope decision table and the condition decision table. If the decision table artifacts have not yet been created, the rule mining area 470 can appear similar to how the implementation area 450 appears in
An area 480 can list information regarding individual data rules 482 associated with the collective data rule, which information can be generally similar to the information presented in the user interface screen 300 of
Automatic implementation, in some cases, can be limited to individual data rules having specific characteristics. For example, automatic implementation might be indicated as supported it the individual data rule has features such as one or more of a threshold confidence value, a threshold support value, a threshold number of antecedents, a threshold number of consequents, a threshold value for the rule being in scope with respect to a given data set, and a threshold value for the rule being in scope and satisfied for a given data set. If a rule cannot be automatically implemented, extra approval or configuration may be needed before the rule can be implemented. In some cases, a user can adjust an individual data rule, such as altering its antecedents or consequents, in order to put the rule in condition for implementation.
An administrative data section 498 can provide information such as a date and time the collective data rule was created, a date and time the collective data rule was modified, or identifiers of users or processes that created or modified the collective data rule.
The user interface screen 500 lists an identifier 520 for the relevant scope expression for the data quality rule, and includes a summary of actions 524 that can be taken if the rule is in scope (e.g., determine whether particular data complies with the rule, and return TRUE/FALSE), and a summary of actions 528 that can be taken if the rule is not in scope (e.g., return an indication that the rule is out of scope for particular data). The actions 524, 528 can include actions that are used to generate execution results that can be returned to a user, such as information useable to determine a percentage of data having a rule in scope, in scope and valid, in scope and invalid, or summarizing actual values that exist for data items for which the collective data rule was in scope, but invalid. In particular examples, the actions 524, 528 can be automatically generated when a user selects to implement a collective data rule.
The identifier 520 for the scope expression can indicate a particular scope expression artifact, which in turn can reference a scope decision table. Actions 524, when a rule is in scope, can refer to a condition expression artifact, such as the artifact shown in
A selection of a second plurality of individual data rules of the first plurality of individual data rules is received at 808, where the second plurality of individual data rules are to be associated with a collective data rule. At 812, at least one collective data rule artifact is automatically generated at least in part from at least a portion of the antecedents, consequents, or a combination thereof, of individual data rules of the second plurality of individual data rules.
At 1012, one or more data definition language statements are automatically executed to generate a second table. The second table has a plurality of rows. A given row corresponds to a data rule of the plurality of data rules and includes rule antecedent field values and rule consequent values for the given data rule. A first condition expression is automatically generated at 1016. The first condition expression is configured to return a first value if a data item corresponds to a row of the first table corresponding to a data rule and a second value otherwise. At 1020, a second condition expression is automatically generated. The second condition expression is configured to return the first value if a data item corresponds to a row of the second table corresponding to a data rule and the second value otherwise.
With reference to
A computing system 1100 may have additional features. For example, the computing system 1100 includes storage 1140, one or more input devices 1150, one or more output devices 1160, and one or more communication connections 1170. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 1100. Typically, operating system software (not shown) provides an operating environment for other software executing in the computing system 1100, and coordinates activities of the components of the computing system 1100.
The tangible storage 1140 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information in a non-transitory way and which can be accessed within the computing system 1100. The storage 1140 stores instructions for the software 1180 implementing one or more innovations described herein.
The input device(s) 1150 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system 1100. The output device(s) 1160 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing system 1100.
The communication connection(s) 1170 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use an electrical, optical, RF, or other carrier.
The innovations can be described in the general context of computer-executable instructions, such as those included in program modules, being executed in a computing system on a target real or virtual processor. Generally, program modules or components include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Computer-executable instructions for program modules may be executed within a local or distributed computing system.
The terms “system” and “device” are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on a type of computing system or computing device. In general, a computing system or computing device can be local or distributed, and can include any combination of special-purpose hardware and/or general-purpose hardware with software implementing the functionality described herein.
In various examples described herein, a module (e.g., component or engine) can be “coded” to perform certain operations or provide certain functionality, indicating that computer-executable instructions for the module can be executed to perform such operations, cause such operations to be performed, or to otherwise provide such functionality. Although functionality described with respect to a software component, module, or engine can be carried out as a discrete software unit (e.g., program, function, class method), it need not be implemented as a discrete unit. That is, the functionality can be incorporated into a larger or more general-purpose program, such as one or more lines of code in a larger or general-purpose program.
For the sake of presentation, the detailed description uses terms like “determine” and “use” to describe computer operations in a computing system. These terms are high-level abstractions for operations performed by a computer, and should not be confused with acts performed by a human being. The actual computer operations corresponding to these terms vary depending on implementation.
The cloud computing services 1210 are utilized by various types of computing devices (e.g., client computing devices), such as computing devices 1220, 1222, and 1224. For example, the computing devices (e.g., 1220, 1222, and 1224) can be computers (e.g., desktop or laptop computers), mobile devices (e.g., tablet computers or smart phones), or other types of computing devices. For example, the computing devices (e.g., 1220, 1222, and 1224) can utilize the cloud computing services 1210 to perform computing operators (e.g., data processing, data storage, and the like).
Although the operations of some of the disclosed methods are described in a particular, sequential order for convenient presentation, it should be understood that this manner of description encompasses rearrangement, unless a particular ordering is required by specific language set forth below. For example, operations described sequentially may in some cases be rearranged or performed concurrently. Moreover, for the sake of simplicity, the attached figures may not show the various ways in which the disclosed methods can be used in conjunction with other methods.
Any of the disclosed methods can be implemented as computer-executable instructions or a computer program product stored on one or more computer-readable storage media, such as tangible, non-transitory computer-readable storage media, and executed on a computing device (e.g., any available computing device, including smart phones or other mobile devices that include computing hardware). Tangible computer-readable storage media are any available tangible media that can be accessed within a computing environment (e.g., one or more optical media discs such as DVD or CD, volatile memory components (such as DRAM or SRAM), or nonvolatile memory components (such as flash memory or hard drives)). By way of example, and with reference to
Any of the computer-executable instructions for implementing the disclosed techniques as well as any data created and used during implementation of the disclosed embodiments can be stored on one or more computer-readable storage media. The computer-executable instructions can be part of, for example, a dedicated software application or a software application that is accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed, for example, on a single local computer (e.g., any suitable commercially available computer) or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.
For clarity, only certain selected aspects of the software-based implementations are described. Other details that are well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any specific computer language or program. For instance, the disclosed technology can be implemented by software written in C, C++, C#, Java, Perl, JavaScript, Python, R, Ruby, ABAP, SQL, XCode, GO, Adobe Flash, or any other suitable programming language, or, in some examples, markup languages such as html or XML, or combinations of suitable programming languages and markup languages. Likewise, the disclosed technology is not limited to any particular computer or type of hardware. Certain details of suitable computers and hardware are well known and need not be set forth in detail in this disclosure.
Furthermore, any of the software-based embodiments (comprising, for example, computer-executable instructions for causing a computer to perform any of the disclosed methods) can be uploaded, downloaded, or remotely accessed through a suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.
The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Instead, the present disclosure is directed toward all novel and nonobvious features and aspects of the various disclosed embodiments, alone and in various combinations and sub combinations with one another. The disclosed methods, apparatus, and systems are not limited to any specific aspect or feature or combination thereof, nor do the disclosed embodiments require that any one or more specific advantages be present or problems be solved.
The technologies from any example can be combined with the technologies described in any one or more of the other examples. In view of the many possible embodiments to which the principles of the disclosed technology may be applied, it should be recognized that the illustrated embodiments are examples of the disclosed technology and should not be taken as a limitation on the scope of the disclosed technology. Rather, the scope of the disclosed technology includes what is covered by the scope and spirit of the following claims.