Patent Grant
7818311
Regular expressions, or more generally patterns, describe sets of character strings. The pattern determines character strings that belong to the set. Accordingly, patterns can be employed to identify character strings, for example, to select specific strings from a set of character strings. Furthermore, regular expressions are often defined as a context-independent syntax that can represent a wide variety of character sets and character set orderings.
In operation, regular expressions can be employed to search and match data as a function of a predefined pattern or set of patterns. As such, patterns employ a specific syntax by which particular characters or strings are selected from a body of text. More specifically, the expressions can consist of constants and operators that denote sets of strings and operations over these sets, respectively. Using the specific syntax of a regular expression or other pattern language, advanced text pattern matching can be performed. Table 1 that follows lists exemplary regular expression operators and their definitions. The syntax illustrated in the table is frequently employed to establish both simple and complex string pattern identifications.
Regular expressions are a useful tool many areas. For example, regular expressions are utilized by compilers to identify tokens and otherwise translate computer-programming code. Similarly, code completion and/or highlighting systems utilize regular expressions in integrated development environments. Regular expressions are also useful in the data flow field, which pertains to the movement and transformation of data to and amongst storage mediums.
The following presents a simplified summary in order to provide a basic understanding of some aspects of the claimed subject matter. This summary is not an extensive overview. It is not intended to identify key/critical elements or to delineate the scope of the claimed subject matter. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
Briefly described, the subject disclosure pertains to regular expressions and construction thereof. Regular expressions are a powerful way to search for patterns within text streams. However, complex patterns such as those associated with constructs of programming languages can be overly burdensome, if not nearly impossible, for programmers to specify directly.
In accordance with an aspect of the disclosed subject matter a mechanism is provided allow complex patterns to be composed of a plurality of simpler patterns. More specifically, complex regular expressions can be generated automatically as a function of a collection of simpler rules. Subsequently, a regular expression engine can be fed the regular expression to enable pattern matching based thereon.
To the accomplishment of the foregoing and related ends, certain illustrative aspects of the claimed subject matter are described herein in connection with the following description and the annexed drawings. These aspects are indicative of various ways in which the subject matter may be practiced, all of which are intended to be within the scope of the claimed subject matter. Other advantages and novel features may become apparent from the following detailed description when considered in conjunction with the drawings.
Systems and methods are provided with respect to facilitating pattern matching utilizing regular expressions. Rather than forcing users to attempt to specify complex regular expressions directly, they can be composed utilizing a set of simpler rules. These rules can then be transformed to a complex regular expression automatically removing the burden from users. Subsequently, the regular expression can be provided to a regular expression engine for matching against a set of textual data.
Various aspects of the subject disclosure are now described with reference to the annexed drawings, wherein like numerals refer to like or corresponding elements throughout. It should be understood, however, that the drawings and detailed description relating thereto are not intended to limit the claimed subject matter to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the claimed subject matter.
Referring initially to
The regex engine 110 provides textual pattern matching as a function of an input regular expression. The regex engine 110 can be either a text-directed engine or a regex-directed engine, wherein a text-directed engine is a deterministic finite automation (DFA) and a regex-directed engine is a non-deterministic finite automation (NFA). In either instance, the regex engine 110 receives, retrieves, or otherwise obtains as input a regular expression and a string of textual data for which to identify matches. Regular expressions comprise a plurality of normal characters and operators that describe a set of one or more strings in the form of an expression or pattern. The regex engine 110 utilizes the regular expression to process input text. By way of example, consider an overly simplified scenario where the input text is a string “exemplary test string” and the pattern corresponds to “test.” In this case, the regex engine 110 will search the string incrementally (e.g., “e,” “x,” “e,” “m,” “p,” “l,” “a,” “r,” “y”) until it finds the first letter “t.” Subsequently, the engine will look for the remaining letters “est” to ultimately match the pattern.
In practice regular expressions can be quite complex. Moreover, programmers are conventionally responsible for direct specification of regular expressions. Specifying complex patterns such as constructs of a programming language is thus nearly impossible. The rule compilation component 120 provides a solution to this dilemma.
As input, the rule compilation component 120 receives, retrieves or otherwise obtains or acquires rules. Rather than requiring direct specification, complex regular expressions can be provided as a set or collection of simpler rules. In one instance, each rule can comprise a name definition pair. Furthermore, higher-level rules can be defined in terms of lower-level or primitive rules. The rule compilation component 120 produces a regular expression based on the specified rules. Subsequently, this generated regular expression can be provided as input to the regex engine 110.
While the regex engine 110 and rule compilation component 120 can be combined, in accordance with one aspect of the claim subject matter these components are independent. There are many benefits of separating the components and interests thereof. For example, regular expressions can still be generated without requiring specification of rules. In addition, a new engine need not be generated. The rule compilation component 120 can simply interact with conventional regex engines. Furthermore, conventional regular expression based systems need not be re-written to produce rules rather than regular expressions, for instance.
Turning attention to
The rules acquired by the interface component 210 can subsequently be provided or made accessible to the regex generator component 220. From such rules, as the name suggests, the regex generator component 220 can generate a regular expression. Although not limited thereto, according to one aspect, rules can be an extension of the regular extension language. As a result, a resulting regular expression can be generated by compiling individual regex rules into a single regular expression.
The translation component 310 can also interact with error detection component 330 to facilitate identification or rule errors. For instance, the error detection component 330 can detect circular rules or missing definitions. Upon detection, an exception, error message, and/or other like can be produced.
Further yet, optimization component 340 can be employed by the translation component 310 to facilitate optimized translation of rules to regular expressions. In one instance, the optimization component 340 can ensure that a non-terminal representation is computed at most once. Other optimizations are also possible and are to be considered within the scope of the subject disclosure. For example, various caching schemes (e.g., deferred loading, eager loading . . . ) can be employed to facilitate processing of rules and generation of regular expressions.
To facilitate clarity and understanding, consider a scenario in which a new language is developed and one needs to parse a script of multiple statements. Conventionally, a language parser would have to be coded since a regular expression mechanism would be nearly impossible to generate for a language. Now, however, it is possible to generate and employ a complex regular expression by specifying some rules.
By way of example and not limitation, Appendix A provides a tabular overview of a sample language to be parsed. In sum, the desire is to parse a script of a language with the following characteristics:
Here, rules are specified as name definition pairs delineated by triangle brackets and parenthesizes in accordance with the exemplary rule grammar. From these eighteen grammar rules, the following regular expression can be generated by the regex generation component 220 that correctly matches each statement:
This exemplary implementation utilizes an extension to the regular expression language that adds a minimum amount of new syntax. For instance, to refer to a non-terminal symbol, “\K” (capital “k”) is employed. That is similar to “\k” (lowercase “k”) which is used for backward reference. Then each rule is a standard named group pattern.
The same mechanism can also be utilized for different reasons with a different set of rules. For example, color coding and intelligent assistance can be specified in this manner. In this case, the each type of word in a script can extracted to enable rending a proper color. Below is a sample set of grammar rules that can be specified to facilitate generation of an appropriate regular expression to effect such functionality with respect to the previously described exemplary language.
Here, only thirteen rules need be specified and no code written.
Referring to
System 400 further includes assistance component 410 to aid specification of rules. The assistance component 410 adds yet another layer on top of the rule compilation component 120 to assist generation of regular expressions. For example, assistance component 410 can provide automatic completion functionality via suggestions, drop-down menus and/or the like based on current rule specification and/or regular expression grammar, among other things. The assistance component can also provide color-coding to aid rule specification and/or error detection. In one embodiment, the assistance component 410 can be an integrated development environment (IDE) and/or code editor plug-in or add-on to support development of rules.
Additionally or alternatively, it should be appreciated that the assistance component 410 can enable automatic generation of rules. For example, a wizard can be provided to acquire information from a user that the assistance component 410 can utilize to infer a set of rules. Further yet, the assistance component 410 can also interpret and/or utilize alternate representations of language grammars such as BNF (Backus-Naur Form) to help infer rules relating to parsing and/or color-coding a related language, among other things.
The aforementioned systems, architectures and the like have been described with respect to interaction between several components. It should be appreciated that such systems and components can include those components or sub-components specified therein, some of the specified components or sub-components, and/or additional components. Sub-components could also be implemented as components communicatively coupled to other components rather than included within parent components. Further yet, one or more components and/or sub-components may be combined into a single component to provide aggregate functionality. Communication between systems, components and/or sub-components can be accomplished in accordance with either a push and/or pull model. The components may also interact with one or more other components not specifically described herein for the sake of brevity, but known by those of skill in the art.
Furthermore, as will be appreciated, various portions of the disclosed systems and methods can include or consist of artificial intelligence, machine learning, or knowledge or rule based components, sub-components, processes, means, methodologies, or mechanisms (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, classifiers . . . ). Such components, inter alia, can automate certain mechanisms or processes performed thereby to make portions of the systems and methods more adaptive as well as efficient and intelligent. By way of example and not limitation, the assistance component 410 can utilize such mechanisms to infer aid such as suggestions of automatic statement completion as a function of current rule specification, the regular expression grammar, and/or a target language grammar.
In view of the exemplary systems described sura, methodologies that may be implemented in accordance with the disclosed subject matter will be better appreciated with reference to the flow charts of
The same mechanism can also be utilized for different reasons with a different set of rules. For example, color coding and intelligent assistance can be specified in this manner. In this case, the (e.g., each type of word in a script can be extracted to enable rendering a proper color. Below is a sample set of grammar rules that can be specified to facilitate generation of an appropriate regular expression to effect such functionality with respect to the previously described exemplary language.
Turning to
Method 800 can be employed in a plurality of situations. For example, the method can be utilized with respect to compiler features and/or functionality including identification of tokens and translation thereof at compile time and provisioning of design time assistance such as color-coding, formatting, automatic code completion and/or error detection. Further, yet the method 800 can be employed with respect to other conventional and/or unconventional regular expression uses including data flow technologies, among other things.
As used herein, the terms “component,” “system,” “engine,” and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an instance, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
The word “exemplary” or various forms thereof are used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Furthermore, examples are provided solely for purposes of clarity and understanding and are not meant to limit or restrict the claimed subject matter or relevant portions of this disclosure in any manner. It is to be appreciated that a myriad of additional or alternate examples of varying scope could have been presented, but have been omitted for purposes of brevity.
As used herein, the term “inference” or “infer” refers generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic—that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources. Various classification schemes and/or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines . . . ) can be employed in connection with performing automatic and/or inferred action in connection with the subject innovation.
Furthermore, all or portions of the subject innovation may be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed innovation. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device or media. For example, computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
In order to provide a context for the various aspects of the disclosed subject matter,
With reference to
The system memory 916 includes volatile and nonvolatile memory. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within the computer 912, such as during start-up, is stored in nonvolatile memory. By way of illustration, and not limitation, nonvolatile memory can include read only memory (ROM). Volatile memory includes random access memory (RAM), which can act as external cache memory to facilitate processing.
Computer 912 also includes removable/non-removable, volatile/non-volatile computer storage media.
The computer 912 also includes one or more interface components 926 that are communicatively coupled to the bus 918 and facilitate interaction with the computer 912. By way of example, the interface component 926 can be a port (e.g., serial, parallel, PCMCIA, USB, FireWire . . . ) or an interface card (e.g., sound, video, network . . . ) or the like. The interface component 926 can receive input and provide output (wired or wirelessly). For instance, input can be received from devices including but not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, camera, other computer and the like. Output can also be supplied by the computer 912 to output device(s) via interface component 926. Output devices can include displays (e.g., CRT, LCD, plasma . . . ), speakers, printers and other computers, among other things.
The system 1000 includes a communication framework 1050 that can be employed to facilitate communications between the client(s) 1010 and the server(s) 1030. The client(s) 1010 are operatively connected to one or more client data store(s) 1060 that can be employed to store information local to the client(s) 1010. Similarly, the server(s) 1030 are operatively connected to one or more server data store(s) 1040 that can be employed to store information local to the servers 1030.
By way of example and not limitation, the functionality of the rule compilation component 120 and/or assistance component 410 can be provided as a web service supplied by one or more servers 1030 to one or more requesting clients 1010 over the communication framework 1050. Thus, programmers could utilize the service to generate rules or simply provide rules to the service and receive a regular expression in return. Additionally or alternatively, such components can be downloaded from server(s) 1030 to client(s) 1010 utilizing communication framework 1050 to facilitate local storage and/or execution.
What has been described above includes examples of aspects of the claimed subject matter. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the claimed subject matter, but one of ordinary skill in the art may recognize that many further combinations and permutations of the disclosed subject matter are possible. Accordingly, the disclosed subject matter is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the terms “includes,” “contains,” “has,” “having” or variations in form thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5526477 | McConnell | Jun 1996 | A |
| 6618697 | Kantrowitz | Sep 2003 | B1 |
| 6684381 | Bening | Jan 2004 | B1 |
| 6714905 | Chang et al. | Mar 2004 | B1 |
| 7093231 | Alderson | Sep 2007 | B2 |
| 20020129005 | Ojanen | Sep 2002 | A1 |
| 20040002850 | Shaefer | Jan 2004 | A1 |
| 20040111400 | Chevalier | Jun 2004 | A1 |
| 20050097514 | Nuss | May 2005 | A1 |
| 20060167873 | Degenaro | Jul 2006 | A1 |
| 20060179054 | Levi | Aug 2006 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20090083265 A1 | Mar 2009 | US |
