Patent Grant
10896163
As provided for under 35 U.S.C. § 120, this patent claims benefit of the filing date of the following U.S. patent application, herein incorporated by reference in its entirety:
“Methods and Apparatuses For Clustered Storage of Information and Query Formulation,” filed 2011 Oct. 24 (y/m/d), having inventors Mark Edward Bowles, Jens Erik Tellefsen, and Ranjeet Singh Bhatia, and App. No. 13280294.
This application is related to the following U.S. patent application(s), which are herein incorporated by reference in their entirety:
“Method and Apparatus For Frame-Based Search,” filed 2008 Jul. 21 (y/m/d), having inventors Wei Li, Michael Jacob Osofsky and Lokesh Pooranmal Bajaj and App. No. 12177122 (“the '122 Application”);
“Method and Apparatus For Frame-Based Analysis of Search Results,” filed 2008 Jul. 21 (y/m/d), having inventors Wei Li, Michael Jacob Osofsky and Lokesh Pooranmal Bajaj and App. No. 12177127 (“the '127 Application”);
“Method and Apparatus For Determining Search Result Demographics,” filed 2010 Apr. 22 (y/m/d), having inventors Michael Jacob Osofsky, Jens Erik Tellefsen and Wei Li and App. No. 12765848 (“the '848 Application”);
“Method and Apparatus For HealthCare Search,” filed 2010 May 30 (y/m/d), having inventors Jens Erik Tellefsen, Michael Jacob Osofsky, and Wei Li and App. No. 12790837 (“the '837 Application”); and
“Method and Apparatus For Automated Generation of Entity Profiles Using Frames,” filed 2010 Jul. 20 (y/m/d), having inventors Wei Li, Michael Jacob Osofsky and Lokesh Pooranmal Bajaj and App. No. 12839819 (“the '819 Application”).
Collectively, the above-listed related applications can be referred to herein as “the Related Applications.”
The present invention relates generally to the clustered storage of information, and more particularly to efficiently representing hierarchical information within a framework of records searchable by an inverted index. The present invention also relates generally to the formulation of queries and more particularly to the selection of exclusion terms.
Inverted Index Databases (or IIDBs) are well known. An example IIDB is the well-known Open Source software “Lucene,” that uses an inverted index to perform rapid searches of a collection of records (Lucene is provided by “The Apache Software Foundation,” a not-for-profit Delaware corporation, with a registered office in Wilmington, Del., U.S.A.). IIDBs like Lucene are sufficiently efficient and scalable such that they can be used for searching a large-scale corpus, a function provided by web-accessed search engines.
A limitation of IIDBs like Lucene is that the only inherent structural relationship supported, between records, is the single-level linear collection. It should be noted that the basic item of indexed data, supported by generic Lucene (i.e., Lucene that lacks the present invention), is called a “document.” However, herein, for purposes of generality, we shall refer to the basic item of indexed data as a “record.” Each record of an IIDB is identified by a unique ID number (where Lucene currently has capability to store up to 231 records, since the unique ID for each record is a 32 bit signed integer).
It would therefore be desirable to augment IIDBs to permit efficient representation of structural relationships, between records of an IIDB, that are more complex than just a single-level linear collection.
An important use of IIDBs is the searching of a “Corpus of Interest” (or C_of_I) for mentions of an “Object of Interest” (or O_of_I). A particular type of O_of_I is a brand of consumer products (also referred to herein as a “Consumer Brand” or “C_Brand”). C_Brands can be the subject of large-scale database searches, particularly of Internet content, by Brand Managers (persons responsible for the continued success of a C_Brand). In particular, a Brand Manager is often interested, for example, in the sentiment of consumers toward his or her C_Brand.
The names of many C_Brands, however, can be ambiguous.
Ambiguity, in a lexical unit, means that the same lexical unit can have two or more distinctly different meanings. Some example C_Brands, with ambiguous names, include the following:
It would therefore be highly desirable to provide techniques for the formulation of queries that are more precise at the identification of an O_of_I (such as a C_Brand), while still achieving a high level of recall.
The accompanying drawings, that are incorporated in and constitute a part of this specification, illustrate several embodiments of the invention and, together with the description, serve to explain the principles of the invention:
Reference will now be made in detail to various embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Please Refer to the Section 4 (“Related Applications”) and Section 5 (“Glossary of Selected Terms”), Included as the Last Two Sections of the Detailed Description, for the Definition of Selected Terms Used Below.
1 Clustered Storage
1.1 First Embodiment
1.2 Pseudo-code
2.1 Consumer Sentiment Search
2.2 Exclude Term Assistant
A first embodiment of the present invention permits efficient representation of structural relationships, between records of an IIDB, that are more complex than just a single-level linear collection. In particular, the present invention is directed to data storage situations where, at a local level, it is useful to view records as being organized into clusters. A “cluster” can encompass any kind of data organization that relies on relatively localized connections between its records. The structural organization for clusters focused upon herein is the hierarchical (or “tree”) structure, but it should be understood that the present invention can be applied to clusters with other organizational arrangements.
The two basic types of operations, provided by an IIDB that lacks the present invention, are as follows. For simplicity of explanation, the operations are described as producing, or operating upon, ordered lists of record ID numbers. However, any suitable data structure, that provides functionally similar results, can be used.
To the two above-listed basic operations, the present invention adds an intra-cluster level-conversion capability, referred to herein as a “To” operator. Before explaining the “To” operator, it is useful to explain how localized clusters of data can be encoded within a single-level linear collection of records.
Records 0-14 have been divided into the following two clusters of records:
While any collection of symbols can be used to denote the levels of a hierarchy, it is often useful to assign a unique integer to each level. For purposes of discussion herein, we shall follow the convention of assigning a zero to the highest level and increment, by one, the value representing each successive level. Therefore, the four levels discussed just-above can be represented as follows:
As can be seen from the assignment of record ID numbers, in
Even without the “To” operator, there are still a variety of operations that can be performed upon an IIDB of serialized clusters. For example, consider the four-level hierarchical clusters discussed just-above. Each level of the hierarchy can be given its own “class” of record, and each such class can be accessible by its own selection of indexed fields. An example class structure follows:
Using the above-defined class structure, following are four example IIDB search queries (where each query is followed by explanatory commentary):
Further, Boolean operators can be used to combine search queries. For example, the following query finds all focus sentences that have the word “Tide”:
To the above, the “To” operator adds the ability for a query result, at one level of a cluster hierarchy, to be translated into its equivalent form at another level the cluster hierarchy.
The “To” operator receives (either explicitly or implicitly) at least two parameters:
For each record “r” of the first parameter, the “To” operator does the following:
If the clusters are hierarchical, there are two main possibilities:
To continue with the same example of ID numbers (4, 11), discussed just-above, an example target level, for these instances, could be the “role” level. Conversion of the ID numbers (4, 11), from the instance to role level, produces the following list of ID numbers: (5, 6, 12, 13). In the form of an IIDB query expression, the example can be expressed as follows:
The above example “To” expression can be related, as follows, to the above example of going from (4, 11) “To” (5, 6, 12, 13). We will assume that Record 4 (or instance 130), of
Conversion of ID numbers, from their starting level in a hierarchy to the equivalent ID numbers at a target level, requires that at least the following basic structural information is preserved in the serialization:
The two above-listed types of information, along with knowing the visitation procedure (e.g., depth-first) by which a serialization sequence was produced, is sufficient to accomplish the “To” operation. To make the “To” operation as fast as possible, so that it can be applied (for example) to search results produced from the querying of very large databases, both types of basic structural information can be preserved with bit maps.
If the clusters are hierarchical,
A discussion, of each of
Example pseudo-code, for interpreting bit maps of similar structure to that shown in
Line 1 of
The “To” procedure returns (see line 18 of
For either case, the “To” procedure calls the “Single_To” procedure (either at line 10 or 14 of
The “Single_To” procedure returns (see line 23 of
For purposes of example, let us first assume that Single_To has been invoked as follows on the IIDB of
Single_To starts by initializing the start and target levels (
Next, Single_To prepares for its reconstruction, of at least part of a cluster's serialized records, by setting the current record to the start record (
If the target level is less than the start level, cluster reconstruction is performed by the “while” loop of lines 7-21. For the example, since 0<3, this “while” loop is executed.
The “while” loop will perform a first execution, of the loop body, if its test (line 7) is satisfied. In our example, since 3≠0, a first iteration is begun.
For purposes of the pseudo-code, serialization is assumed to be done in a depth-first “left to right” order. Thus, reconstructing a cluster's hierarchy in an upwards fashion, when starting from the start record, involves a step-by-step accessing of records in a “leftwards” direction. In the example, this ordering can be seen when viewing
Now that the current record has been processed, the left record is made the new “current record” (line 20) and the second iteration of the “while” is started. A diagrammatic representation, of the reconstruction produced by the first “while” iteration, when processing the example, is shown in
Successive iterations of the “while” loop (of
Briefly, each of the following iterations can be described as follows:
When attempting to start iteration 7, since the current record is Record 0, the condition of the “while” is not satisfied (because level (Record 0)=target_level). Therefore, the “while” loop ends with the reconstruction being shown in
Thus, “Single_To” has mapped Record 6 to Record 0 and Record 5 is available as an additional result. This additional result could be useful, for example, if the “To” procedure was called with an input list (or “recs_2b_mapped”) that included both Records 5 and 6. As can be seen from
In order to explain the portion of the Single_To procedure, illustrated by
As was the case with the previous example of mapping up the hierarchy, Single_To starts by initializing the start and target levels (
Next, the Single_To procedure prepares for reconstruction, of at least part of a cluster's serialized records, by setting the current record to the start record (
If the target level is greater than the start level (see “if” of
For the example, the “if” of
In terms of returning a value for “Single_To,” line 19 returns those records (Records 5 and 6) of the reconstructed tree that are at the target level (of level 3).
Distinguishing a usage of a lexical unit that is intended to refer to an “Object of Interest” (O_of_I), from a usage of a lexical unit that is intended to refer to something other than the O_of_I, can be greatly assisted by the inclusion of “Exclude Terms” in a search query.
In general, an Exclude Term can be defined as follows. It is a term that can be included as part of a query where, if the term is found in a record of an IIDB, that record is excluded from inclusion in the search result.
The present invention, for the formulation of Exclude Terms for inclusion in a database query, can be applied to any “Corpus of Interest” (C_of_I) for which mentions, of an O_of_I, are to be identified. A particular type of search, to provide an example where the present invention can be utilized, is presented in this Section 2.
The particular type of O_of_I is a brand of consumer products (also referred to herein as a “Consumer Brand” or “C_Brand”). C_Brands can be the subject of large-scale database searches, particularly of Internet content, by Brand Managers (persons responsible for the continued success of a C_Brand). In particular, a Brand Manager is often interested, for example, in the sentiment of consumers toward his or her C_Brand.
A C_of_I can be collected and searched for mentions of the O_of_I. In the case of a C_Brand, an example C_of_I can be a database that represents the collection, in a large scale and comprehensive way, of postings (such as “tweets” on Twitter) to Social Media (SM) web sites or services. We can refer to such Social Media database as “SM_db.”
As has been described in the above-referenced Related Applications (please see
Cross Reference to Related Applications), a frame-based search tool can be provided, by which instances of an O_of_I can be sought, in a C_of_I, in connection with a particular type of concept or concepts. More particularly, a Brand Manager can be provided with a frame-based search tool by which instances of a C_Brand can be sought, in a SM_db, in connection with a particular type of concept. For purposes of example herein, the “concept” presented is that of a consumer expressing the fact that he or she “likes” a C_Brand.
An example set of roles, for a “Like” frame, are as follows (each role name is in capitals, with a brief explanation following):
The above “Like” frame is typically applied to the analysis of an individual sentence (referred to herein as the “focus sentence”). Following is an example focus sentence, to which appropriate Natural Language Processing (NLP) can be applied to produce an instance of the “Like” frame. The following example sentence discusses a fictitious brand of soda called “Barnstorm”:
Given a suitable NLP analysis, by application of suitable frame extraction rules, the following instance of the “Like” frame can be produced:
In addition to the focus sentence, each post to Social Media can be summarized as a three sentence “snippet,” with the focus sentence forming the middle sentence. A single type of record, let us call it “SentenceObj,” can include both the focus sentence and the snippet as fields. These fields can be called, respectively, “FocusSentence” and “Snippet,” with each field being indexed and therefore available for queries. Thus, when searching for all SentenceObjs, that satisfy a particular query, there are at least two indexes that can be used. As an example, if the FocusSentence index is to be searched for all occurrences of the word “Tide” and the Snippet index is to be searched for all occurrences of the word “government,” then an IIDB syntax, for expressing these queries, can be (respectively) as follows:
In a similar manner to that discussed above (Section 1, “Clustered Storage”), each SentenceObj record, of the SM_db, can part of a separate “cluster” of an IIDB. Including to the SentenceObj, the cluster can be hierarchically organized to contain the following three record types:
As has been discussed in the Related Applications, a large-scale database (such as the SM_db), and its indexes, is typically created before a user query is input. The IIDB of the present application, however, can differ from those discussed in the Related Applications because of the cluster storage invention of above Section 1 (“Clustered Storage”).
A user can formulate a query by identifying lexical unit or units representative of the O_of_I (e.g., a C_Brand). For purposes of example, it is assumed that the O_of_I is identified by only one lexical unit. Further, for purposes of example, we will address the C_Brand called “Tide” (a brand of laundry detergent) and assume it is to be identified, by a Brand Manager, by the single lexical unit “Tide.”
All focus sentences, of the SM_db, can be searched for usage of the lexical unit “Tide.” Based upon the CSH, the following query can return all resulting focus sentences:
If a user has already identified an Exclude Term, to be used in conjunction with the search query, it can also be utilized. All focus sentences, where its snippet contains at least one Exclude Term, can be excluded from the search results. For example, based upon the CSH, the following query can return the list of all focus sentences where its snippet contains the Exclude Term “Government”:
Using the “NOT” operator, according to the following expression, the above ordered list (of focus sentences where its snippet contains the Exclude Term) can be converted to the list of focus sentences where its snippet does not contain the Exclude Term:
Finally, the Exclude Term can be applied to the above-listed search for “Tide” with the AND operator:
An example user interface, for entering this type of search is shown in
For each focus sentence found, the “To” operator (discussed above in Section 1, “Database Storage”) can be applied twice:
For purposes of example, we will assume that the role of interest, for the “Tide” brand, is “Emotion.” It is assumed that the Brand Manager wishes to know all the positive emotions consumer associate with “Tide.”
Going from the set of focus sentences “To” the set of all instances within such focus sentences can be accomplished with the following expression:
The role values, of the “Like” frames, can be found by use (as described above) of a second “To” in the following expression:
For clarity of explanation, the above expression can be represented, symbolically, by the identifier “ROLE_VALUES_SEARCH_RESULT.”
The different role values found can be subjected to grouping analysis. In grouping analysis, similar roles values can be put into a single group and the group given a generic name (or “g_name”). For the particular example, the following are several role values that can be placed in a common group:
The common lexical unit, among a group of role values, can be identified. For the example above, the common lexical unit is “love.” Thus, all three role values can be presented to the user (e.g., the Brand Manager) as a single emotion of interest “love.”
Grouping can continue recursively, with subgroups being identified within a group. For the example above, it can be seen that “really love” can be identified as a subgroup of “love.”
In addition to placing role values into groups, and presenting generic role value names (or g_name's) to the user, the order of presentation of such g_name's can be determined by the frequency with which each such g_name appears in the search result. Thus, for example, if the g_name “like” represents 953 focus sentences (where each of the 953 focus sentences contains at least one occurrence of the word “like”), and the g_name “love” represents only 262 focus sentences, then the g_name “like” is presented before the g_name “love.”
An example user interface, for presenting such g_names is shown in
When a user selects a g_name, it can be useful to see its usage in context. In other words, in can be useful to see at least a sampling of the focus sentences in which the g_name appears. For
When displaying such focus sentences to the user, it can be useful to highlight (or otherwise emphasize), within each such focus sentence, the occurrence of the g_name that caused the sentence to be displayed.
While the above-described search process of Section 2.1 can often be very useful, it can have certain limitations. An example is the search of a SM_db for mentions of a C_Brand where the lexical unit or units, that represent the C_Brand, are ambiguous.
For the Section 2.1 example, of a Brand Manager of the C_Brand “Tide” searching a SM_db, an example query with an exclude term was already discussed. It is the following expression that excludes the word “government”:
The present invention provides techniques for greatly improving the process by which Exclude Terms are identified. A step-by-step presentation of a process, that illustrates these techniques, follows.
The Exclude Term identification process begins with a search for the lexical unit or units, that can refer to the O_of_I in the C_of_I. For clarity of explanation, we shall refer to one lexical unit, as potentially referring to the O_of_I. We can refer to that one lexical as the lexical unit of interest (or LU_of_I). (It is clear to anyone of ordinary skill in the art, that the following procedure can be expanded to accommodate more than one LU_of_I.)
The LU_of_I is assumed to be ambiguous, and therefore have at least two meanings:
We will continue with the above-described example of a Brand Manager, seeking to research a C_Brand in a SM_db. In particular, we will use the example of the LU_of_I being “Tide,” and the single Exclude Term of “government” having been identified, as is shown in
By selecting the Exclude Term Assistant (ETA) button 1004, of
Any Exclude Terms, already identified, can be stored in stored in a list referred to herein as the “Exclude Term List” (or “ET_list”). When performing the search of the snippets, even if a snippet has the LU_of_I, if the snippet also includes a member of the ET_list, then the snippet is not included in the search result. For the case of “n” Exclude Terms, and the LU_of_I being “Tide,” the following expression can be used to produce a list of snippets that is reduced by each member of the ET_list:
For clarity of explanation, the list of focus sentences resulting from this step can be referred to as “ETA_FOCUS_SENTENCES.”
For each focus sentence, from the set of focus sentences retrieved (i.e., ETA_FOCUS_SENTENCES), the “To” operator can be used twice (in a manner similar to that discussed above in Section 2.1, “Consumer Sentiment Search”):
However, unlike Section 2.1, the two just-above listed uses of the “To” operator can differ, respectively, as follows:
Given the size of the IIDB's that can be processed, there may be too many resulting role values from the previous step (i.e., the role values indicated by ETA_ROLES), for processing by the next step of frequency and cluster analysis. For example, the step of Section 2.2.2 can produce 20-30 million role values.
A sampling can be performed, of only a portion of the result of ETA_ROLES, to produce a computationally tractable number of role values. Any of the known statistical techniques, for approximating the range of values of a larger population from a smaller sample of that population, can be used. For example, assume that ETA_ROLES represents an ordered list of 107 role values and that only 105 values can be processed, by the next step (Section 2.2.4), in a sufficiently small time period. This means that for each 102 role values, of ETA_ROLES, only 1 is included in the set of role values passed-along by this step for further processing.
Regardless of whether down-sampling is preformed, for purposes of the next step (Section 2.2.4), it is assumed that ETA_ROLES indicates the ordered list of roles for processing.
Given the ordered list of role values produced by either of Section 2.2.2 or Section 2.2.3 (either of which is indicated by ETA_ROLES), the following four main steps can be performed. Collectively, the following four steps can be referred to herein as “Basic Frequency and Cluster Analysis” or “Basic FCA”:
The above-described Basic FCA, along with the Example FCA, can be related, as follows, to
Column 1025 has certain similarities to the Example FCA. As is shown for the Example FCA, for the fourth step of Basic FCA, column 1025 also shows “tide” as a group name, with “crimson tide” and “high tide” being sub-members of that group. Also like the Example FCA, column 1025 shows “crimson tide” listed before “high tide” because of the greater frequency of “crimson tide.” Specifically, frequency column 1026 shows “crimson tide” and “high tide” as having, respectively, relative frequency indicators of 8% and 6%.
Although not specifically shown in column 1025, as illustrated in
The term “high” as a group name, as found in the Example FCA at the fourth step of Basic FCA, is not illustrated in
Frequency and cluster analysis has been described as being performed on the values of the roles, that correspond to the focus sentences found by search of Section 2.2.1. However, frequency and cluster analysis could be performed directly upon the focus sentences found by search of Section 2.2.1. In this case, candidate Exclude Terms can be found by determining the various n-grams, to a suitable level of “n,” on the focus sentences.
As has already been introduced as a topic above, in Section 2.2.4, the user can select one or more items, from the list of candidate Exclude Terms. Such selection can be based upon any combination of the following factors (including other factors not specified herein)
With specific reference to
Precision information, regarding the nature of the focus sentences associated with a candidate Exclude Term, can be available in screen 1020 of
The user can loop back, to the beginning of the Exclude Term selection process, by returning to the search of Section 2.2.1. Upon such loop-back, the search will differ by inclusion the Exclude Terms selected. In
Alternatively, if the user is satisfied with the quality of the query, he or she can escape the ETA by “closing” screen 1020 (user-interface for closing not shown) and returning to a screen such as screen 1000 of
Cloud 930 represents data, such as online opinion data, available via the Internet. Computer 910 can execute a web crawling program, such as Heritrix, that finds appropriate web pages and collects them in an input database 900. An alternative, or additional, route for collecting input database 900 is to use user-supplied data 931. For example, such user-supplied data 931 can include the following: any non-volatile media (e.g., a hard drive, CD-ROM or DVD), record-oriented databases (relational or otherwise), an Intranet or a document repository. A computer 911 can be used to process (e.g., reformat) such user-supplied data 931 for input database 900.
Computer 912 can perform the indexing needed for formation of an appropriate frame-based database (FBDB). FBDB's are discussed in the Related Applications. The indexing phase scans the input database for sentences that refer to an organizing frame (such as the “Like” frame), produces a snippet around each such sentence and adds the snippet to the appropriate frame-based database.
Databases 920 and 921 represent, respectively, stable “snapshots” of databases 900 and 901. Databases 920 and 921 can provide stable databases that are available for searching, about an O_of_I in a C_of_I, in response to queries entered by a user at computer 933. Such user queries can travel over the Internet (indicated by cloud 932) to a web interfacing computer 914 that can also run a firewall program. Computer 913 can receive the user query, collect snippet and frame instance data from the contents of the appropriate FBDB (e.g., FBDB 921), and transmit the results back to computer 933 for display to the user. The results from computer 913 can also be stored in a database 902 that is private to the individual user. When it is desired to see the snippets, on which a graphical representation is based, FBDB 921 is available. If it is further desired to see the full documents, on which snippets are based, input database 920 is also available to the user.
In accordance with what is ordinarily known by those in the art, computers 910, 911, 912, 913, 914 and 933 contain computing hardware, and programmable memories, of various types.
The information (such as data and/or instructions) stored on computer-readable media or programmable memories can be accessed through the use of computer-readable code devices embodied therein. A computer-readable code device can represent that portion of a device wherein a defined unit of information (such as a bit) is stored and/or read.
The description presented herein relies on many parts of the Related Applications. This section makes reference to particular portions of the '837 Application, which is a member of the group of the Related Applications.
In general, sections of the '837 Application can be referred to herein by the following convention. Where “X” is a section number, the section can be referred to as: Section X, '837. If the title of the section is to be included, where the title is “Title,” it can be referred to as: Section X, '837 (“Title”) or Section X, '837, “Title.”
Section 4, '837 (“FBSE”) describes a Frame-Based Search Engine (or FBSE). This FBSE is a more generic form of the kind of search described herein in Section 2.1 (“Consumer Sentiment Search”).
Section 4.2, '837 discusses frames as a form of concept representation (Section 4.2.1) and the use of frame extraction rules to produce instances of frames (Section 4.2.2). A pseudo-code format for frame extraction rules is presented in Section 6.2, '837 (“Frame Extraction Rules”).
Snippets are discussed in Section 6.4, '837.
The “Frame-Based Database” (FBDB), discussed herein in Section 3 (“Computing Environment”), is described in Section 4.3.2 (“Pre-Query Processing”), '837.
While the invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications and variations will be apparent in light of the foregoing description. Accordingly, the invention is intended to embrace all such alternatives, modifications and variations as fall within the spirit and scope of the appended claims and equivalents.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5694523 | Wical | Dec 1997 | A |
| 5850561 | Church | Dec 1998 | A |
| 5940821 | Wical | Aug 1999 | A |
| 5963940 | Liddy et al. | Oct 1999 | A |
| 5995922 | Penteroudakis et al. | Nov 1999 | A |
| 6202064 | Julliard | Mar 2001 | B1 |
| 6269356 | Hatton | Jul 2001 | B1 |
| 6278967 | Akers et al. | Aug 2001 | B1 |
| 6453312 | Goiffon et al. | Sep 2002 | B1 |
| 6560590 | Shwe et al. | May 2003 | B1 |
| 6571240 | Ho | May 2003 | B1 |
| 6578022 | Foulger et al. | Jun 2003 | B1 |
| 6584464 | Warthen | Jun 2003 | B1 |
| 6671723 | Nguyen et al. | Dec 2003 | B2 |
| 6675159 | Lin et al. | Jan 2004 | B1 |
| 6738765 | Wakefield et al. | May 2004 | B1 |
| 7302383 | Valles | Nov 2007 | B2 |
| 7356540 | Smith | Apr 2008 | B2 |
| 7496593 | Gardner et al. | Feb 2009 | B2 |
| 7779007 | West et al. | Aug 2010 | B2 |
| 7805302 | Chelba et al. | Sep 2010 | B2 |
| 8046348 | Rehling et al. | Oct 2011 | B1 |
| 8055608 | Rehling et al. | Nov 2011 | B1 |
| 8131540 | Marchisio et al. | Mar 2012 | B2 |
| 8935152 | Li et al. | Jan 2015 | B1 |
| 9047285 | Li et al. | Jun 2015 | B1 |
| 20020091671 | Prokoph | Jul 2002 | A1 |
| 20030172061 | Krupin et al. | Sep 2003 | A1 |
| 20030216905 | Chelba et al. | Nov 2003 | A1 |
| 20040044952 | Jiang et al. | Mar 2004 | A1 |
| 20040078190 | Fass et al. | Apr 2004 | A1 |
| 20040186827 | Anick et al. | Sep 2004 | A1 |
| 20050149494 | Lindh et al. | Jul 2005 | A1 |
| 20050165600 | Kasravi et al. | Jul 2005 | A1 |
| 20070156677 | Szabo | Jul 2007 | A1 |
| 20090112892 | Cardie | Apr 2009 | A1 |
| 20090319517 | Guha | Dec 2009 | A1 |
| 20150293997 | Smith et al. | Oct 2015 | A1 |
| Entry |
|---|
| Gautam et al., published Feb. 17, 2008 (y/m/d), pp. 2040-2042. “Document Retrieval Based on Key Information of Sentence,” IEEE ICACT. |
| Ku et al., published Mar. 27, 2006 (y-m-d), 8 pgs. “Opinion Extraction, Summarization and Tracking in News and Blog Corpora,” AAAI Spring Symposium Series 2006. |
| Manning et al., published 1999. “Foundations of Statistical Natural Language Processing,” The MIT Press, Table of Contents (12 pgs), sec. 8.5.1 (pp. 296-303) & sec. 15.2 (pp. 539-544). ISBN 0-262-13360-1. |
| Ruppenhofer et al., published Aug. 25, 2006 (y/m/d), 166 pages. “FrameNet II: Extended Theory and Practice,” International Computer Science Institute, University of California at Berkeley, USA. |
| Wu, Tianhaow et al., published May, 3, 2003 (y/m/d), 12 pgs. “A Supervised Learning Algorithm for Information Extraction From Textual Data,” Proceedings of the Workshop on Text Mining, Third Siam International Conference on Data Mining. |
| Zadrozny, Slawomir et al., published 2003, 5 pgs. “Linguistically quantified thresholding strategies for text categorization,” Systems Research Institute, Polish Academy of Sciences, Warszawa, Poland. |
| Zhang et al., published Jun. 22, 2010 (y/m/d), 10 pgs. “Voice of the Customers: Mining Online Customer Reviews for Product Feature-based Ranking,” Proceedings of the 3rd Wonference on Online social networks (WOSN '10). USENIX Association, Berkeley, CA, USA. |
| Lucene Support p. 2454, with comments dated May 10, 2010-Jul. 16, 2010; https://issues.apache.org/jira/browse/LUCENE-2454; retrieved Jul. 24, 2019 (y/m/d); 9 pages. |
| Lucene Slide Share Presentation, dated May 7, 2010; https://www.slideshare.net/MarkHarwood/proposal-for-nested-document-support-in-lucene; retrieved Jul. 24, 2019 (y/m/d); 15 pages. |
| readme.txt in LuceneNestedDocumentSupport.zip, creation date May 10, 2010; retrieved Jul. 25, 2019 (y/m/d); 2 pages. |
| NestedDocumentQuery.java in LuceneNestedDocumentSupport.zip, creation date Aug. 25, 2010; retrieved Jul. 25, 2019 (y/m/d); 8 pages. |
| PerParentLimitedQuery.java in LuceneNestedDocumentSupport.zip, creation date Sep. 8, 2010; retrieved Jul. 25, 2019 (y/m/d); 10 pages. |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13280294 | Oct 2011 | US |
| Child | 14793644 | US |
