This invention relates to digital rights display and methods and apparatus for determining reuse rights for content. Works, or “content”, created by an author is generally subject to legal restrictions on reuse. For example, most content is protected by copyright. In order to conform to copyright law, content users often obtain content reuse licenses. A content reuse license is actually a “bundle” of rights, including rights to present the content in different formats, rights to reproduce the content in different formats, rights to produce derivative works, etc. Thus, depending on a particular reuse, a specific license to that reuse may have to be obtained.
Many knowledge workers attempt to determine which rights are available for particular content before using that content in order to avoid infringing legitimate rights of rightsholders. If rights are sought for a particular publication, several alternatives are available. For example, the worker can often determine the publisher of the publication from a standard publication number, such as an ISBN, from the author or from the content itself. The worker can then visit the publisher's website to determine what rights are available. Alternatively, the worker can visit the website of a rights clearing house, such as the Copyright Clearance Center, located in Danvers, Mass. This organization partners with many publishers to offer licensed rights from each publisher so that the worker can search for publications using information, such as an ISBN, an author's name or words in the publication title. Once the publication has been located, a variety of reuse rights are displayed from various sources. The worker can then select the most appropriate right at an appropriate price. For example, the worker may belong to an organization that has pre-purchased licenses from certain publishers, but not others, in which case the worker will select a publication that is available from a source which is already licensed.
However, if rights are sought only for a particular article, identifying an appropriate source is more difficult. More specifically, authors frequently submit the same article to a variety of publications, so that the article appears in several publications over a period of time. In addition, some publications reprint articles that originally appeared in other publications, these reprinted articles may appear singly or in collections. The identification is further complicated because no single source offers a comprehensive database of all articles and where they have been published. Some publishers expose a search service offering the ability to search their content, but such searches must be conducted publisher by publisher. These searches are inconvenient because each publisher has a specific format in which queries must be submitted and a specific format in which results are returned so that a comprehensive search requires knowledge of each publisher and a consolidation of the search results.
In accordance with the principles of the invention, a federated search program receives a generic query from a client associated with a user and generates a plurality of sub-queries from the generic query. Each sub-query is generated by a connector object that is associated with a particular content source and the generic query is dispatched simultaneously to all connector objects. Each connector object contains source specific code that reformats the generic query into a proprietary format required for the associated content source. The proprietary query is then dispatched to the content source. When the results at the content source are ready, the result set is fetched by the connector. The fetched results are then mapped into a standard format. The standard result sets from the different content sources are then merged into a single consolidated result set. Duplicate documents are removed from the consolidated result set and the final results are sorted in accordance with criteria specified by the user and presented to the user.
Client 102 could be any application that generates an article level search. For example, one such application is a web application that is published with the URL www.copyright.com by Copyright Clearance Center, Inc. (CCC). This web application generates several search displays of which screen shots are shown in
Both, the basic search initiated from the display shown in
This search is initiated when the client 102 provides a generic query to the search service 106, and specifically to the dispatcher 108 as indicated by arrow 104 and as set forth in step 204. As an example, this query might look like:
Title: Geophysics
Author: Akerberg
As previously mentioned, the search is conducted simultaneously over a plurality of content sources. One embodiment uses four content sources or search “targets”: an internal CCC database, a Nature database, a PubGet database and a New York Times (NYT) database. Each search target has its own specific query language in which it expects queries to be expressed. For example the CCC internal database uses SoIr technology which uses internally the Lucene engine language. Details of this language can be found at: lucene.apache.org/java/2—3—2/queryparsersyntax.html. Similarly, details of the Nature query language can be found at: nature.com/opensearch/. The Pubget and NYT query language details can be found at corporate.pubget.com/services/premium and developer.nytimes.com/, respectively.
Therefore, the generic search must be converted into the local query language for each content source. Accordingly, next, in step 206, the dispatcher 108 simultaneously dispatches the generic query to a plurality of connector objects, of which three 112, 114 and 116, are shown in
The details of a connector object are shown in
This query includes parts that are created to shape a relevancy ranking calculation.
The same query would look like:
in the query language used to access the Nature database.
The corresponding queries in the PubGet and NYT site specific languages are:
where the “key” clause is a special key that allows access to NYT repository of articles.
In addition, an ISSN or ISBN number for the publication or book (obtained from user input in the basic or advanced search displays shown in
After, the generic query has been reformatted into query format for a particular content provider, the reformatted query is provided as indicated schematically by arrow 606 to a database interface 608 which logs onto the database (if necessary) and, in step 210, transmits the reformatted query to the content provider as schematically illustrated by arrow 610 in
The connector objects 112, 114 and 116 then wait for search results to become available at the content providers sites, and when available as indicated by step 212, a data fetcher 612 fetches the results as indicated schematically by arrow 614 and provides the results to a format mapper 618. Format mapping is necessary because, as with the query language, the results are generally in a format that is specific to each content provider, such as XML or JSON.
The process then proceeds, via off-page connectors 214 and 216, to step 218 where the format mapper 618 in the connector object 600 maps the query result metadata from each content provider into a common format. The results of step 218 produce a result list from each search connector and generate a “list of lists” with search results—each search target produced its own selection (list) of records. Next, in step 220, the results from each connector object, for example, connector objects 112, 114 and 116, are provided to a merge module 144 as schematically indicated by arrows 138, 140 and 142 where the results are merged by indentifying duplicates between search targets.
The merging process involves comparing the metadata of pairs of documents with each document of the pair being taken from a different target to create a consolidated list. Documents in the consolidated list are then compared to documents of a target other then the two targets used to compose the consolidated list. This process is repeated until all documents in the consolidated list have been compared to all documents in the different target lists. The merging process for a pair of documents in shown in more detail in
Alternatively, if the DOIs of the two documents do not match as determined in step 704, the documents are considered different and the process proceeds to step 710 where both documents are retained. The process then finishes in step 712.
Alternatively, if in step 702 it is determined that at least one of the two documents being compared does not have a DOI, then the process proceeds to step 706 where a “title group” match is performed. The title group includes metadata such as title, volume, issue, start page. If the number of matching words (tokens) in the title is less than fifty percent of total number of words in the longer of the two titles, the documents are considered to be different and the process proceeds to step 710 where both records are added to the consolidated search list.
If the number of matching tokens in the title is equal to, or more than, fifty percent of total number of words in the longer of the two titles, then the volume, issue and start page of each document are compared. If at least two out of three of these latter metadata values match, the works are considered the same and the process proceeds to step 708. Otherwise the works are considered different and the process proceeds to step 710. After duplicate works between targets have been identified, there is a consolidated result set created for further processing.
Returning to
The RAMDirectory sort requires a sort data structure called InMemoryWork to be defined which includes, for each record, the searching/sorting fields: title, author, standard number and standard number, type (DOI, Pubmed ID) and date, plus a reference to the entire set of metadata for each document. Documents from the consolidated record set were then mapped to this data structure and added to the in-memory Lucene index. Then this index was re-queried in the sort order requested by the calling client. This arrangement took about 100-250 milliseconds to pull 100 documents from four connector objects (400 works total), to build an in-memory index from these documents, to re-query and retrieve the document works in the desired sort order.
While the invention has been shown and described with reference to a number of embodiments thereof, it will be recognized by those skilled in the art that various changes in form and detail may be made herein without departing from the spirit and scope of the invention as defined by the appended claims.