Processing a Non-XML Document for Storage in a XML Database

Abstract

A method for processing a non-XML document for storage in a XML database. The method comprises analyzing the non-XML document and extracting metadata from the non-XML document. The method then generates a shadow XML document for the non-XML document in accordance with a predetermined XML schema, wherein the shadow XML document comprises the metadata extracted from the non-XML document. The XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure. The shadow XML document and the non-XML document are then stored in the XML database.

Description

PRIORITY CLAIM

This application claims benefit of priority of European application no. EP ______ titled “Method and System for Processing a Non-XML Document for Storage in a XML Database”, filed May 25, 2007, and whose inventor is Dr. Michael Gesmann.

INCORPORATION BY REFERENCE

European application no. ______ titled “Method and System for Processing a Non-XML Document for Storage in a XML Database”, filed May 25, 2007, and whose inventor is Dr. Michael Gesmann, is hereby incorporated by reference in its entirety as though fully and completely set forth herein.

1. TECHNICAL FIELD

The present invention relates to a method and a database system for processing a non-XML document for storage in a XML database

2. DESCRIPTION OF THE RELATED ART

XML (eXtensible Markup Language) databases are one of the most important technical tools of modern information societies. The high degree of flexibility of such a database allows to store and to retrieve data in a very efficient manner. Generally, XML databases are designed for XML documents. However, in the prior art it is also known to extend a XML database so that it is capable of storing other types of documents. For example the XML database Tamino of applicant is adapted to store non-XML documents such as text files, MS Office files, PDF files, images and audio files, etc. To enable the retrieval of such non-XML documents from the database, it is known to analyze a non-XML document to be stored and to extract some metadata for generating a so-called XML shadow document corresponding to the non-XML document. Using XQuery, the shadow XML document can later be searched and the corresponding non-XML document can be retrieved.

The analysis and the extraction of the metadata is typically performed by a piece of software of the database system, wherein this software is specific for a certain type of non-XML document. Alternatively, a more generic analysing and extracting software can be provided for the handling of non-XML documents, which comprises several components, each of which is specifically designed to process a predefined type of non-XML document. Similar methods and systems are known from U.S. Pat. No. 6,549,922 and published US patent application US 2005050086.

However, all of the methods and systems of the prior art for processing non-XML documents for storage in a XML database use a predefined format or schema for the generated XML documents. In other words, all types of non-XML documents will always lead to a certain type of shadow XML document. For example the above mentioned Tamino database of applicant uses a fixed XML schema, which is in accordance with the “Dublin Core Metadata Initiative” (http://dublincore.org/) and follows the OpenOffice-formats (http://openoffice.org) As a result, the content of the shadow XML document is sometimes not very useful if the fixed XML schema does not allow to store metadata on the shadow XML document with meaningful information. Searches for the non-XML documents based on the shadow XML documents known in the prior art are therefore ineffective and slow.

The above outlined approach for processing non-XML documents furthermore leads to problems, if new types of non-XML documents are to be processed for storing and/or if software components of different providers are to be used for handling different types of non-XML documents. This applies in particular, if the new type of document is not a standard office document but for example an image, wherein the metadata to be extracted (e.g. color distribution, resolution, size or any result of an image processing software) is very different from the metadata for a standard office document.

The present invention is therefore in one aspect based on the technical problem to provide a more flexible approach for generating shadow XML documents, which overcomes at least some of the above explained disadvantages of the prior art.

3. SUMMARY

One embodiment of the invention relates to a method for processing a non-XML document for storage in a XML database comprising the steps of

• • generating a shadow XML document for the non-XML document in accordance with a predetermined XML schema, the shadow XML document comprising metadata extracted from the non-XML document,
  • storing the shadow XML document and the non-XML document in the XML database;wherein the XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure.

The received non-XML document may be any of various types of documents, such as a text file, e.g., a .pdf document or a Microsoft Office document; an image file, an audio file; a movie file, or other types of documents or files. The received non-XML document may also be a compressed file using any of various types of compression, such as a compressed text file (using, e.g., LZ compression), a compressed image file (e.g., JPEG), a compressed movie file (e.g., MPEG), etc.

One embodiment of the method may store two separate documents in the XML database, the non-XML document itself and the corresponding shadow document. The structure of the shadow XML document, as defined in the XML schema, is flexible and may vary. This is because there is no complete definition of the structure of the XML content wrapped by the wrapping element of the XML schema. On the contrary, any well-formed XML content can be arranged inside the wrapping element. As a result, the described method provides more flexibility for the components generating the XML shadow document, since they no longer have to strictly adhere to an inflexible, fixed XML schema.

Even though the wrapping element can wrap any kind of well-formed XML content regardless of its structure and content, the XML content of the wrapping element is adapted to be searched using an XQuery with a wildcard.

According to another embodiment, the method further comprises creating an index on the shadow XML document, wherein in one example information for the index is defined in the XML schema. Accordingly, the flexibility of the structure of the XML content of the wrapping element is combined with some definitions, which are adapted for providing an index for later search and retrieval of the shadow XML documents and their non-XML counterparts. In one embodiment, the shadow XML document comprises a unique identifier identifying the corresponding non-XML document.

Another embodiment of the invention concerns a XML database system with an analyzer adapted to analyze a non-XML document. This embodiment may comprise at least one extractor adapted to extract metadata from the non-XML document and to generate a shadow XML document for the non-XML document in accordance with a predefined XML schema, wherein the shadow XML document comprises the metadata. The XML database system further comprises a wrapper adapted to wrap the extracted metadata in the shadow XML document, wherein the structure of the wrapped metadata is at least partly undefined in the XML schema.

The analyzer, the extractor and the wrapper are in one embodiment provided as an extension of a database server, which therefore provides all the functionality for the structured storage of non-XML documents and their respective metadata.

Additionally, the XML database system may further comprise an index based on content of the shadow XML document. This index can be based on information in the wrapped metadata of the shadow XML document.

Further modifications of the described methods and XML database systems are defined in further dependent claims.

5. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1: An exemplary embodiment of the XML database system of the present invention;

FIG. 2: An example of a flexible XML schema in accordance with an embodiment of the present invention;

FIG. 3: A section of a shadow XML document in accordance with the XML schema of FIG. 1; and

FIG. 4: A further example of a flexible XML schema adapted for indexing.

While the invention is susceptible to various modifications and alternative forms specific embodiments are shown by way of example in the drawings and may herein be described in detail. It should be understood however, that drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed. But on the contrary the invention is to cover all modifications, equivalents and alternative following within the spirit and scope of the present invention as defined by the appended claims.

6. DETAILED DESCRIPTION OF THE EMBODIMENTS

Terms

The following is a glossary of terms used in the present application:

Memory Medium—Any of various types of memory devices or storage devices. The term “memory medium” is intended to include an installation medium, e.g., a CD-ROM, floppy disks 104, or tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; or a non-volatile memory such as a magnetic media, e.g., a hard drive, or optical storage. The memory medium may comprise other types of memory as well, or combinations thereof. In addition, the memory medium may be located in a first computer in which the programs are executed, and/or may be located in a second different computer which connects to the first computer over a network, such as the Internet. In the latter instance, the second computer may provide program instructions to the first computer for execution. The term “memory medium” may include two or more memory mediums which may reside in different locations, e.g., in different computers that are connected over a network. The term “memory medium” encompasses a storage area network (SAN), a database, etc.

In the following, exemplary embodiments of the XML database system and method of the present invention are described. It will be understood that functionality described below can be implemented in a number of alternative ways, for example on a single database server, a distributed arrangement of a plurality of database servers, with an integral storage or an external storage, etc. None of these implementation details is essential for the present invention.

FIG. 1 presents an overview of an exemplary XML database system 1. The system 1 generally serves to store and to retrieve XML documents (not shown in FIG. 1). However, the XML database system 1 of FIG. 1 is also capable to process non-XML documents such as the exemplary media file 10 shown in FIG. 1. The media file 10 can be any type of non-XML document, e.g. a video file, an audio file, a combination thereof, an image, a MS Office document, an arbitrary set of binary data such as measurement results, etc. In one embodiment various of the elements in FIG. 1, e.g., elements 2, 4, 5 (6-8) and 11, may be implemented as software executing on a computer system or other type of processing device.

For processing the media file 10, the XML database system 1 comprises in one embodiment a document processor 2. The document processor 2 drives the process for storing a document. As illustrated by the dotted arrow on the left side of FIG. 1, the media file 10 is stored in the storage means 3, for example a RAID array (not shown) or a similar storage device of the XML data base system 1. Any volatile or non-volatile storage means known to the person skilled in the art can be used as the storage means 3 of the XML database system 1.

In addition, the media file 10 is forwarded to a schema processor 4. The operation of the schema processor 4 and the further elements of the XML database system 1 which are shown on the right side of FIG. 1 serves to process the media file 10 so that it can be searched and retrieved similar to other XML documents stored in the database. In the exemplary embodiment of FIG. 1, the schema processor 4 provides information about a ServerExtension 5 to be called, which comprises the further processing elements 6-8. However, it is to be noted that the components 6-8 as well as the schema processor 4 could also be integrated into the standard processing engine of a database sever (not shown in FIG. 1) of the overall XML database system and do not have to be provided as separate entities. The provision of a ServerExtension 5 described below facilitates the upgrading of an existing XML database system with the functionality for the handling of non-XML files, such as the media file 10.

In a first step, an analyzer 6 analyzes the media file 10 and determines which extractors 7 are to be called. Each extractor 7 processes the media file 10 and generates content for a shadow XML document 20. Depending on the type of media file 10, different extractors 7 can be used. For example, there might be an extractor 7 performing image processing on an image and outputting metadata about the image such as its resolution, colour distribution or any other type of image related information. Another extractor 7 may be adapted to process video files and a further extractor 7 may be provided for extracting metadata about an audio file, such as its length, the sampling frequency etc. Whereas in the described embodiment there are distinct extractors 7 for each type of media file 10, there could also be one or more integrated extractors 7 being able to extract metadata from more than one type of file.

Finally, a wrapper 8 creates a common doctype element around the generated XML content. It is to be noted that this content, which was generated by one or more extractors 7, can be any well-formed XML content, regardless of its specific structure. Therefore, the described embodiment of the XML database system can be quickly adapted to new media files by adding or modifying an extractor 7 so that the new type of files can be processed.

Whereas the schema processor 4, the analyzer 6, the extractors 7 and the wrapper 8 have been described and shown in FIG. 1 as separate components it is also possible to integrate one or more of theses components into a single processing component of the described XML database system 1.

Although the resulting XML shadow document 20 is fully flexible with respect to the structure and the content of the XML metadata generated by the extractors 7 from the media file 10, it is nevertheless in accordance with a predefined XML schema. An example of such a flexible XML schema 50 for shadow XML documents of the XML database system is shown in FIG. 2. As can be seen, the XML schema 50 comprises among others a wrapping element 51, which contains in the corresponding XML shadow document the XML content generated by one or more of the extractors

Looking more in detail, the XML schema 50 of FIG. 2 refers to a ServerExtension “NewBlobIndexer”, which is one possible embodiment of the ServerExtension 5 shown in FIG. 1. The ServerExtension NewBlobIndexer calls for a given non-XML document a suitable extractor 7. The extractor 7 then generates in accordance with its specific implementation the content for the shadow XML document 20. To this end, the ServerExtension NewBlobIndexer takes the XML content as provided by the one or more extractors 7 and generates a root element having in this embodiment the name “wrapping element”. The structure of the XML content of the wrapping element is not defined so that any arbitrarily structured XML content can be comprised in the wrapping element. In this embodiment, there is no unified structure of the resulting XML shadow documents generated for media files of different types.

In addition to the elements shown in FIG. 2, the XML schema 50 for the shadow XML documents 20 may comprise further elements, for example to meet the requirements of the above mentioned “Dublin Core Metadata Initiative” and/or the OpenOffice-formats.

An exemplary section of a XML shadow element 20 generated as explained above is shown in FIG. 3. As can be seen, it comprises a “wrapping element”, which contains XML content with information about a photo, such as its width and height, the photographed subject, the photographer etc. As mentioned before, the structure of the XML metadata in the wrapping element of the shadow XML element 20 is arbitrary. For example, the various photo related information may be differently arranged within the wrapping element or there might be other information about the photo, for example the exposure time or the selected lens used for taking the picture.

Regardless of the specific structure of the XML content of the wrapping element, it is still possible to execute queries on the shadow XML documents. One option of such a query is the use of wildcards, which do not require information about a specific structure of the XML content of the shadow XML document. For example a query for

• /*[//author=“X”]
• will yield all shadow XML documents, which somewhere have an element “author” with the value “X”. In another example the query
• /WrappingElement[//Creator=“X” or //Photographer=“X”]
• yields all documents having a creator or photographer “X” somewhere in the wrapping element. As a result, in spite of the increased flexibility for the generation of the shadow XML documents 20, it is still possible to perform powerful searches and to effectively retrieve the relevant shadow XML documents. Once a desired shadow XML document 20 has been retrieved, the respective non-XML document 10 can also be immediately accessed using for example a unique identifier identifying for each shadow document 20 the corresponding non-XML document 10.

In addition to the generation of the shadow XML documents 20, the XML database system 1 of FIG. 1 may comprise an index processor 11, which creates indexes for shadow XML documents 20. The result of the index processing is in the embodiment of FIG. 1 also stored in the storage means 3 of the overall XML database system 1 (cf. the corresponding dotted arrow in FIG. 1). However, the index information may be stored in other locations. For example, the index information can also be stored separately from the media file 10 and the corresponding shadow XML document 20.

There are various ways how to generate an index over the shadow XML documents 20. In one embodiment, one or more attributes and/or elements for the index are defined in the XML schema for the shadow XML documents. An example for such an extended XML schema is shown in FIG. 4. As can be seen, an attribute 52 and an element 53 are defined, which allow to base an index over the shadow XML documents thereon. The attribute “mimetype” is defined as a string and indicates the type of the media file 10, for example a jpeg file (cf. also the shadow XML document of FIG. 3). The element “Photographer” is also defined as a string and can be used to create an index for photos according to the photographer having taken the respective photo. Using the attribute and the element defined in the XML schema of FIG. 4, a query can be defined to obtain all jpeg files of a certain photographer.

It is to be noted that the definition of the information necessary for the index does not imply a certain XML structure for the content of the wrapping element so that the above explained flexibility is preserved. On the contrary, the attribute and the element for the index in the example of FIG. 4 can be defined regardless of the specific structure of the XML content of the wrapping element. In other words querying the index based on the defined attribute and the defined element of the XML schema will retrieve a XML shadow document, whenever there is agreement of the search values of the attribute and the element with the values in a shadow XML document.

Although the system and method of the present invention has been described in connection with various embodiments, it is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents, as can be reasonably included within the spirit and scope of the invention as defined by the appended claims.

Claims

1. Method for processing a non-XML document for storage in a XML database comprising: a. generating a shadow XML document for the non-XML document in accordance with a predetermined XML schema, the shadow XML document comprising metadata extracted from the non-XML document;b. storing the shadow XML document and the non-XML document in the XML database;c. wherein the XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure.

2. Method according to claim 1, wherein the wrapping element is defined as a root element of the XML schema.

3. Method according to claim 1, wherein the wrapping element is defined using a XML doctype definition.

4. Method according to claim 1, wherein the XML content of the wrapping element is adapted to be searched using an XQuery with a wildcard.

5. Method according to claim 1, further comprising creating an index on the shadow XML document.

6. Method according to claim 5, wherein information for the index is defined in the XML schema.

7. Method according to claim 1, wherein the non-XML document comprises an image and wherein the metadata are extracted using an image processing software.

8. Method according to claim 1, wherein the non-XML document comprises a text document.

9. Method according to claim 1, wherein the non-XML document comprises an audio and/or a video file.

10. Method according to claim 1, wherein the non-XML document is a compressed file.

11. Method according to claim 1, wherein the shadow XML document comprises a unique identifier identifying the corresponding non-XML document.

12. A memory medium comprising program instructions for processing a non-XML document for storage in a XML database, wherein the memory medium comprises program instructions executable to: a. generate a shadow XML document for the non-XML document in accordance with a predetermined XML schema, the shadow XML document comprising metadata extracted from the non-XML document, wherein the XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure;b. store the shadow XML document and the non-XML document in the XML database.

13. The memory medium of claim 12, wherein the wrapping element is defined as a root element of the XML schema.

14. The memory medium of claim 12, wherein the wrapping element is defined using a XML doctype definition.

15. The memory medium of claim 12, wherein the XML content of the wrapping element is adapted to be searched using an XQuery with a wildcard.

16. The memory medium of claim 12, wherein the program instructions are further executable to create an index on the shadow XML document.

17. A memory medium which implements an XML database, wherein the memory medium stores: a non-XML document; anda shadow XML document, wherein the shadow XML document has a predetermined XML schema, wherein the shadow XML document is generated from the non-XML document in accordance with the predetermined XML schema, the shadow XML document comprising metadata extracted from the non-XML document, wherein the XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure.

18. A XML database system comprising: a. an analyzer adapted to analyze a non-XML document;b. at least one extractor adapted to extract metadata from the non-XML document and to generate a shadow XML document for the non-XML document in accordance with a predefined XML schema, the shadow XML document comprising the metadata; andc. a wrapper adapted to wrap the extracted metadata in the shadow XML document, wherein the structure of the wrapped metadata is at least partly undefined in the XML schema.

19. The XML database system of claim 18 further comprising a storage unit adapted to store both the non-XML document and the shadow XML document.

20. The XML database system of claim 18, wherein the analyzer, the extractor and the wrapper are provided as an extension of a database server.

21. The XML database system of claim 18, further comprising an index based on content of the shadow XML document.

22. The XML database system of claim 21, wherein the index is based on information in the wrapped metadata of the shadow XML document.

23. The XML database system of any of claim 18, wherein the shadow XML document comprises a unique identifier identifying the corresponding non-XML document.

24. A system, comprising: an input for receiving a non-XML document;a memory medium comprising program instructions;a processor coupled to the memory medium, wherein the processor is operable to execute the program instructions from the memory medium to:a. generate a shadow XML document for the non-XML document in accordance with a predetermined XML schema, the shadow XML document comprising metadata extracted from the non-XML document;b. store the shadow XML document and the non-XML document in an XML database;c. wherein the XML schema comprises a wrapping element adapted to wrap XML content of an at least partly undefined XML structure.

25. A system, comprising: an input for receiving a non-XML document;a memory medium comprising program instructions;a processor coupled to the memory medium, wherein the processor is operable to execute the program instructions from the memory medium to:a. analyze the non-XML document;b. extract metadata from the non-XML document;c. generate a shadow XML document for the non-XML document in accordance with a predefined XML schema, the shadow XML document comprising the metadata; andd. wrap the extracted metadata in the shadow XML document, wherein the structure of the wrapped metadata is at least partly undefined in the XML schema.