This is a non-provisional application claiming the benefit of International application number PCT/KR2008/002285 filed Apr. 23, 2008.
The present invention relates to an apparatus and method for retrieving multimedia contents; and, more particularly, to multimedia contents retrieving apparatus that can retrieve multimedia contents represented based on Moving Picture Experts Group 7 (MPEG-7) by transforming a user query into an MPEG-7 query format, and a method thereof.
This work was supported by the IT R&D program of MIC/IITA [2005-S-117-03, “Development of Intelligent Personal Media Managing Technology for Ubiquitous Environment”].
Moving Picture Experts Group 7 (MPEG-7) is an international standardization on the architectures of metadata representing multimedia information, such as image, audio and moving picture. An MPEG-7 query format is used to retrieve multimedia contents represented based on the MPEG-7. An MPEG-7 multimedia contents retrieving system retrieves multimedia contents related to a query inputted in an MPEG-7 query format.
The MPEG-7 query format defines syntaxes for retrieving MPEG-7 documents. The syntaxes can represent diverse types of queries that can be used for the retrieval of MPEG-7 documents. For example, they can represent not only natural sentence-type query such as “an image with mountain” but also example-based query using a multimedia file as a query and MPEG-7 textual description-based query.
While representing such diverse queries, referring to the same or different portions of an MPEG-7 document occurs frequently. To be specific, there is a case where more than one retrieval condition should be all satisfied in the same structure. For example, to retrieve moving picture segments with “mountain” and “sea”, the presence of “mountain” and “sea” could be represented for one region. As for joint operation, two different MPEG-7 documents should be referred to. For this, it should be clearly represented that two different documents are referred to.
Conventional MPEG-7 query formats may satisfy more than two retrieval conditions within the same architecture, but they have a shortcoming that they cannot clearly represent reference to two different MPEG-7 documents.
An embodiment of the present invention, which is invented to resolve the problem, is directed to providing a Moving Picture Experts Group 7 (MPEG-7) query format that can satisfy more than two retrieval conditions within the same structure and clearly represent that different MPEG-7 documents are referred to.
Another embodiment of the present invention is directed to providing an apparatus and method that can accurately retrieve multimedia contents by precisely analyzing the meaning of a user query in a retrieving process.
Other objects and advantages of the present invention can be understood by the following description, and become apparent with reference to the embodiments of the present invention. Also, it is obvious to those skilled in the art of the present invention that the objects and advantages of the present invention can be realized by the means as claimed and combinations thereof.
In accordance with an aspect of the present invention, there is provided a method for retrieving multimedia contents, which includes: representing a user query by using an indicator indicating a specific region of a Moving Picture Experts Group 7 (MPEG-7) document and a reference for referring to the indicator; analyzing a meaning of the user query represented by using the indicator and the reference to thereby produce an analysis result; and retrieving multimedia contents according to the analysis result.
In accordance with another aspect of the present invention, there is provided a method for processing a user query to retrieve multimedia contents, which includes: receiving a query for retrieving multimedia contents from a user; representing the user query by using an indicator for indicating a specific region of an MPEG-7 document and a reference for referring to the indicator.
In accordance with another aspect of the present invention, there is provided an apparatus for retrieving multimedia contents, which includes: a query input unit for receiving a query for retrieving multimedia contents from a user; a query representation unit for representing the user query inputted through the query input unit by using an indicator for indicating a specific region of an MPEG-7 document and a reference for referring to the indicator; a query analysis unit for analyzing a meaning of the user query represented in the query representation unit by using the indicator and the reference to thereby produce an analysis result; and a contents retrieval unit for retrieving multimedia contents according to the analysis result.
In accordance with another aspect of the present invention, there is provided a data structure for representing a user query to retrieve multimedia contents, which includes: an indicator for indicating a specific region of an MPEG-7 document; and a reference for referring to the indicator.
The present invention described above provides an MPEG-7 query format that can satisfy more than two retrieval conditions within the same structure and clearly represent that different MPEG-7 documents are referred to. Also, since the meaning of a user query is precisely analyzed during a retrieving process, it is possible to retrieve multimedia contents that accurately agree with the user query.
The advantages, features and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings, which is set forth hereinafter. When it is considered that detailed description on a related art may obscure a point of the present invention, the description will not be provided herein. Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings.
In step S10, a user query is represented as a query for retrieving multimedia contents. The user query is represented using an indicator and a reference for referring to the indicator to precisely represent the meaning of the user query. The indicator denotes a specific region of a Moving Picture Experts Group 7 (MPEG-7) document, and the reference is used to refer to the indicator. For example, when moving picture segments with “mountain” and “sea” is retrieved for, there is an indicator for a moving picture segment and a reference of the indicator may represent the presence of “mountain” and another reference, the presence of “sea.” In subsequent joint operation, two indicators may be established for two different MPEG-7 documents, respectively, and each of the two indicators may have references to clearly represent the two different MPEG-7 documents from each other.
In step S20, a query processor analyzes the user query represented using the indicator and references. In step S30, a retrieval engine retrieves multimedia contents related to the user query analyzed in the query processor and, in step S40, provides a retrieval result.
An MPEG-7 document is described in an XML format, and an indicator indicates a specific region of the MPEG-7 document. For this, the indicator region descriptor 102 is used to designate an uppermost node of the specific region. The indicator limiting descriptor 103 is used when an additional limiting condition is needed in connection with a region represented by indicator region descriptor. The indicator ID number 101 is used when an indicator is referred to.
The following Table 1 shows
In
For example, a query for “retrieving images whose horizontal length×vertical length is greater than 1024×768” can be represented as the following Table 3 based on the XML schema defined in the Tables 1 and 2. In the Table 3, an indicator is referred to by using a reference “href,” and a specific part related to a region indicated by the indicator can be indicated by describing an additional path.
In the step S504 where the indicator and the reference are processed, references referring to the same indicator are regarded as values for referring to a value in the inside of the same region to analyze the meaning of the user query. For example, since “@height” and “@width” refer to “VisualCodingFrameID” in the user query, it is analyzed that the two refer to a value in the inside a region indicated by the “VisualCodingFrameID.”
The query input unit 702 receives a query for retrieving multimedia contents from a user. The query representation unit 704 represents the user query inputted through the query input unit 702 into an MPEG-7 query format by using an indicator indicating a specific region of an MPEG-7 document and a reference for referring to the indicator. An indicator includes an indicator ID number used for a reference to refer to the indicator, a descriptor for describing limiting conditions for the region indicated by the indicator, and a descriptor for designating an uppermost node of the region indicated by the indicator. The user query is represented in an XML format.
The query analysis unit 706 analyzes the meaning of the user query represented using the indicator and the reference in the query representation unit 704. The query analysis unit 706 includes an XML parser 712 for parsing a user query, a descriptor processor 714 for processing an indicator and a reference based on the parsing result of the XML parser 712, and a meaning analyzer 716 for analyzing the meaning of the user query based on the indicator and the reference processed in the descriptor processor 714. The contents retrieval unit 708 retrieves multimedia contents according to the analysis result of the user query analysis unit 706. The contents retrieval unit 708 may retrieve a database 718 or search the internet 722 through a communication unit 720. The database 718 may be set up inside or outside the multimedia contents retrieving apparatus 700. The output unit 710 provides multimedia contents retrieved by the contents retrieval unit 708 to the user.
The method of the present invention described above may be realized as a program and stored in a computer-readable recording medium, such as CD-ROM, RAM, ROM, floppy disks, hard disks, magneto-optical disks and the like. Since this process can be easily implemented by those skilled in the art to which the present invention belongs, further description will not be provided herein.
While the present invention has been described with respect to the specific embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the following claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2007-0039475 | Apr 2007 | KR | national |
10-2008-0035896 | Apr 2008 | KR | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/KR2008/002285 | 4/23/2008 | WO | 00 | 10/22/2009 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2008/130182 | 10/30/2008 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
6070167 | Qian et al. | May 2000 | A |
6411724 | Vaithilingam et al. | Jun 2002 | B1 |
6490370 | Krasinski et al. | Dec 2002 | B1 |
6593936 | Huang et al. | Jul 2003 | B1 |
6629088 | Rising, III | Sep 2003 | B1 |
6961446 | Imagawa et al. | Nov 2005 | B2 |
7231394 | Walker et al. | Jun 2007 | B2 |
7624326 | Watanabe et al. | Nov 2009 | B2 |
7664830 | Rising, III | Feb 2010 | B2 |
20020063718 | Choi et al. | May 2002 | A1 |
20040205510 | Rising, III | Oct 2004 | A1 |
20040267720 | Liu et al. | Dec 2004 | A1 |
20060112124 | Ando et al. | May 2006 | A1 |
20060153537 | Kaneko et al. | Jul 2006 | A1 |
20070233673 | Seo et al. | Oct 2007 | A1 |
Number | Date | Country |
---|---|---|
1396770 | Feb 2003 | CN |
1815612 | Aug 2006 | CN |
1 276 327 | Jan 2003 | EP |
2005-501343 | Jan 2005 | JP |
10-2001-0092899 | Oct 2001 | KR |
10-2002-0006623 | Jan 2002 | KR |
10-2005-0043901 | May 2005 | KR |
10-2005-0066790 | Jun 2005 | KR |
WO 2004023341 | Mar 2004 | WO |
WO 2006009768 | Jan 2006 | WO |
Entry |
---|
Doller et al, “Towards an MPEG-7 Query Language”, University Passau, 2006. |
Fatemi et al, “How to retrieve multimedia documents described by MPEG-7”, Switzerland, 2004. |
Martinez et al, “MPEG-7: the generic Multimedia Content Description Standard”, IEEE, 2002. |
Liu et al, “Queries of Digital Content Descriptions in MPEG-7 and MPEG-21 XML documents”, 2002. |
Mamou et al, “A Query Language for Multimedia Content”, ACM, 2007. |
Nack et al, “Everything You Wanted to Know About MPEG-7: Part 2”, IEEE, 1999. |
Westermann et al, “An Analysis of XML Database Solutions for the Management of MPEG-7 Media Descriptions”, ACM, 2003. |
Ndjiki-Nya et al, “Video Content Analysis Using MPEG-7 Descriptors”, Heinrich-Hertz-Institut, Germany, 2004. |
Seo, H.-C., et al., “Revision of Proposed Input Query Format for MPEG-7 Query Format,” Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q6), No. M14421 (Apr. 18, 2007), 32 pages. |
Seo, H.-C., et al., “CE Report for Query Expression of MPEG-7 Query Format,” Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q6), No. M14420 (Apr. 18, 2007), 29 pages. |
K. Adistambha et al., “The MPEG-7 Query Format: A New Standard in Progress for Multimedia Query by Content,” ISCIT 2007, Oct. 17-19, 2007, pp. 479-484. |
N. Fatemi et al., “An XQuery Adaptation for MPEG-7 Documents Retrieval,” XML Conference & Exposition, Dec. 2003, pp. 1-9. |
Ji-Hoon Kang et al., “An XQuery engine for digital library systems that support XML data,” Proceedings of the 2004 International Symposium on Applications and the Internet Workshops, Jan. 2004, pp. 255-259. |
Sun, Y., et al., “Content-Based Multimedia Retrieval Model Based MPEG-7 Standard,” Mini-Micro Systems, vol. 25, No. 3, Mar. 2005, China Academic Journal Electronic Publishing House, pp. 470-473. |
Shi, L. et al., “A Semantic Image Retrieval System Based on MPEG-7,” Telecommunicanons for Electric Power System, vol. 26, No. 156, Oct. 10, 2005, China Academic Journal Electronic Publishing House, pp. 10-13. |
Melton, J., et al., “XML Syntax for XQuery 1.0 (XQueryX),” W3C Recommendation, Jan. 23, 2007, http://www.w3.org/TR/2007/REC-xqueryx-20070123, pp. 1-98. |
Number | Date | Country | |
---|---|---|---|
20100131557 A1 | May 2010 | US |