Patent Application
20060184576
The invention relates generally to computer systems, and more particularly to an improved system and method for an extensible metadata architecture for digital images.
Image formats for digital images continue to evolve as the popularity of digital images grows. As applications for digital images expand, extensions to metadata formats used to describe digital images emerge regularly in evolving industry standards for specific image formats. However, existing implementations of codecs used to encode and decode an image format typically have fixed metadata formats for standard image types. For instance, one common approach for including metadata within an image file is to set aside a block of data for the metadata. When additional metadata is introduced in an image format, the implementation of an encoder and decoder in a codec built for that image format must be updated to handle the additional metadata. Unfortunately, the process for updating a codec is expensive and time consuming.
Furthermore, existing implementations of codecs may not provide support for third party metadata formats. Moreover, the tight coupling and dependencies between a codec and a particular image format prevent easy reuse of executable code for encoding and decoding metadata for different image formats that may be included in a single image file with multiple images. What is needed is a way for a computer system to easily adapt to the introduction of additional metadata for image formats and without having to release a new implementation of a codec in order to support additional metadata types for an image format. Such a system and method should also be able to support applications using third party implementations of image formats and extensions to existing image formats.
Briefly, the present invention provides an improved system and method for an extensible metadata architecture for digital images. To this end, executable software code may be operably coupled to a metadata query reader and a metadata query writer for requesting operations for manipulating metadata in an image file. The metadata query reader may be operably coupled to a decoder having a block reader for identifying metadata blocks in an image file and associating a metadata reader with each metadata block. Each metadata reader may then enumerate the metadata in the metadata block associated with that metadata reader. The metadata query writer may be operably coupled to an encoder having a block writer for associating a metadata writer with each metadata block to be written to an image file. Each metadata writer may then write metadata in the metadata block associated with that metadata writer.
Each metadata reader and each metadata writer may be operably coupled to an image file that may include metadata blocks with one or more nested metadata blocks. In one embodiment, a metadata block may include a nested metadata block for a different type of image. Each nested metadata blocks may also include, in turn, one or more nested metadata blocks so that metadata blocks may be nested for any number of levels. Correspondingly, a metadata reader and/or a metadata writer may be associated with each metadata block within a hierarchy of nested metadata blocks.
Furthermore, the system and method support a query language so that the metadata query reader and the metadata query writer may receive a query string. If the query string specified does not correspond to a fully qualified location, then the request may be resolved by a policy component which may be operably coupled to the metadata query reader and the metadata query writer. The policy component may resolve the query string by mapping the query string to a specific location within the metadata hierarchy.
Advantageously, the system and method allow the addition of new metadata types to be added to image files, and new metadata readers and writers may be easily added for decoding and encoding metadata in an image file associated with the new metadata types. The framework provided may accordingly support third party implementations of image formats and extensions to existing image formats. Moreover, enumerated metadata items may be easily changed using a fast metadata encoder to perform in-place editing of the metadata without the need to use a separate image stream for writing modified metadata to an image file. Other advantages will become apparent from the following detailed description when taken in conjunction with the drawings, in which:
Exemplary Operating Environment
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to: personal computers, server computers, hand-held or laptop devices, tablet devices, headless servers, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in local and/or remote computer storage media including memory storage devices.
With reference to
The computer 110 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computer 110 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by the computer 110. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer-readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media, discussed above and illustrated in
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110, although only a memory storage device 181 has been illustrated in
Extensible Metadata Architecture for Digital Images
The present invention is generally directed towards a system and method for an extensible metadata architecture for digital images. Metadata, as used herein, may mean data that may describe attributes of multimedia content, such as an image, including, without limitation, author, creation data, width, height, shutter speed, and so forth. Multimedia content may generally mean any type of video content including without limitation a digital image or digital video, any type of audio content including without limitation digital music, or a combination of video and audio content. The system and method may advantageously allow the addition of new metadata types to be added to image files, and new metadata readers and writers may be easily added for decoding and encoding metadata in an image file associated with the new metadata types. The framework provided may accordingly support third party implementations of image formats and extensions to existing image formats. To do so, the present invention may provide one or more metadata block readers that may be associated with a metadata block within an image file. A metadata block, as used herein, may mean a collection of one or more metadata items that may or may not be related. For example, some imaging formats may specify a collection of metadata items each represented by a keyword and value pair.
The present invention may also provide one or more metadata block writers that may be associated with a metadata block to be written to an image file. As will be seen, enumerated metadata items may be easily changed using a fast metadata encoder to perform in-place editing of the metadata without the need to use a separate image stream for writing modified metadata to an image file. As will be understood, the various block diagrams, flow charts and scenarios described herein are only examples, and there are many other scenarios to which the present invention will apply.
Turning to
Executable software 202 shown in
There may be a codec, such as codec 210, provided for each type of image file supported by the computer system. For example, there may be a codec for GIF, JPEG, PNG, TIFF, and other image file formats. Each codec may include a decoder 212 for decoding an image and an encoder 216 for encoding an image. Decoder 212 may include a metadata block reader 214 that may be operably coupled to one or more metadata readers 220. The metadata block reader 214 may identify recognizable metadata blocks 226 within an image file 224. Each metadata reader 220 may then provide functionality for parsing a type of metadata block recognized within an image file. In one embodiment, a decoder may thus read metadata in an image file by using the metadata block reader to identify recognizable metadata blocks within the image file and then may use the same or a different metadata reader to decipher the metadata items in each metadata block. In various embodiments, a decoder may also use one or more metadata readers to parse a metadata block 226 that may include nested metadata blocks 228. Either the same or a different metadata reader may be used to decipher the metadata items in each nested metadata block. In this way, a decoder may provide metadata items requested by a metadata query reader for an executable.
And encoder 216 may include a metadata block writer 218 that may be operably coupled to one or more metadata writers 222. The metadata block writer 218 may identify and add a metadata writer 222 for each metadata block 226 to be written within an image file 224. Each metadata writer 222 may then provide functionality for writing metadata items for a type of metadata block to be written within an image file. In one embodiment, an encoder may thus write metadata in an image file by using the metadata block writer to identify and add metadata writers for each metadata block to be written in the image file. Thus, an encoder may write metadata items requested by an executable using a metadata query writer.
The metadata reader 220 and the metadata writer 222 may be operably coupled to an image file 224 that may include metadata blocks 226 and image data blocks 230. A metadata block 226 may include one or more nested metadata blocks 228. In one embodiment, a metadata block may include a nested metadata block for a different type of image than the metadata block. Each nested metadata blocks may also include, in turn, one or more nested metadata blocks so that metadata blocks may be nested for any number of levels. Correspondingly, a metadata reader may be associated with each metadata block within a hierarchy of nested metadata blocks.
Those skilled in the art will appreciate that the metadata architecture shown in
Once a decoder may be found, the image stream stored in the file image may be parsed at step 306, for instance, by the instantiated decoder to confirm that the decoder may be able to read the metadata in the image file. At step 308, a metadata block reader may be found for the image file. In one embodiment, the decoder may use an API to find and instantiate the metadata block reader based upon the type of the image file. Next, a metadata block may be read at step 310 from the image file by a metadata block reader, and, then, a metadata reader that may decipher the metadata items in the metadata block may be found at step 312. For instance, a metadata reader may be registered in an embodiment by a metadata GUID that may identify the type of metadata block that it may decipher. In one embodiment, an unknown reader for the metadata block may be instantiated for the metadata block, if a reader cannot be found for a metadata block.
The metadata in the metadata block may then be enumerated at step 314. Once the metadata in the metadata block may be enumerated, it may be determined at step 316 if the last block of metadata may have been read from the image file. If not, then another metadata block may be read by returning to step 310. For each metadata block in the image file, steps 310 to 314 may be performed. Thus, each metadata block may be read, a metadata reader found, and the metadata in the metadata block may be enumerated in an embodiment. If it may be determined at step 316 that the last metadata block has been read, then the metadata enumerated may be displayed at step 318 or may used for any other purpose by an application, and processing for reading the metadata from the file may be finished.
In one embodiment, an acylic tree representing the metadata hierarchy may be constructed by using the steps described in
Once an encoder may be found, a metadata block writer may be found for the image file at step 406. In one embodiment, the encoder may use an API to find and instantiate the metadata block writer based upon the type of the image file. Next,an ordering of metadata blocks that may be written according to a format of the image file may be determined at step 408, for instance, by the instantiated encoder.
Then, a metadata writer may be found that may write the metadata items in the metadata block may be found at step 410. In an embodiment, an API may be invoked for finding and instantiating a metadata writer for the metadata block. The metadata block may then be written at step 412 in a metadata stream for the image file. Once the metadata block may be written in the metadata steam for the image file, it may be determined at step 414 if the last block of metadata in the ordering of metadata blocks to be written to the metadata stream has been written to the metadata stream for the image file. If not, then another metadata writer may be found by returning to step 410. For each metadata block in the ordering of metadata blocks to be written to the metadata stream for the image file, steps 410 to 412 may be performed.
Thus for each metadata block in the ordering, a metadata reader may be found, the metadata may be written in the metadata block, and the metadata block may be serially written into the metadata stream for the image file in an embodiment. If it may be determined at step 414 that the last metadata block in the ordering has been written, then the image file including the written metadata stream may be persistently stored at step 416 or may used for any other purpose by an application, and processing for writing the metadata in an image file may be finished.
Accordingly, executable software 202 may request that the metadata query reader 204 read a metadata item from an image file, such as TIFF image file 504 illustrated in
Metadata block reader 214 operably coupled to decoder 212 may find and instantiate one or more metadata readers 220 for parsing metadata blocks to decipher the metadata items in each metadata block. For example, one metadata reader able to parse metadata block 506 may be instantiated and may establish a reference 514 that may point to metadata block 506. Metadata block 506 may represent an image file directory (IFD), labeled IFD A, within TIFF image file 504 and may provide metadata information requested about image data block 508. For example, image data block 508 that may represent a page of a facsimile and the metadata item requested by executable 202 may be the Author of the facsimile image.
Upon receiving the metadata item requested, the executable may request that metadata query writer 208 change the name of the Author for the page of the facsimile. A fast metadata encoder 502 may be instantiated in order to perform in-place editing of metadata block 506. Fast metadata encoder 502 may be an encoder provided for writing metadata to a TIFF image file. The fast metadata encoder 502 may include a metadata block writer 218 that may be operably coupled to one or more metadata writers 222.
To do so, the metadata block writer 218 may use the list of metadata readers 220 operably coupled to the metadata block reader 214 to instantiate corresponding metadata writers 222 initialized with a reference to each metadata block. For example, a metadata writer, which may correspond to the metadata reader with the reference 514 to metadata block 506, may be instantiated and initialized with reference 516 pointing to metadata block 506. And another metadata writer, which may correspond to the metadata reader with the reference 518 to metadata block 510, may be instantiated and initialized with reference 520 pointing to metadata block 510. In this way, the metadata block writer 218 may identify and add a metadata writer 222 for metadata block 506 which may include the metadata item of Author requested to be changed for the image block data 508 within the TIFF image file 504. As part of instantiating metadata writer 222, reference 516 may be established to metadata block 506 to perform in-place editing to change the metadata item of Author within the TIFF image file stream 504.
In one embodiment, a mechanism may be provided for adding padding to a metadata block to allow sufficient space for a fast metadata encoder to write a metadata block with additional metadata, rather than having to write an entire image file. For example, padding may be added when encoding to an image file. Accordingly, an executable, such as an application program, may set an amount of padding by using the query language to identify a location within the metadata hierarchy to add padding. For instance, in the case of a TIFF image, an application may set a metadata item, such as “/ifd/PaddingSchema:padding” with the value of 4096 to allocate 4K of padding. Then, when a metadata writer may write to an image file, it may include 4K of additional space for use in future metadata updates.
In various embodiments, a tracking mechanism may also be used to record free space that may be made available when a metadata item may be removed. For example, free space may be recorded in a data structure as an offset and size of deleted data. When new metadata may be added during in-place editing, the metadata may be written in available free space that may be allocated from the data structure.
Once the metadata in the metadata block may be enumerated, a fast metadata encoder with a metadata block writer may then be instantiated for writing metadata in the image file at step 612. A metadata writer corresponding to the metadata reader may be instantiated at step 614. In one embodiment, the metadata block writer of the fast metadata encoder may use the list of metadata readers operably coupled to the metadata block reader to instantiate corresponding metadata writers initialized with a reference to each metadata block referenced by each respective metadata reader. After a metadata writer may be instantiated with a reference to a metadata block, metadata may be modified in the metadata block at step 616. Finally, the modified metadata block may be written to the image file that may be persistently stored, and processing for fast writing of metadata in an image file may be finished.
Advantageously, the system and method for fast writing of metadata in an image file may allow in-place editing of metadata without using a separate image stream for writing modified metadata to an image file. Instead, a fast metadata encoder may use the same image stream read by a decoder. In one embodiment, a mechanism may be provided for adding padding to a metadata block to accommodate metadata that may be later added. This may also allow sufficient space for a fast metadata encoder to simply write a metadata block with additional metadata, rather than having to write an entire image file.
Furthermore, the system and method support a query language so that a metadata query reader and a metadata query writer may receive a query string and resolve the string to a specific location within a metadata hierarchy. In an embodiment, the query string may be of a list of names identifying a navigation path through the metadata hierarchy. Each name may refer to a metadata namespace which may be a specific metadata block within the hierarchy. For example, the query string, “/ifd/exif/xmp/exif:Author”, may correspond to the navigation path=>“IFD reader”→“EXIF reader”→“XMP reader”→property “Author” in “exif” schema. By following the navigation path, the value for the property of “Author” may be retrieved within the “exif” schema. In one embodiment, each metadata namespace may also uniquely identified by a metadata global unique identifier (GUID) which may be used in place of the namespace name, “/{GUID}/[n]{GUID}/{GUID}:Value”, for example.
If the query string specified does not correspond to a fully qualified location, then the request may be resolved by the policy component which may map a keyword to a fully qualified location. The policy component may also be responsible for mapping high level requests for metadata to specific locations within the underlying metadata hierarchy. For instance, there may be several places within a metadata hierarchy where a metadata item may be stored. For example, “Author” may be stored in several different metadata blocks. The policy component may apply a set of rules, such as a preferred order of locations specified for a particular metadata item, to map a request for a metadata item to a specific location within the metadata.
As can be seen from the foregoing detailed description, the present invention provides an improved system and method for an extensible metadata architecture for digital images. The system and method may advantageously allow the addition of a new metadata type to be added to an image file, and new metadata readers and writers may be easily added for decoding and encoding metadata in an image file associated with the new metadata type. Moreover, an enumerated metadata item may be changed using a fast metadata encoder to perform in-place editing of the metadata. The present invention may also support a query language for mapping high level requests for metadata items to specific locations within the metadata hierarchy. As is now understood, the system and method thus provide significant advantages and benefits needed in contemporary computing.
While the invention is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention. For example, the present invention may be used for any type of multimedia content that may include metadata such as digital video, audio, and so forth.
The present invention is related to the following copending United States patent application filed concurrently herewith, assigned to the assignee of the present invention, and hereby incorporated by reference in its entirety, “System and Method for Extensible Metadata Architecture For Digital Images Using In-place Editing,” Attorney Docket No. 5271.
