Patent Application
20140351685
The invention relates to a method and to an apparatus for interactive review of a dataset. More specifically, a method and an apparatus are described, which allow a reviewer to easily access and analyze data of a large dataset using a graphic user interface.
Digital motion pictures, also referred to as a digital image sequence, often come along with metadata information. Preferably, metadata information is available for every single frame of the digital image sequence. Metadata is typically generated either by the camera itself or, more likely, by a real-time or non-real-time post processing algorithm. Metadata comprises a plurality of information, e.g. the noise level, image contrast or, in case of more sophisticated algorithms, a number of faces or cars or the like detected within the respective frame. However, for a 90 minutes digital movie with typically 24 frames per second a single metadata file comprises 129600 metadata elements. In professional post production environments, a human reviewer working on multiple datasets per movie or even with multiple movies having a plurality of datasets needs to inspect the metadata information for certain quality criteria. Due to the huge amount of information the human reviewer has a vital interest in optimizing the time needed to inspect the metadata.
Common techniques for displaying such large amount of data in a single plot, e.g. in a graph or bar plot, typically use downsampling in order to match the amount of data that is desired to be displayed (e.g. the more than 100000 metadata elements) to the available pixels or dots of a display unit. A display unit within the meaning of the term is e.g. a monitor especially used for display of the metadata, a graphical user interface (typically referred to as a GUI), e.g. a window that is used for that purpose. The available resolution is defined by the monitor itself, i.e. by the hardware resolution of the respective monitor or by a number of pixels inside a GUI-window that is used for display of the metadata.
In this regard EP 2 485 190 discloses a solution for representing a large amount of metadata corresponding to sequences of image frames on a displaying media with finite resolution. In order to ensure that a generated data graph fits the displaying media, the dataset is divided into clusters of a defined length. Each cluster represents a single value of the plot and is processed individually. Processing of the clusters is performed in such way that outliners are maintained. This is well suited for giving a quick overview of the metadata for quality control. However, while this approach ensures that important data samples are visible to the reviewer, how to allow the reviewer to handle such important data samples is not discussed.
It is an object of the invention to provide an improved solution for interactive review of a large dataset using a graphic user interface.
According to one aspect of the invention, a method for interactive review of a dataset comprises the steps of:
Accordingly, an apparatus configured to enable interactive review of a dataset comprises:
Similarly, a computer readable storage medium has stored therein instructions enabling interactive review of a dataset, which when executed by a computer, cause the computer to:
One aspect of the present invention is to simultaneously provide a representation of the dataset, e.g. metadata associated to a video sequence, in a coarse and a fine scale plot. In this way, in addition to the quick overview of the dataset also a direct graphical representation of a subset of the dataset without restrictions caused by the media resolution is available for a detailed inspection.
Preferably content from which the dataset is derived, e.g. a single frame of the video sequence, is displayed together with the displayed coarse view and fine view, e.g. for interactive data review and processing purposes. In this way the proposed approach combines visualization of a dataset on different temporal scales and display of the underlying content for interactive review. The representation of the dataset preferably serves as a timeline for interactive jumping to, selecting of and fast preview of content. This easily allows further inspection and handling of anomalies that have been discovered on the coarse scale.
For a better understanding the invention shall now be explained in more detail in the following description with reference to the figures. It is understood that the invention is not limited to this exemplary embodiment and that specified features can also expediently be combined and/or modified without departing from the scope of the present invention as defined in the appended claims.
In the following the invention shall be described at the example of interactive review of metadata corresponding to an image sequence comprising frames. It is assumed that metadata varies frame by frame. Of course, use of the invention is not limited to this example. The general approach is likewise applicable to other types of datasets.
Above the coarse view 2 a fine view 3 representing a section of the set of metadata is shown, favorably with the metadata for each single frame. The section is advantageously highlighted in the coarse view 2 with a section window 6. The size of the section interval is preferably set in a way that metadata for each frame is represented by at least one horizontal pixel. Typically, for convenience a couple of pixels are used together with spacing between the graphical representation of the data. In addition to the displayed metadata, advantageously the content of the current frame associated to the metadata is displayed in a frame window 4. In the fine view 3 the metadata corresponding to the current frame under investigation is shown in the middle of the fine view 3, favorably identified by a marker 7 for easy inspection of the metadata corresponding to the frames before and after. As an option, index frames are additionally displayed in an index view 5 together with the metadata figures.
Interactive behavior is achieved by clicking in the coarse view 2 or the fine view 3, e.g. by touching or pointing, and jumping to the corresponding frame. To this end a processing device with a keyboard, a mouse or a touch user interface is used to generate the dynamic display, e.g. a general purpose PC or dedicated hardware. Upon jumping to the target frame, the section window 6 in the coarse view 2, the metadata displayed in the fine view 3 and the content view in the frame window 4 are refreshed.
The coarse view 2 favorably is also used as a multi-purpose timeline to set and visualize clip markers 8. This is especially useful when the visualization is accompanied by a processing engine, e.g. for video post-processing or restoration.
In a general-purpose graphical user environment application, windows may usually be resized dynamically by the user. When this is the case, the coarse view 2 is reprocessed as described above in order to adjust to the new virtual screen resolution. The fine view section window 3 is then adjusted accordingly, e.g. by changing the range of data that is displayed or by changing the number of pixels that are used for representing a data value of the space between adjacent data values.
A method for interactive review of a dataset is schematically illustrated in
An apparatus 20 configured to implement the method of
Although the invention has been described hereinabove with reference to a specific embodiment, it is not limited to this embodiment and no doubt further alternatives will occur to the skilled person that lie within the scope of the invention as claimed.
| Number | Date | Country | Kind |
|---|---|---|---|
| 13305688.7 | May 2013 | EP | regional |
| 13306153.1 | Aug 2013 | EP | regional |
