The present invention relates to determining video frame type, in particular distinguishing between a two-dimensional (2D) and three-dimensional (3D) video frames.
Stereoscopic video displays can be used to display both 2D and 3D video signals, by processing the frames of the signals depending on the type of signal. The type of the signal can be specified manually by the user, which can be troublesome for inexperienced users. The type of signal can be also specified by supplementary data included in the signal itself or by a supplementary signal, which requires the video display unit to be able to decode and recognize the supplementary data or the supplementary signal.
A US patent application US2009/0009508 presents an apparatus and a method for driving a 2D/3D switchable display, which includes an image mode determination unit determining whether input image signals of continuous frames are in a 2D mode or 3D mode. The 3D mode is recognized by determining a syntax indicating a stereo or a multiview image included in the header information of the input image signal. Alternatively, the image mode can be determined based on the presence of a stereo sync signal. Therefore, the mode determination requires data or signals supplementary to the basic image data.
A PCT patent application WO2010/014973 presents a method and an apparatus for encoding or tagging a video frame which provide a way to indicate, to a receiver, whether the video content is 3-D content or 2-D content, by replacing lines of at least one video frame in a 3-D content with a specific color or pattern. Therefore, the method is useful only for receivers, which are able to recognize the specific color or patterns as an indication of a 3-D content.
In case the display lacks the functionality of decoding supplementary data included in the signal or is not able to decode supplementary signals describing the video type, or the video signal contains only basic image contents without the image type indicated, the signal must be recognized in an alternative way.
A European patent application EP10175505 presents a computer-implemented method for determining a video frame type, comprising the steps of receiving a video frame, analyzing the video frame in at least one detector module using at least one algorithm configured to output a type coefficient (p) indicative of a probability that the frame is of a 2D or 3D type, wherein each of the detector modules utilizes at least one algorithm different than the algorithms utilized by the other detector modules, and generating a predicted frame type indicator based on the type coefficients.
The aim of the present invention is to provide an alternative method for determining the video frame type by analyzing video signals having no or unknown indication of video frame type.
There is presented a computer-implemented method for determining whether a video frame is of a 3D TB-type (Top-Bottom) or a 3D LR-type (Left-Right) frame, comprising the steps of: receiving a video frame; extracting at least three portions of the frame, each portion belonging to a distinct quarter of the frame and being positioned at the same fragment of the quarter; calculating color histograms for each portion; comparing the color histograms of at least two different pairs of portions; generating a frame type indicator based on the result of comparison of the color histograms.
Preferably, the method further comprises the step of generating a compacted frame by discarding the non-active regions of the received video frame and providing the compacted frame for analyzing.
Preferably, the method further comprises the step of generating a compacted frame by scaling-down the video frame and providing the compacted frame for analyzing.
Preferably, the method further comprises the step of generating a compacted frame by discarding color information of the video frame and providing the compacted frame for analyzing.
Preferably, the portion of the quarter of the frame is smaller than the quarter of the frame.
Preferably, the at least three portions of the frame comprise a portion of the top-left quarter of the frame, a portion of the top-right quarter of the frame and a portion of the bottom-left quarter of the frame.
Preferably, the color histogram of the portion of the top-left quarter of the frame is compared with the color histogram of the portion of the top-right quarter of the frame; and wherein the color histogram of the portion of the top-left quarter of the frame is compared with the color histogram of the portion of the bottom-left quarter of the frame.
Preferably, the at least three portions of the frame comprise a portion of the top-left quarter of the frame, a portion of the top-right quarter of the frame and a portion of the bottom-left quarter of the frame and a portion of the bottom-right quarter of the frame.
Preferably, if TL&TR<TL&BL and BL&BR<TR&BR, the frame is determined as an LR type frame and/or if TL&TR>TL&BL and BL&BR>TR&BR, the frame is determined as a TB type frame. In such case, TL is the color histogram of the top left portion, TR is the color histogram of the top right portion, BL is the color histogram of the bottom left portion, BR is the color histogram of the bottom right portion, wherein the & operator relates to a formula:
wherein:
Preferably, the video frame is received upon a change of an input video signal.
Preferably, the video frame is received with a predetermined frequency.
There is also disclosed a computer program comprising program code means for performing all the steps of the computer-implemented method as described above when said program is run on a computer, as well as a computer readable medium storing computer-executable instructions performing all the steps of the computer-implemented method as described above.
There is also disclosed a system for determining a video frame type, the system comprising: at least three extractors, each configured to extract a portion of the frame, each portion belonging to a distinct quarter of the frame and being positioned at the same fragment of the quarter; color histogram generators, each configured to calculate a color histogram for each extracted portion; an LR-type frame detector and a TB-type frame detector each configured to compare the color histograms of at least two different pairs of portions and to generate a frame type indicator based on the result of comparison of the color histograms.
The present invention will be shown by means of an exemplary embodiment on a drawing, in which:
Therefore, for any 2D or 3D video frame, the most probable non-active regions 110 may form bars at the top, bottom, left, right, horizontal centre and vertical centre of the frame, as shown in
The extracted frame portions 121, 122, 123 may be rectangular regions adjacent to one or more edges of the quarter of the frame, or may be non-adjacent to the frame edges. The width may be equal to the width of the quarter of the frame or shorter. The height may be equal to the height of the quarter or shorter. In one embodiment, the portion may have a form of a strip adjacent to the top and left edges of the quarter, a length equal to half of the length of the quarter and a height equal to ⅛th of the height of the quarter, as shown in
The extracted frame portions 121-123 are input to color histogram generators 221-223, which generate histograms of colors for each portion 121-123, as shown in the examples of
Next, the color histograms generated by modules 221-223 are compared in detectors 231, 232, which compare the histograms using e.g. the least squares method. The LR frame type detector 231 is configured to compare the histogram of the top-left portion 121 and bottom-left portion 122. In case the comparison result is lower than a threshold (as may be deducted from
The extracted frame portions 121-124 are input to color histogram generators 321-324, which generate histograms of colors for each portion 121-124, as seen on
Next, the color histograms generated by modules 321-324 are multiplied in a histogram pair values comparator 330 which calculates the sum of squares of differences in the number of pixels related to particular histogram coefficient, using the formula:
wherein:
Next, the histogram pair values are compared.
In the LR type detector 331, it is checked whether the pair values meet the criteria: TL&TR<TL&BL and BL&BR<TR&BR. If so (as may be deducted from
In the TB type detector 332, it is checked whether the pair values meet the criteria: TL&TR>TL&BL and BL&BR>TR&BR. If so (as may be deducted from
Otherwise, the frame type is determined to be undefined or mono frame or undefined type.
In case the system according to the invention is embedded in a video display unit, the determined frame type can be used to select the method of processing the signal to be displayed. In case the system according to the invention is embedded in a video decoder, such as a television set-top box, the determined frame type can be used to select the method of processing the signal to be passed to a display unit, for example converting a 2D signal to a 3D format in case the display unit is set to receive 3D video signals.
It can be easily recognized, by one skilled in the art, that the aforementioned system and method for determining video frame type may be performed and/or controlled by one or more computer programs. Such computer programs are typically executed by utilizing the computing resources of a processing unit which can be embedded within various video signal receivers, such as personal computers, personal digital assistants, cellular telephones, receivers and decoders of digital television, video display units or the like. The computer programs can be stored in a non-volatile memory, for example a flash memory or in a volatile memory, for example RAM and are executed by the processing unit. These memories are exemplary recording media for storing computer programs comprising computer-executable instructions performing all the steps of the computer-implemented method according the technical concept presented herein.
While the invention presented herein has been depicted, described, and has been defined with reference to particular preferred embodiments, such references and examples of implementation in the foregoing specification do not imply any limitation on the invention. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader scope of the technical concept. The presented preferred embodiments are exemplary only, and are not exhaustive of the scope of the technical concept presented herein. Accordingly, the scope of protection is not limited to the preferred embodiments described in the specification, but is only limited by the claims that follow.
Number | Date | Country | Kind |
---|---|---|---|
14175233 | Jul 2014 | EP | regional |
Number | Name | Date | Kind |
---|---|---|---|
20090300698 | Quigley | Dec 2009 | A1 |
20100321390 | Kim | Dec 2010 | A1 |
20130044192 | Mukherjee | Feb 2013 | A1 |
20130083838 | Touze | Apr 2013 | A1 |
Number | Date | Country | |
---|---|---|---|
20160007003 A1 | Jan 2016 | US |