The instant application claims priority to Italian Patent Application No. TO2011A000414, filed May 11, 2011, which application is incorporated herein by reference in its entirety.
An embodiment of the disclosure relates to processing of video signals. Certain embodiments may relate to techniques for merging a third dimension in 2D bi-dimensional or two-dimensional) video signals.
Multi Video Coding or MVC schemes may be further improved under at least two aspects:
An embodiment provides such an improvement.
An embodiment refers to a method and corresponding apparatus (encoder and decoder) and to a computer program product, loadable in the memory of at least one computer and including software code portions capable of implementing the steps of the method when the product is run on at least one computer, as well as to a video signal encoded according to an embodiment.
An embodiment completely merges the bitstream generated by encoding the second view (e.g., the right view) into the bistream generated by encoding the first view (e.g., the left view), without adding any supplementary bit to the bitstream of the first view. This implies producing as a result what essentially looks like a “mono-view” video coded bitstream, even if it actually embeds a second view into it.
In an embodiment, the bitstream related to the second view may be generated either from the second stereoscopic view as such, or from said view presented in the form of a “depth map sequence”, namely a sequence of gray scale images, with values between black and white, which represent the distance between the objects shown within the frames and the camera that acquired the scene. In an embodiment, the second view can be either encoded by prediction from the first one (inter-view prediction MVC-like) or encoded independently (simulcast).
One or more embodiments will now be described, by way of example only, with reference to the annexed figures, wherein which:
Illustrated in the following description are various specific details aimed at an in-depth understanding of one or more embodiments. The one or more embodiments may be obtained without one or more specific details, or through other methods, components, materials, etc. In other cases, known structures, materials or operations are not shown or described in detail to avoid obscuring the various aspects of the embodiments. Reference to “an embodiment” in this description indicates that a particular configuration, structure or characteristic described regarding the embodiment is included in at least one embodiment. Hence, expressions such as “in an embodiment”, possibly present in various parts of this description do not necessarily refer to the same embodiment. Furthermore, particular configurations, structures or characteristics may be combined in any suitable manner in one or more embodiments. References herein are used for facilitating the reader and thus they do not necessarily define the scope of protection or the range of the embodiments.
Parts, elements or functions identical or equivalent to parts, elements or functions described in connection with any of
Also, it will be appreciated that throughout this disclosure the roles played by the two sequences in a stereo video signal (i.e., right and left) can be exchanged: for instance, while the exemplary one or more embodiments described in the following refer to the “right” sequence being merged into the “left” sequence, various embodiments may provide for the “left” sequence being merged into the “right” sequence.
A simple way of representing three dimensions with a bi-dimensional image is a stereo method. A stereo image is configured with a right and left images, which may be inherently disadvantageous in view of the massive volume of data involved. In simple words, the amount of data that has to be stored and transmitted for stereo-view video applications may be twice as much if compared to conventional “mono” view video applications if a mono-view video coding method is applied directly.
It has been noted that, although not designed for stereo-view video coding, video coding tools such as, e.g., H.264 coding tools can be configured to take advantage of the correlations between the pair of views of a stereo-view video, and provide reliable and efficient compression performance as well as stereo/mono-view scalability.
The block diagrams of
In an embodiment, the encoder of
In an embodiment, the decoder of
The arrangements exemplified in
Compression for stereo images can be achieved by taking advantage of redundancies in the source data, e.g., spatial and temporal redundancies for monocular images and video. The simplest solution for compressing the two channels is by using independent coding for each image/video with existing compression standards (H.264 Multicast Coding).
A stereoscopic video sequence contains a large amount of inter-view statistical dependencies, since the two cameras capture the same scene from two different viewpoints. Therefore, combined temporal and inter-view prediction may be a key for efficient stereoscopic coding. A frame from a given camera can be predicted not only from temporally related frames from the same camera, but also from the frames of any neighboring cameras. These interdependencies can be used for efficient prediction (Multi Video Coding or MVC).
An exemplary coding architecture for stereoscopic video may be as represented in
Specifically,
By way of further reference,
The same reference numerals appearing in
It will be appreciated that processing arrangements dual to the encoding layouts of
The diagram of
According to the variable block-size, it may be assumed that the message length is either less than 1/128 of the frame size if the block size in the encoder is set as small as 4×4, or less than 1/2048 if the block size is set as large as 16×16. As represented in
are encoded as two-number entities wherein in each block the non-zero level is preceded by a “run” value, indicating the number of zeroes before level and terminated by an End Of Block (EOB) flag, namely:
wherein [03] is representative of the last (non-zero) level.
In the exemplary case of
As schematically represented in
The thus modified sequence (i.e., with the last level modified from “3” to “2”, i.e., with [02] substituted for [03]) is encoded as usual using entropy encoding (VLC) in the module/step 120.
This procedure is repeated for each block until all the bits of the message are inserted into the left coding, so that the data for the right video signal is hidden in the “noise” of the image, producing an encoded signal, which may apparently look like a “mono-view” signal but in fact carries information of a “stereo-view” signal. In an embodiment, the bistream generated by encoding the right sequence can be considered as the message to be merged into the bitstream of the first view either in its entirety or only partly. In the latter case, the parts related to the information related to the structure of the bitstream itself (e.g., headers, sequence parameter set fields, picture parameter set fields, etc. . . . , i.e., those parts which do not contain data directly related to the images) are the same as those conveyed by the bitstream for the other (i.e., left) view; these parts can thus be derived from the left view bitstream when reconstructing the right view bitstream as described in the following.
Again, parts, elements or functions identical or equivalent to parts, elements or functions already described in connection with any of the previous figures are indicated in
By direct comparison with
In fact, generation of the message signal for the right video signal to be merged (e.g., at the level of the run-length coder 105 for the left sequence) into the left video sequence may dispense with separate processing modules/steps downstream of entropy (VLC) encoding 120 in the right-signal encoder, namely the modules/steps 122 and 126, and with the quantization step control module/step 124 as well.
Once more, parts, elements or functions identical or equivalent to parts, elements or functions already described in connection with any of the previous Figures (e.g.,
In the exemplary decoder arrangement of
In the exemplary decoder arrangement of
In the module/step 304, the least significant bit of each last level of the sequence is identified and collected to reconstruct the message representative of the right video signal as better detailed in the following.
Decoding of the left video signal proceeds otherwise as usual. The length of the message representative for the right video image will be expectedly lower than the number of transform blocks of the left (principal) view, otherwise at the decoder side an error while reconstructing the message might occur.
In that case, in certain embodiments, in order to preserve backward compatibility, the merging phase may operate at the frame level, that is by merging a right frame in the correspondent left one. Then, knowing the dimension of the image, the decoder may complete the extraction when the left frame is completely built.
The case that the length of the message representative for the right video image is higher than the number of transform blocks of the left (principal) view is highly unlikely.
In certain embodiments, one may set Constant Bit Rate (CBR) in the right view in order to be sure that the length is adequately small.
Extraction of the data for the right video signal merged into the left video signal takes place on the signal extracted at the module/step 304.
There, the least significant bit of each last level of the sequence is identified and collected to reconstruct the message.
In certain embodiments, the whole bitstream corresponding to the right encoded view might be obtained either by just reconstructing the message from the bitstream for the left view as depicted in
Variable Length Decoding (VLD) may be performed on the message reconstructed from the left view and decoding continues as usual.
It will be appreciated that certain embodiments may make it possible to achieve the following advantages:
Without prejudice to the underlying principles of the disclosure, the details and embodiments may vary, even significantly, with respect to what has been described herein by way of non-limiting example only, without departing from the scope of the disclosure. Specifically, it will be appreciated that while provided in connection with the exemplary case of H.264 coding, the present disclosure is not limited to H.264 coding, but is generally applicable to any digital video signal including a first video sequence and a second video sequence jointly forming a stereo-view digital video signal, and wherein the first video sequence is subjected to discrete cosine transform, quantization and run-length coding to produce a sequence of blocks of non-zero digital levels representative of the first video sequence, while the second video sequence is subjected to discrete cosine transform, quantization, run-length coding and variable length coding to produce digital messages representative of the second video sequence.
Application of the disclosure can be detected also at the level of the resulting encoded digital video signal by comparing such a signal, which is representative of a first video sequence and a second video sequence jointly forming a stereo-view digital video signal, with a test signal produced based on the instant disclosure starting form the same video sequences. The encoded signal will include a sequence of blocks of non-zero digital levels representative of said first video sequence as subjected to discrete cosine transform, quantization and run-length coding, wherein the Least Significant Bits of one of the digital levels in the blocks representative of the first video sequence have substituted therefor the bits of digital messages representative of the second video sequence as subjected to discrete cosine transform, quantization, run-length coding and variable length coding, whereby the bits of the digital messages are merged into the sequence of blocks of digital levels.
An embodiment of the above-described encoders may be part of a system that includes an integrated circuit (e.g., a controller such as a microprocessor or microcontroller) coupled the encoder, where the encoder and integrated circuit may be disposed on a same or different dies.
Similarly, an embodiment of the above-described decoders may be part of a system that includes an integrated circuit (e.g., a controller such as a microprocessor or microcontroller) coupled the decoder, where the decoder and integrated circuit may be disposed on a same or different dies.
In closing, it is pointed out that an embodiment of a decoder such as described above needs only the information in a data stream encoded according to an embodiment such as described above to recover the first and second (e.g., left and right) image sequences. Although a conventional decoder may be able to recover the first image sequence from the data stream, the conventional decoder would be unable to recover the second image sequence from the data stream because it would not “know” what portion of the data stream represents the second image sequence; therefore, to a conventional decoder, the second image stream may appear as noise.
From the foregoing it will be appreciated that, although specific embodiments have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the disclosure. Furthermore, where an alternative is disclosed for a particular embodiment, this alternative may also apply to other embodiments even if not specifically stated.
Number | Date | Country | Kind |
---|---|---|---|
TO2011A000414 | May 2011 | IT | national |