Patent Application
20060045187
The present invention relates to a video communication system, which is able to receive a digital video signal, according to the preamble of claim 1. The invention further relates to a method of processing a digital video signal according to the preamble of claim 8.
Such a system may be used, for example, for 3D video applications within MPEG standards.
A video communication system typically comprises a transmitter with an encoder and a receiver with a decoder. Such a system receives an input digital video signal, encodes said signal via the encoder, transmits the encoded signal to the receiver, then decodes the transmitted signal via the decoder resulting in an output digital video signal, which is the reconstructed signal of the input digital video signal. The receiver then displays said output digital video signal. A digital video signal comprises some sets of objects, which are characterized by components such as shapes, textures, motion information, disparity map (in the case of 3D video signal), etc.
When an object is encoded, the components of said object are encoded. The encoding and decoding processes of each video component can more or less depend on each other. Coding performance relies on this inter-dependence.
Let us give an example with 3D video objects encoded by using the MPEG4 standard. In this document referred to under the MPEG-4 document number w3056 at ISO and entitled “Information Technology—Coding of audio-visual objects—Part 2: Visual, ISO/IEC JTC 1/SC 29/WG 11, Maui, December 1999”, when one encodes a stereo video sequence comprising two views, one view—say the left one—and the corresponding disparity maps (or depth maps) are encoded. The right view is reconstructed by projecting the left view using the disparity map. Parts of the right view that were not in the left one (occlusion parts) cannot be reconstructed and holes remain. In order to reconstruct the right view properly, one encodes the occlusion parts retrieved from the original right view as an MPEG-4 video object, whose shape corresponds to the holes. If the left view is encoded at a bit rate of 3.5 Mbit/s, the enhancement layer defined to handle occlusion parts should not be larger than 10% of this bit rate. Here it takes 340 kbit/s. Occlusion objects are defined by shape and texture. Using the standard way of coding objects, their bit rate costs are:
Accordingly, it is an object of the invention to provide a video communication system as defined in the preamble of claim 1 and a method as defined in the preamble of claim 8, which lower the bit rate needed to encode objects.
To this end, according to a first object of the invention, there is provided a video communication system as claimed in claim 1.
In addition, according to a second object of the invention, there is provided a method as claimed in claim 8.
As we will see in detail below, by providing the possibility of not encoding a first component and therefore not transmitting it while encoding the other components as if it was the case, the encoding efficiency will be improved, because fewer bits will be necessary to encode the whole object.
Additional objects, features and advantages of the invention will become apparent upon reading the following detailed description and upon reference to the accompanying drawings in which:
In the following description, functions or constructions well known to the person skilled in the art are not described in detail because they would obscure the invention in unnecessary detail.
The present invention relates to a video communication system for processing a digital video signal.
Such a system, depicted in
In order to transmit efficiently some video signals through the transmission medium CH, in which the transmitted bits of the video signals are known as bit stream BIT_STR, said encoder ENC applies an encoding on a video signal, then the encoded video signal is sent to a decoder DEC, which decodes said signal. Finally, the receiver RECEIV displays said video signal.
A video signal comprises some sets of objects OBJ with different components COMP such as shape, texture, motion vectors, disparity map, colors, etc.
When the encoder ENC encodes an object OBJ, it encodes effectively all the components COMP of said object OBJ. Much encoding of components is dependent from other components. For example, in the INTER mode scheme, well known to the person skilled in the art, texture information can be used only if we have motion information. When it comes to video object coding, in MPEG-4 using the block-based principle, texture block positions are determined by shape, and knowing this shape allows improvement of coding efficiency by using spatial redundancy between co-located texture blocks.
The encoder ENC comprises availability information FLAG for determining if a component COMP of an object is to be encoded, or not, encoding means for encoding said component COMP if said availability information FLAG is enabled, and transmission means for transmitting said component COMP of said object OBJ if the availability information FLAG is enabled.
The decoder DEC comprises decoding means for decoding a component COMP if the availability information FLAG is enabled and retrieving means for retrieving said component COMP if the availability information FLAG is disabled.
The encoding of an object OBJ is done by the encoder ENC as follows, illustrated in
For each component COMP of the object OBJ, the encoder ENC decides if said component COMP will be included in the bit stream BIT_SIR or not. Decision depends on the type of video application: for example, in a stereo system where one encodes the occlusion part, one knows that there is no need to encode and transmit the shape of these occlusions.
Availability information FLAG is assigned to each component COMP of the object OBJ. If a component COMP is to be included in the bit stream, i.e. encoded and transmitted, the availability information FLAG is enabled. Preferably, this availability information FLAG is placed at video object level (VO in MPEG-4).
In a first non-limitative embodiment, this availability information FLAG has two values: 1, when enabled, and 0 when disabled.
In a second non-limitative embodiment, this availability information FLAG has an extended syntax: 1, when enabled, and 0 plus a description codeword when disabled. The description codeword tells how to retrieve the missing component COMP (wait for extra input, like a pre-computed shape, wait for extra input and nth component information, etc. . . . ). Afterwards, the decoder DEC will use this description codeword to retrieve a component COMP. It is supposed that at the decoder side, the way to retrieve a missing component COMP is well known and retrieval algorithms are available.
Subsequently, the encoder ENC encodes all the components COMP (step ENC_COMP in
Note that the availability information FLAG and potentially description codeword (depending on the kind of availability information used in the application) are defined for each object components COMP. The encoded video object OBJ then corresponds to all these availability information FLAG, along with the included encoded components COMP. Hence, the bit stream BIT_STR that is transmitted to the decoder comprises the availability information FLAG of the missing components, and for every other component, their corresponding availability information FLAG and the encoded component COMP itself.
Note that the following hypothesis has been made. We suppose that we have an encoding algorithm where video objects OBJ fall into several components and that they correspond to different, separable parts of the bit stream BIT_STR It means that it is possible to replace one component without preventing the decoding process of another component. For example, in MPEG4, if we encode a video sequence with texture and depth information, if we replace or modify the encoded texture information, it is still possible to decode depth. Furthermore, if we change the motion vector components, it is possible to decode the texture—even though we have a sequence that is visually different from the original, but the encoded texture information is still correct.
Said encoded bit stream BIT_STR is then sent to the decoder DEC of the receiver RECEIV, via the transmission medium CH.
The decoding of the encoded bit-stream BIT_STR is performed by the decoder DEC as follows and as illustrated in
When receiving the encoded bit-stream BIT_STR, the decoder DEC first checks the availability information FLAG in said bit-stream BIT_STR. If availability information FLAG is set to 1, it waits for the corresponding components COMP in the bit-stream BIT_STR, which follow their assigned availability information FLAG. Then, it decodes them. It first decodes the components COMP that do not need any other encoded components or missing components COMP_M (step DEC_COMP in
Once all of the encoded components COMP are decoded, the decoder DEC retrieves the last missing components COMP_M (step RETR_COMP_M2 on the
The bit stream BIT_STR is as if none of the components has been removed. The decoding process, well known to the person skilled in the art, is possible, as if every component has been regularly encoded and transmitted.
Thus, one advantage of the present invention is to simply decrease the bit rate used to encode an object by only using availability information FLAG, and to simply informing the decoder that it needs to retrieve some components itself.
It is to be understood that the present invention is not limited to the aforementioned embodiments and variations and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims. In this respect, the following closing remarks are made.
It is to be understood that the present invention is not limited to the aforementioned video application. It can be used within any application using a system for processing a signal that is decomposed into several components, the encoding of which results in separated bit-stream parts. In particular, the invention applies to video compression algorithms of the other MPEG standards family (MPEG-1, MPEG-2) and to the ITU H26X family (H261, H263 and extensions, H261 being the latest today, reference number Q15-K-59).
It is to be understood that the method according to the present invention is not limited to the aforementioned implementation.
There are numerous ways of implementing functions of the method according to the invention by means of items of hardware or software, or both, provided that a single item of hardware or software can carry out several functions. It does not exclude that an assembly of items of hardware or software or both carries out a function, thus forming a single function without modifying the method of processing the video signal in accordance with the invention.
Said hardware or software items can be implemented in several manners, such as by means of wired electronic circuits or by means of an integrated circuit that is suitably programmed. The integrated circuit can be accommodated in a computer or in a video system communication. In the second case, the video system communication comprises encoding means for encoding a component of an object if the availability information is enabled, decoding means for decoding said component of said object if the availability information is enabled, and retrieving means for retrieving said component of said object if the availability information is disabled, as described previously, said means being hardware or software items as stated above.
The integrated circuit comprises a set of instructions. Thus, said set of instructions comprised, for example, in a computer programming memory or in a video communication system may cause the computer or the video communication system to carry out the different steps of the encoding method.
The set of instructions may be loaded into the programming memory by reading a data carrier such as, for example, a disc. A service provider can also make the set of instructions available via a communication network such as, for example, the Internet.
Any reference sign in the following claims should not be construed as limiting the claim. It will be obvious that the use of the verb “to comprise” and its conjugations does not exclude the presence of any other steps or elements besides those defined in any claim. The article “a” or “an” preceding an element or step does not exclude the presence of a plurality of such elements or steps.
| Number | Date | Country | Kind |
|---|---|---|---|
| 02292353.6 | Sep 2002 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB03/04083 | 9/17/2003 | WO | 3/22/2005 |
