Patent Application
20050100091
1. Field of the Invention
The present invention relates to a coded signal separating apparatus, coded signal merging apparatus, coded signal separating and merging system, and methods thereof capable of performing scalable transmission of images.
2. Description of the Related Art
In the art of digitalizing a moving picture, there is a standard of coding method for digital video and related audio, ISO/IEC 13818 (commonly called “MPEG-2” (Moving Picture Expert Group Phase 2)) as a method of compressing and coding a considerable amount of generated information. A bit stream thus generated in compliant with the MPEG-2 standard (hereinafter, referred to as “MPEG-2 bit stream”) is used in a wide range of fields such as communication, television broadcasting, and so on.
The MPEG-2 bit stream has a hierarchical structure and is constituted of respective layers in order from a sequence layer at the top to a GOP (Group of Pictures) layer, a picture layer, a slice layer, a macroblock layer and a block layer.
In MPEG-2, respective images of a moving picture constituted of a series of images are once stored in frame memories, differences between frames are taken out to eliminate redundancy in a time axis direction, and further, plural pixels constituting each frame are subjected to an orthogonal conversion process such as discrete cosine transformation (hereinafter, abbreviated to “DCT”) to eliminate redundancy in a spatial axis direction, thereby realizing efficient compression and coding of a moving picture.
The coded signal is sent to a decoder to be decoded and reproduced. In the decoder, an image is reproduced and stored in a first frame memory, and a subsequent image is predicted based on differential information and stored in a second frame memory, and from the two frames, an image to be inserted therebetween is further predicted to thereby construct a series of images to reproduce a moving picture. Such a method is called a bidirectional prediction.
In order to realize this bidirectional prediction, three types of pictures, I picture, P picture and B picture, are defined in MPEG-2. The I picture stands for Intra-coded picture, which is an image coded independently from other pictures as a still image, the P picture stands for Predictive coded picture in a forward direction, which is an image predictively coded based on an I picture or P picture positioned in the past temporally. The B picture stands for Bidirectional predictive coded picture, which is an image predictively coded based on a picture in a forward direction, backward direction, or bidirectional direction using an I picture or P picture located temporally before or after the B picture. Specifically, after the I picture and the P picture are subjected to a coding process in advance, the B picture to be inserted therebetween is coded.
The MPEG-2 bit stream coded in a coder is transmitted at a predetermined transmission rate to a transmitting path, and inputted to a decoder on the transmitting path to be decoded and reproduced. However, the amount of information generated by coding a moving picture is not constant. Particularly, when a scene is changed, the amount of information rapidly increases. In order to transmit such a non-constant coded signal to a transmitting path having a fixed rate, it is necessary to perform a rate control of coded data in advance so that the amount of information of transmission buffer level or more will not be generated.
In MPEG-2, a rate controlling method is described in ISO/IEC JTC1/SC29/WG11I/N0400 Test Mode 5 (April, 1993) (hereinafter, abbreviated to “TM5”).
In the rate control in the TM5 of MPEG-2, in step 1 first, bits are allocated based on an assigned code amount R for uncoded pictures in a GOP for each picture type. In step 2, a quantization scale used when performing coding processing in macroblock unit is calculated from the occupancy amount of a virtual buffer, which is calculated based on the bit allocation.
Further, there exist a large number of decoders having a compression format other than the MPEG-2 as well as decoders connected to a transmitting path having a different transfer rate, so that a moving picture compressed coded signal converter for converting an MPEG-2 bit stream into a different compression format or a different transfer rate is needed. An apparatus for realizing this is a so-called transcoder. The compressed coded signal of a moving picture transmitted from a coder is converted into an appropriate signal in the transcoder, and the signal is supplied to the respective decoders.
By the VLD 51 and the inverse quantizer 53, the first MPEG-2 bit stream b1 is decoded in a macroblock unit up to a DCT coefficient region, and an obtained DCT coefficient signal is coded by the quantizer 55 and the VLC 57 to generate the second MPEG-2 bit stream b2 having a smaller code amount than the first MPEG-2 bit stream.
In a quantization process in the quantizer 55, coefficients obtained by the DCT transformation is divided by a predetermined quantizing step. Thus, the image signal is compressed. This quantization step is obtained by multiplying plural quantization matrix values included in a predetermined quantization table by a quantization scale.
In the transcoder 50, most of coded information of the sequence layer, GOP layer, picture layer, slice layer and macroblock layer in the first MPEG-2 bit stream b1 is reused. Basically, there is performed only processes of converting the DCT coefficient of the block layer and converting the code of the macroblock layer needed to be modified along with the conversion of the block layer.
In the transcoder 50 configured as such, the rate control unit 59 performs the rate control described in the TM5 of MPEG-2.
A flowchart of a rate control process in the conventional transcoder 50 is shown in
In subsequent step A2, indicators Xi, Xp and Xb representing complexity of I, P and B pictures are calculated by the following equation (a1), equation (a2), and equation (a3).
Xi=Si×Qi equation (a1)
Xp=Sp×Qp equation (a2)
Xb=Sb×Qb equation (a3)
Here, Si, Sp and Sb are generated code amounts of the I, P and B pictures respectively, and Qi, Qp and Qb are average quantization parameters which are average values of quantization scale codes of all macroblocks in the I, P and B pictures respectively. However, the average quantization parameters are normalized in a range from 1 to 31.
The indicators Xi, Xp and Xb of complexity of images become large in an image for which a large coded information amount is generated, namely, an image having a low compression rate, and inversely become small in an image having a high compression rate.
Further, initial values of the parameters Xi, Xp, and Xb representing the complexity of images of the I, P and B pictures are provided by the following equation (a4), equation (a5), and equation (a6), respectively.
Xi=160×target_Bitrate/115 equation (a4)
Xp=60×target_Bitrate/115 equation (a5)
Xp=42×target_Bitrate/115 equation (a6)
Here, the “target_Bitrate” is a target bit rate of the transcoder 50.
In subsequent step A3, assigned code amounts Ti, Tp and Tb for the I, P and B pictures in a GOP are calculated by the following equation (a7), equation (a8), and equation (a9) respectively. However, Np and Nb represent the number of uncoded P pictures and the number of uncoded B pictures in the GOP respectively.
Here, Kp and Kb represent percentages of the quantization scale codes of the P and B pictures on the basis of the quantization scale code of the I picture, and it is presumed that the overall picture quality is constantly optimized when Kp=1.0 and Kb=1.4.
In subsequent step A4, whether the variable n is 1 or not is judged. Specifically, whether a picture that is the coding target is a first picture pic(1) or not is judged. If it is the first picture, the flow goes to step A5. If it is not the first picture, the flow goes to step A6. In step A5, an assigned code amount R for an uncoded picture in the GOP at the time of coding the first picture pic(1) in the GOP is obtained by the following equation (a10).
R=target_Bitrate×N/picture_rate+R equation (a10)
Here, N is a value representing the total number of pictures in the GOP, the “picture_rate” is a value representing the resolution per time of an input image and representing the number of images decoded and displayed in one second.
In step A6, the assigned code amount R for an uncoded picture in the GOP is updated by one of the following equation (a11), equation (a12) and equation (a13) based on the generated code amount Si, Sp or Sb of the I, P and B pictures at the time when the (n−1)th picture pic(n−1) is coded.
R=R−Si equation (a11)
R=R−Sp equation (a12)
R=R−Sb equation (a13)
Steps A5 and A6 both go to step A7, and a variable j is set to 1. Here, the variable j represents numbers assigned to plural macroblocks in one picture, and hereinafter, a j-th macroblock will be represented by MB(j).
In subsequent step A8, occupancy amounts di(j), dp(j) and db(j) of virtual buffers at the time of coding the j-th macroblock MB(j) in the I, P and B pictures are calculated by the following equation (a14), equation (a15) and equation (a16) respectively.
Here, B(j−1) is a generated code amount of all the macroblocks up to (j−1)th macroblock MB(j−1).
Further, di(0), dp(0) and db(0) are initial values of virtual buffer occupancy amounts of the I, P and B pictures respectively, and provided by the following equation (a17), equation (a18) and equation (a19) respectively.
di(0)=10×r/31 equation (a17)
dp(0)=Kp×di(0) equation (a18)
db(0)=Kb×di(0) equation (a19)
Here, “r” is called a reaction parameter, which is shown by the following equation (a20), and controls the response speed of a feedback loop.
r=2×target_Bitrate/picture_rate equation (a20)
Further, virtual buffer occupancy amounts at the time of completing coding of the I, P and B pictures, namely, virtual buffer occupancy amounts di(NMB), dp(NMB) and db(NMB) at the time of coding an NMB-th macroblock MB(NMB) are used as initial values di(0), dp(0) and db(0) of virtual buffer occupancy amounts at the time of next coding for respective picture types.
In subsequent step A9, based on the above-described virtual buffer occupancy amount d(j), a quantization scale code Q(j) for a j-th macroblock MB(j) for each picture is obtained by the following equation (a21).
Q(j)=d(j)×31/r equation (a21)
In subsequent step A10, the quantization scale code Q(j) calculated in step A9 is used to quantize the j-th macroblock MB(j). In subsequent step A11, the variable j is incremented, and the flow goes to step A12 to judge whether or not the variable j exceeds the macroblock total number NMB. Here, NMB is the total number of macroblocks included in an n-th picture pic(n). When the variable j does not exceed the macroblock total number NMB, the flow returns to step A8. When the variable j exceeds the macroblock total number NMB, the flow goes to step A13.
Thus, the variable j is also used as a loop counter for repeating the coding process in steps A8 to A11. Accordingly, the coding process can be performed sequentially on all macroblocks from a first macroblock MB(1) to an NMB-th macroblock MB(NMB) in an n-th picture pic(n).
In step A13, the variable n is incremented, and the flow goes to step A14 to judge whether or not the variable n exceeds the total number NPIC of pictures which are the coding targets. Here, when the variable n does not exceed the total number NPIC of pictures, the flow returns to step A2. When the variable n exceeds the total number NPIC of pictures, the process is terminated.
Thus, the first transcoder 50 is not able to have information regarding image structures such as I and P picture cycles, so that a method of allocating bits, such as the rate control in TM5 shown in
Then, as an example of adopting a method of performing the rate control without presuming a GOP structure, there is a second conventional transcoder 60 shown in
The flow of a process in the transcoder 60 configured as such is shown in
Further, similarly, as another example of adopting a method of performing the rate control without presuming the GOP structure, a third example of a conventional transcoder is shown in
In the third conventional transcoder 80, a code amount is described as information in the bit stream b3 in advance, and the rate control is performed based on this information.
However, the transcoder targets at signals after a coding process, and thus it does not know original signals before the coding. Therefore, in a coding amount control, reduction of the code amount must be realized while restraining deterioration in the image quality by focusing attention not on the distortion of an image itself after a transcode process but on a distortion that is newly generated due to the re-quantization process and restraining this distortion.
Accordingly, the applicant of the present application proposed a method and apparatus of converting a moving picture compression coded signal and a medium recording a conversion program product, which realize calculation of an optimum quantization parameter based on a decoding quantization parameter and a quantization parameter calculated in a preceding stage by considering in advance a re-quantization rate distortion function depending on the decoding quantization parameter and a re-quantization parameter (for example, refer to Japanese Patent Application Laid-open No. Hei 2001-169283 and related corresponding European Patent Application EP 1067798).
In the above described method and apparatus, in a transcoder provided with an inverse quantizer configured to perform inverse quantization and a quantizer configured to perform re-quantization, there is provided a quantization parameter switching part configured to switch the quantization parameter by considering a rate distortion function based on an input quantization parameter, so that errors in the conversion from quantization coefficient area data to re-quantization coefficient area data can be suppressed as much as possible.
Thus, the transcoder is a processor which realizes a bit stream conversion into forms suitable in various use environments.
Furthermore, on a network in which various bands are mixed, a bit rate scaling method and a progressive coding method exist as a technology to realize a scalability for generating and providing image streams suitable in use environments.
For example, in a bit rate scaling transcoder placed in each router on a network, a stream conversion by bit rate reduction according to a required bit rate is realized. Thus, scalability of images corresponding to variances of a network is realized.
However, since the bit rate scaling transcoder itself performs the rate control and the stream conversion, highly technical processing is required in a router, thereby increasing a load on the router.
Further, a stream inputted in advance to a server is converted into a stream having a hierarchical structure, and the scalability is realized by hierarchy selection at a router. This renders the rate control by the transcoder unnecessary, and also eliminates a load such as signal processing at each router on a network. However, since there is a dependent relationship between the streams converted into the hierarchical structure, each hierarchy is given an order of priority, which poses a problem of requiring a control with consideration of the order of priority when selecting a hierarchy.
The present invention is made in view of solving such conventional problems, and an object thereof is to provide a coded signal separating apparatus, coded signal merging apparatus, coded signal separating and merging system, and methods thereof capable of generating multiplexed separated streams to perform scalable transmission of images.
A coded signal separating apparatus according to the present invention is characterized by including a separator input means for inputting a first coded signal in which a first moving picture constituted of a plurality of image information is coded; a separator separating means for separating the first coded signal inputted to the separator input means into a basic coded signal having a smaller code amount than the first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the first moving picture than the second moving picture reconstructed with the basic coded signal; a separator multiplexing means for optionally combining and multiplexing the basic coded signal with the plurality of extended coded signals, which are separated by the separator separating means, to generate a plurality of transmission coded signals; and a separator output means for outputting the plurality of transmission coded signals multiplexed by the separator multiplexing means.
Preferably, in the coded signal separating apparatus according to the present invention, the separator separating means includes: a first coefficient converting means for converting the first coded signal into first quantization coefficient values for reconstructing the first moving picture; a basic extended hierarchy separating means for separating the first quantization coefficient values converted by the first coefficient converting means into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture and extended hierarchy coefficient values of an extended hierarchy used when reconstructing the third moving picture; a basic coded signal generating means for generating the basic coded signal from the extended hierarchy coefficient values; and an extended coded signal generating means for generating the plurality of extended coded signals from the extended hierarchy coefficient values.
Incidentally, the basic extended hierarchy separating means may have operation to separate the basic hierarchy coefficient values and the extended hierarchy coefficient values according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the first quantization coefficient values.
Further, the separator separating means may include a basic quantization coefficient converting means for converting the basic hierarchy coefficient values separated by the basic extended hierarchy separating means into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with the re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values,
Further, the separator separating means may include a basic quantization coefficient converting means for converting the basic hierarchy coefficient values separated by the basic extended hierarchy separating means into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with the re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, and
Preferably, in the coded signal separating apparatus according to the present invention, the separator multiplexing means generates each of the basic coded signal and the plurality of extended coded signals as each of the transmission coded signals.
Preferably, in the coded signal separating apparatus according to the present invention, the separator multiplexing means multiplexes the basic coded signal to each of the plurality of extended coded signals to generate the plurality of transmission coded signals.
Preferably, in the coded signal separating apparatus according to the present invention, the separator multiplexing means multiplexes the plurality of extended coded signals to generate the plurality of transmission coded signals each having a different code amount.
Preferably, in the coded signal separating apparatus according to the present invention, the basic extended hierarchy separating means of the separator separating means separates the basic hierarchy coefficient values and the extended hierarchy coefficient values according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Preferably, in the coded signal separating apparatus according to the present invention, the extended coded signal generating means of the separator separating means codes quantization parameter reconstructing information, which is for reconstructing the input quantization parameter from the re-quantization parameter, within each of the plurality of extended coded signals.
Furthermore, the basic extended hierarchy separating means of the separator separating means may be configured to generate a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property, calculate the re-quantization parameter from the input quantization parameter according to the re-quantization parameter deriving constant, and separate the basic hierarchy coefficient values and the extended hierarchy coefficient values, and the extended coded signal generating means of the separator separating means may be configured to code information of the re-quantization parameter deriving constant as the quantization parameter reconstructing information within the plurality of extended coded signals.
Furthermore, the extended coded signal generating means may be configured to code differential values between macroblocks of the re-quantization parameter deriving constant as the quantization parameter reconstructing information within the plurality of extended coded signals.
Furthermore, the extended coded signal generating means may be configured to code the extended quantization coefficient sequence by a coding table in accordance with the re-quantization parameter deriving constant.
Preferably, the coded signal separating apparatus according to the present invention further includes a separating information input means for inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, and the extended quantization coefficient separating means of the separator separating means separates the extended hierarchy coefficient values separated by the basic extended hierarchy separating means according to the extended hierarchy separating pattern information inputted by the separating information input means into the plurality of extended quantization coefficient sequences.
Preferably, in the coded signal separating apparatus according to the present invention, the basic extended hierarchy separating means of the separator separating means separates the basic hierarchy coefficient values and the extended hierarchy coefficient values according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Furthermore, the prediction error coefficient sequence coding means of the basic coded signal generating means may be configured to code the quantization parameter reconstructing information, which is for reconstructing the input quantization parameter from the re-quantization parameter, with the prediction error coefficient sequence to the prediction error coded sequence.
Furthermore, the basic extended hierarchy separating means of the separator separating means may be configured to generate a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property, calculate the re-quantization parameter from the input quantization parameter according to the re-quantization parameter deriving constant, and separate the basic hierarchy coefficient values and the extended hierarchy coefficient values, and the prediction error coefficient sequence coding means of the basic coded signal generating means may be configured to code the information of the re-quantization parameter deriving constant as the quantization parameter reconstructing information with the prediction error coefficient sequence to the prediction error coded sequence.
Furthermore, the prediction error coefficient sequence coding means may be configured to code differential values between macroblocks of the re-quantization parameter deriving constant as the quantization parameter reconstructing information with the prediction error coefficient sequence to the prediction error coded sequence.
Furthermore, the prediction error coefficient sequence coding means may be configured to code the prediction error coefficient sequence by a coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, the extended coded signal generating means of the separator separating means may be configured to code the extended quantization coefficient sequence by the coding table in accordance with the re-quantization parameter deriving constant.
Preferably, the coded signal separating apparatus according to the present invention further includes a separating information input means for inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, and the extended quantization coefficient separating means of the separator separating means separates the extended hierarchy coefficient values separated by the basic extended hierarchy separating means according to the extended hierarchy separating pattern information inputted by the separating information input means into the plurality of extended quantization coefficient sequences.
Furthermore, the separator multiplexing means may be configured to generate the transmission coded signals so that a code amount ratio of the plurality of transmission coded signals each having a different code amount becomes power of 2.
A coded signal merging apparatus according to the present invention is characterized by including: a merging device input means for inputting a plurality of independent transmission coded signals in which a first moving picture constituted of a plurality of image information is coded; a merging device separating means for separating the plurality of transmission coded signals inputted to the merging device input means into a basic coded signal having a smaller code amount than a first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the first moving picture than the second moving picture reconstructed with the basic coded signal; a merging device merging means for merging the basic coded signal with the plurality of extended coded signals, which are separated by the merging device separating means, to generate a third coded signal for reconstructing the third moving picture; and a merging device output means for outputting the third coded signal merged by the merging device merging means.
Preferably, in the coded signal merging apparatus according to the present invention, the merging device merging means includes: a basic coded signal converting means for converting the basic coded signal into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture; an extended coded signal converting means for converting the plurality of extended coded signals into extended hierarchy coefficient values of an extended hierarchy used when reconstructing the third moving picture; a basic extended hierarchy merging means for merging the basic hierarchy coefficient values converted by the basic coded signal converting means with the extended hierarchy coefficient values converted by the extended coded signal converting means to generate third quantization coefficient values; and a third coefficient converting means for converting the third quantization coefficient values merged by the basic extended hierarchy merging means into the third coded signal.
Preferably, in the coded signal merging apparatus according to the present invention, the merging device separating means separates the plurality of transmission coded signals into the plurality of basic coded signals in which the same information is coded and the plurality of extended coded signals.
Preferably, the coded signal merging apparatus according to the present invention further includes a signal error notifying means for selecting a basic coded signal having less received signal errors from a plurality of basic coded signals separated by the merging device separating means and notifying the selected basic coded signal to the merging device merging means, and the merging device merging means selects the basic coded signal notified by the signal error notifying means and merges the selected basic coded signal and the plurality of extended coded signals.
Preferably, the coded signal merging apparatus according to the present invention further includes a merging control means for receiving error correcting status information from an error detector on a network and notifying the error correcting status information to the merging device merging means, and the merging device merging means merges the basic coded signal and the plurality of extended coded signals based on the error correcting status information notified by the merging control means.
Preferably, the coded signal merging apparatus according to the present invention further includes a coded signal decoding apparatus for inputting the third coded signal from the coded signal merging apparatus and decoding the third coded signal to reproduce the third moving picture.
A coded signal separating and merging system according to the present invention is characterized by including: a separator for separating a first coded signal in which a first moving picture constituted of a plurality of image information is coded into a basic coded signal having a smaller code amount than the first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the first moving picture than the second moving picture reconstructed with the basic coded signal, and reconstructing and converting the separated signals into a plurality of transmission coded signals to be transmitting to a network;
Preferably, in the coded signal separating and merging system according to the present invention, the separator separating means includes: a first coefficient converting means for converting the first coded signal into first quantization coefficient values for reconstructing the first moving picture; a basic extended hierarchy separating means for separating the first quantization coefficient values converted by the first coefficient converting means into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture and extended hierarchy coefficient values of an extended hierarchy used when reconstructing the third moving picture; a basic coded signal generating means for generating the basic coded signal from the extended hierarchy coefficient values; and an extended coded signal generating means for generating the plurality of extended coded signals from the extended hierarchy coefficient values, and
Preferably, in the coded signal separating and merging system according to the present invention, in the separator, the separator separating means further includes a basic quantization coefficient converting means for converting the basic hierarchy coefficient values separated by the basic extended hierarchy separating means into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with a re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, the basic coded signal generating means of the separator separating means includes a basic quantization coefficient sequence coding means for inputting the basic quantization coefficient values and coding the basic quantization coefficient values to a basic quantization coefficient coded sequence which is the basic coded signal, and, the extended coded signal generating means of the separator separating means inputs the prediction error coefficient values converted by the basic quantization coefficient converting means and the extended hierarchy coefficient values separated by the basic extended hierarchy separating means to generate the plurality of extended coded signals from the prediction error coefficient values and the extended hierarchy coefficient values,
Preferably, in the coded signal separating and merging system according to the present invention, in the separator, the separator separating means further includes a basic quantization coefficient converting means for converting the basic hierarchy coefficient values separated by the basic extended hierarchy separating means into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with a re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, the basic coded signal generating means of the separator separating means includes: a basic quantization coefficient sequence coding means for inputting the basic quantization coefficient values and coding the basic quantization coefficient values to a basic quantization coded sequence; a prediction error coefficient sequence coding means for inputting the prediction error coefficient values and coding the prediction error coefficient values to a prediction error coded sequence; and a basic coded signal merging means for merging the basic quantization coded sequence with the prediction error coded sequence to generate the basic coded signal,
Preferably, in the coded signal separating and merging system according to the present invention, the basic extended hierarchy separating means of the separator separating means separates the basic hierarchy coefficient values and the extended hierarchy coefficient values according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Preferably, in the coded signal separating and merging system according to the present invention, the basic extended hierarchy separating means of the separator separating means separates the basic hierarchy coefficient values and the extended hierarchy coefficient values according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Preferably, in the coded signal separating and merging system according to the present invention, the separator multiplexing means multiplexes the basic coded signal to each of the plurality of extended coded signals to generate the plurality of transmission coded signals, the merging device separating means separates the plurality of transmission coded signals into the plurality of basic coded signals and the plurality of extended coded signals, and the merging device merging means selects one basic coded signal from the plurality of basic coded signals separated by the merging device separating means and merges the selected basic coded signal and the plurality of extended coded signals.
Preferably, in the coded signal separating and merging system according to the present invention, the separator multiplexing means multiplexes the plurality of extended coded signals to generate the plurality of transmission coded signals each having a different code amount, and the merging device separating means separates the plurality of transmission coded signals into the basic coded signal and the plurality of extended coded signals.
Preferably, in the coded signal separating and merging system according to the present invention, the extended coded signal generating means of the separator separating means codes quantization parameter reconstructing information, which is for reconstructing the input quantization parameter from the re-quantization parameter, within each of the plurality of extended coded signals, the extended coded signal converting means of the converting device converting means decodes the quantization parameter reconstructing information from the extended coded signal, the basic quantization coefficient merging means of the merging device merging means merges the sequence of the basic hierarchy coefficient values according to the quantization parameter reconstructing information, and the third coefficient converting means of the converting device converting means codes an input quantization parameter, which is reconstructed according to the re-quantization parameter and the quantization parameter reconstructing information, within the third coded signal.
Preferably, the coded signal separating and merging system according to the present invention further includes a signal error notifying means for selecting a basic coded signal having less received signal errors from a plurality of basic coded signals separated by the merging device separating means and notifying the selected basic coded signal to the merging device merging means, and the merging device merging means selects the basic coded signal notified by the signal error notifying means and merges the selected basic coded signal and the plurality of extended coded signals.
Preferably, in the coded signal separating and merging system according to the present invention, the separator multiplexing means generates the transmission coded signals so that a code amount ratio of the plurality of transmission coded signals each having a different code amount becomes power of 2.
Preferably, the coded signal separating and merging system according to the present invention further includes a merging control means for receiving error correcting status information from an error detector on a network and notifying the error correcting status information to the merging device merging means, and the merging device merging means merges the basic coded signal and the plurality of extended coded signals based on the error correcting status information notified by the merging control means.
Preferably, the coded signal separating and merging system according to the present invention further includes a coded signal decoding apparatus for inputting the third coded signal from the coded signal merging device and decoding the third coded signal to reproduce the third moving picture.
A method of separating a coded signal and a program product for separating a coded signal according to the present invention are characterized by including: a separator input step of inputting a first coded signal in which a first moving picture constituted of a plurality of image information is coded; a separator separating step of separating the first coded signal inputted in the separator input step into a basic coded signal having a smaller code amount than the first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the first moving picture than the second moving picture reconstructed with the basic coded signal; and a separator multiplexing step of optionally combining and multiplexing the basic coded signal with the plurality of extended coded signals, which are separated in the separator separating step, to generate a plurality of transmission coded signals; and a separator output step of outputting the plurality of transmission coded signals multiplexed in the separator multiplexing step.
Preferably, in the method of separating a coded signal and the program product for separating a coded signal according to the present invention, the separator separating step includes: a first coefficient converting step of converting the first coded signal into first quantization coefficient values for reconstructing the first moving picture; a basic extended hierarchy separating step of separating the first quantization coefficient values converted in the first coefficient converting step into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture and extended hierarchy coefficient values of an extended hierarchy used when reconstructing the third moving picture; a basic coded signal generating step of generating the basic coded signal from the extended hierarchy coefficient values; and an extended coded signal generating step of generating the plurality of extended coded signals from the extended hierarchy coefficient values.
Preferably, in the method of separating a coded signal and the program product for separating a coded signal according to the present invention, in the basic extended hierarchy separating step in the separator separating step, the basic hierarchy coefficient values and the extended hierarchy coefficient values are separated according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Preferably, in the method of separating a coded signal and the program product for separating a coded signal according to the present invention, in the basic extended hierarchy separating step in the separator separating step, the basic hierarchy coefficient values and the extended hierarchy coefficient values are separated according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Furthermore, in the basic extended hierarchy separating step, the basic hierarchy coefficient values and the extended hierarchy coefficient values may be separated according to the value of the input quantization parameter for quantizing the coefficient values of the first moving picture to the first quantization coefficient values and the value of the re-quantization parameter for re-quantizing the first quantization coefficient values.
Furthermore, the separator separating step may include a basic quantization coefficient converting step of converting the basic hierarchy coefficient values separated in the basic extended hierarchy separating step into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with the re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, the basic coded signal generating step in the separator separating step may include a basic quantization coefficient sequence coding step of inputting the basic quantization coefficient values and coding the basic quantization coefficient values into a basic quantization coded sequence that is the basic coded signal, and in the extended coded signal generating step in the separator separating step, the prediction error coefficient values converted in the basic quantization coefficient converting step and the extended hierarchy coefficient values separated in the basic extended hierarchy separating step may be inputted, and the plurality of extended coded signals may be generated from the prediction error coefficient values and the extended hierarchy coefficient values.
Furthermore, the separator separating step may include a basic quantization coefficient converting step of converting the basic hierarchy coefficient values separated in the basic extended hierarchy separating step into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with the re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, and
Furthermore, in the separator multiplexing step, each of the basic coded signal and the plurality of extended coded signals may be generated as each of the transmission coded signals.
Furthermore, in the separator multiplexing step, the basic coded signal may be multiplexed to each of the plurality of extended coded signals to generate the plurality of transmission coded signals.
Furthermore, in the separator multiplexing step, the plurality of extended coded signals may be multiplexed to generate the plurality of transmission coded signals each having a different code amount.
Furthermore, in the separator multiplexing step, the transmission coded signals may be generated so that a code amount ratio of the plurality of transmission coded signals each having a different code amount becomes power of 2.
A method of merging a coded signal and a program product for merging a coded signal according to the present invention are characterized by including: a merging device input step of inputting a plurality of independent transmission coded signals in which a first moving picture constituted of a plurality of image information is coded; a merging device separating step of separating the plurality of transmission coded signals inputted in the merging device input step into a basic coded signal having a smaller code amount than a first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the first moving picture than the second moving picture reconstructed with the basic coded signal; a merging device merging step of merging the basic coded signal with the plurality of extended coded signals, which are separated in the merging device separating step, to generate a third coded signal for reconstructing the third moving picture; and a merging device output step of outputting the third coded signal merged in the merging device merging step.
Preferably, in the method of merging a coded signal according to the present invention, in the merging device separating step, the plurality of transmission coded signals are separated into the plurality of basic coded signals in which the same information is coded and the plurality of extended coded signals, and
Preferably, in the program product for merging a coded signal according to the present invention, the merging device merging step includes: a basic coded signal converting step of converting the basic coded signal into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture; an extended coded signal converting step of converting the plurality of extended coded signals into extended hierarchy coefficient values of an extended hierarchy to be used when reconstructing the third moving picture; a basic extended hierarchy merging step of merging the basic hierarchy coefficient values converted in the basic coded signal converting step with the extended hierarchy coefficient values converted in the extended coded signal converting step to generate third quantization coefficient values; and a third coefficient converting step of converting the third quantization coefficient values merged in the basic extended hierarchy merging step into the third coded signal.
Furthermore, in the merging device separating step, the plurality of transmission coded signals may be separated into the plurality of basic coded signals in which the same information is coded and the plurality of extended coded signals.
Preferably, the program product for merging a coded signal according to the present invention further includes a signal error notifying step of selecting a basic coded signal having less received signal errors from a plurality of basic coded signals separated in the merging device separating step and notifying the selected basic coded signal to the merging device merging step, and in the merging device merging step, the basic coded signal notified in the signal error notifying step is selected and merged with the plurality of extended coded signals.
Preferably, the method of merging a coded signal and the program product for merging a coded signal according to the present invention further includes a merging control step of receiving error correcting status information from an error detector on a network and notifying the error correcting status information to the merging device merging step, and in the merging device merging step, the basic coded signal is merged with the plurality of extended coded signals based on the error correcting status information notified in the merging control step.
A method of separating and merging a coded signal and a program product for separating and merging a coded signal according to the present invention is characterized by including: a separator controlling step of controlling a separator for separating a first coded signal in which a first moving picture constituted of a plurality of image information is coded into a basic coded signal having a smaller code amount than the first coded signal and configured to reconstruct a second moving picture, which is a pseudo moving picture of the first moving picture, and a plurality of extended coded signals which are used with the basic coded signal to reconstruct a third moving picture closer to the firs t moving picture than the second moving picture reconstructed with the basic coded signal, and reconstructing and converting the separated signals into a plurality of transmission coded signals to be transmitting to a network;
Preferably, in the method of separating and merging a coded signal according to the present invention, the separator separating step includes: a first coefficient converting step of converting the first coded signal into first quantization coefficient values for reconstructing the first moving picture; a basic extended hierarchy separating step of separating the first quantization coefficient values converted in the first coefficient converting step into basic hierarchy coefficient values of a basic hierarchy for reconstructing the second moving picture and extended hierarchy coefficient values of an extended hierarchy used when reconstructing the third moving picture; a basic coded signal generating step of generating the basic coded signal from the extended hierarchy coefficient values; and an extended coded signal generating step of generating the plurality of extended coded signals from the extended hierarchy coefficient values, and
Preferably, in the method of separating and merging a coded signal according to the present invention, in the separator controlling step, the separator separating step further includes a basic quantization coefficient converting step of converting the basic hierarchy coefficient values separated in the basic extended hierarchy separating step into basic quantization coefficient values, which are re-quantization output coefficients obtained by re-quantizing the basic hierarchy coefficient values with a re-quantization parameter, and prediction error coefficient values obtained from difference between values predicted by the basic quantization coefficient values and the re-quantization parameter and the basic hierarchy coefficient values, the basic coded signal generating step in the separator separating step includes a basic quantization coefficient sequence coding step of inputting the basic quantization coefficient values and coding the basic quantization coefficient values to a basic quantization coefficient coded sequence which is the basic coded signal, and in the extended coded signal generating step in the separator separating step, the prediction error coefficient values converted in the basic quantization coefficient converting step and the extended hierarchy coefficient values separated in the basic extended hierarchy separating step are inputted to generate the plurality of extended coded signals from the prediction error coefficient values and the extended hierarchy coefficient values, and
Preferably, in the method of separating and merging a coded signal according to the present invention, in the basic extended hierarchy separating step in the separator separating step, the basic hierarchy coefficient values and the extended hierarchy coefficient values are separated according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Preferably, in the method of separating and merging a coded signal according to the present invention, in the basic extended hierarchy separating step in the separator separating step, the basic hierarchy coefficient values and the extended hierarchy coefficient values are separated according to the value of an input quantization parameter for quantizing coefficient values of the first moving picture to the first quantization coefficient values and the value of a re-quantization parameter for re-quantizing the coefficient values of the first moving picture,
Furthermore, regarding the method of separating and merging a coded signal according to the present invention, each configuration of the above-described coded signal separating and merging system can be replaced with a method (step) to thereby obtain various embodiments. Similarly, regarding the program product for separating and merging a coded signal according to the present invention, each configuration of the above-described coded signal separating and merging system and method of separating and merging a coded signal can be replaced with each step of a program product to form the program product to thereby obtain various embodiments.
Furthermore, the basic extended hierarchy separating means of the separator separating means may be configured to generate a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property, calculate the re-quantization parameter from the input quantization parameter according to the re-quantization parameter deriving constant, and separate the basic hierarchy coefficient values and the extended hierarchy coefficient values, the extended coded signal generating means of the separator separating means may be configured to code information of the re-quantization parameter deriving constant as the quantization parameter reconstructing information within the plurality of extended coded signals, the extended coded signal converting means of the merging device merging means may be configured to decode the re-quantization parameter deriving constant form the extended coded signal, the basic quantization coefficient merging means of the merging device merging means may be configured to merge a sequence of the basic hierarchy coefficient values according to the re-quantization parameter deriving constant, and the third coefficient converting means of the merging device merging means may be configured to code an input quantization parameter reconstructed according to the re-quantization parameter and the re-quantization parameter deriving constant within the third coded signal.
Furthermore, the extended coded signal generating means of the separator separating means may be configured to code differential values between macroblocks of the re-quantization parameter deriving constant as the quantization parameter reconstructing information within the plurality of extended coded signals, the extended coded signal converting means of the merging device merging means may be configured to decode the differential values between the macroblocks of the re-quantization parameter deriving constant from the extended coded signal to obtain the re-quantization parameter deriving constant.
Furthermore, the extended coded signal generating means of the separator separating means may be configured to code the extended quantization coefficient sequence by a coding table in accordance with the re-quantization parameter deriving constant, and the extended coded signal converting means of the merging device merging means may be configured to decode the extended coded signal by the coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, the separator separating means may include a separator separating information input means for inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, the extended quantization coefficient separating means of the separator separating means may be configured to separate the extended hierarchy coefficient values separated by the basic extended hierarchy separating means into the plurality of extended quantization coefficient sequences according to the extended hierarchy separating pattern information inputted by the separator separating information input means, the merging device merging means may include a merging device separating information input means to input the extended hierarchy separating pattern information, and the extended quantization coefficient merging means of the merging device merging means may be configured to merge the plurality of extended quantization coefficient sequences converted by the extended coded signal converting means with a sequence of the extended hierarchy coefficient values according to the extended hierarchy separating pattern information inputted by the merging device separating information input means.
Furthermore, the separator may include a separator separating information transmitting means for transmitting extended hierarchy separating pattern information by which the extended quantization coefficient separating means of the separator separating means separates the extended hierarchy coefficient values into a plurality of extended quantization coefficient sequences to the merging device, the merging device may include a merging device separating information receiving means for receiving the extended hierarchy separating pattern information transmitted from the separator, and the merging device separating information input means of the merging device merging means may be configured to input the extended hierarchy separating pattern information received by the merging device separating information receiving means.
Further, the prediction error coefficient sequence coding means of the basic coded signal generating means of the separator may by configured to code the quantization parameter reconstructing information for reconstructing the input quantization parameter from the re-quantization parameter with the prediction error coefficient sequence to the prediction error coded sequence, the prediction error coefficient sequence converting means of the basic coded signal converting means of the merging device merging means may be configured to decode the quantization parameter reconstructing information from the prediction error coded sequence, the basic quantization coefficient merging means of the merging device merging means may be configured to merge a sequence of the basic hierarchy coefficient values according to the quantization parameter reconstructing information, and the third coefficient converting means of the merging device merging means may be configured to code an input quantization parameter reconstructed according to the re-quantization parameter and the quantization parameter reconstructing information within the third coded signal.
Furthermore, the basic extended hierarchy separating means of the separator separating means may be configured to generate a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property, calculate the re-quantization parameter from the input quantization parameter according to the re-quantization parameter deriving constant, and separate the basic hierarchy coefficient values and the extended hierarchy coefficient values, the prediction error coefficient sequence coding means of the basic coded signal generating means of the separator separating means may be configured to code information of the re-quantization parameter deriving constant as the quantization parameter reconstructing information with the prediction error coefficient sequence to the prediction error coded sequence, the prediction error coefficient sequence converting means of the basic coded signal converting means of the merging device merging means may be configured to decode the re-quantization parameter deriving constant from the prediction error coded sequence, the basic quantization coefficient merging means of the merging device merging means may be configured to merge a sequence of the basic hierarchy coefficient values according to the re-quantization parameter deriving constant, and the third coefficient converting means of the merging device merging means may be configured to code an input quantization parameter reconstructed according to the re-quantization parameter and the re-quantization parameter deriving constant within the third coded signal.
Furthermore, the prediction error coefficient sequence coding means of the basic coded signal generating means of the separator separating means may be configured to code differential values between macroblocks of the re-quantization parameter deriving constant as the quantization parameter reconstructing information with the prediction error coefficient sequence to the prediction error coded sequence, and the prediction error coefficient sequence converting means of the basic coded signal converting means of the merging device merging means may be configured to decode the differential values between the macroblocks of the re-quantization parameter deriving constant from the prediction error coefficient sequence to obtain the re-quantization parameter deriving constant.
Furthermore, the prediction error coefficient sequence coding means of the basic coded signal generating means of the separator separating means may be configured to code the prediction error coefficient sequence by a coding table in accordance with the re-quantization parameter deriving constant, and the prediction error coefficient sequence converting means of the basic coded signal converting means of the merging device merging means may be configured to decode the prediction error coded sequence by the coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, the extended coded signal generating means of the separator separating means may be configured to code the extended quantization coefficient sequence by a coding table in accordance with the re-quantization parameter deriving constant, and the extended coded signal converting means of the merging device merging means may be configured to decode the extended coded signal by the coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, the separator separating means may include a separator separating information input means for inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, the extended quantization coefficient separating means of the separator separating means may be configured to separate the extended hierarchy coefficient values separated by the basic extended hierarchy separating means into the plurality of extended quantization coefficient sequences according to the extended hierarchy separating pattern information inputted by the separator separating information input means, the merging device merging means may include a merging device separating information input means to input the extended hierarchy separating pattern information, and the extended quantization coefficient merging means of the merging device merging means may be configured to merge the plurality of extended quantization coefficient sequences converted by the extended coded signal converting means with a sequence of the extended hierarchy coefficient values according to the extended hierarchy separating pattern information inputted by the merging device separating information input means.
Furthermore, the separator may include a separator separating information transmitting means for transmitting extended hierarchy separating pattern information by which the extended quantization coefficient separating means of the separator separating means separates the extended hierarchy coefficient values into a plurality of extended quantization coefficient sequences to the merging device, the merging device may include a merging device separating information receiving means for receiving the extended hierarchy separating pattern information transmitted from the separator, and the merging device separating information input means of the merging device merging means may be configured to input the extended hierarchy separating pattern information received by the merging device separating information receiving means.
Furthermore, in the extended coded signal generating step in the separator separating step, the quantization parameter reconstructing information for reconstructing the input quantization parameter from the re-quantization parameter may be coded within each of the plurality of extended coded signals,
Furthermore, the separator separating step may include a separator separating information input step of inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences,
Furthermore, in the prediction error coefficient sequence coding step in the basic coded signal generating step of the separator, quantization parameter reconstructing information for reconstructing the input quantization parameter form the re-quantization parameter may be coded with the prediction error coefficient sequence to the prediction error coded sequence,
Furthermore, the separator separating step may include a separator separating information input step of inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences;
Furthermore, the present invention may include a coded signal decoding step of inputting the third coded signal generated in the merging device controlling step and decoding the third coded signal to reproduce the third moving picture.
Further, in the extended coded signal generating step in the separator separating step, quantization parameter reconstructing information for reconstructing the input quantization parameter from the re-quantization parameter may be coded within each of the plurality of extended coded signals.
Furthermore, in the basic extended hierarchy separating step in the separator separating step, a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property may be generated, the re-quantization parameter may be calculated from the input quantization parameter according to the re-quantization parameter deriving constant, and the basic hierarchy coefficient values and the extended hierarchy coefficient values may be separated, and in the extended coded signal generating step in the separator separating step, information of the re-quantization parameter deriving constant may be coded as the quantization parameter reconstructing information within the plurality of extended coded signals.
Furthermore, in the extended coded signal generating step, differential values between macroblocks of the re-quantization parameter deriving constant may be coded as the quantization parameter reconstructing information within the plurality of extended coded signals.
Furthermore, in the extended coded signal generating step, the extended quantization coefficient sequence may be coded by a coding table in accordance with the re-quantization parameter deriving constant. Furthermore, the present invention may include a separating information input step of inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, and in the extended quantization coefficient separating step in the separator separating step, the extended hierarchy coefficient values separated in the basic extended hierarchy separating step may be separated into the plurality of extended quantization coefficient sequences according to the extended hierarchy separating pattern information inputted in the separating information input step.
Furthermore, in the prediction error coefficient sequence coding step in the basic coded signal generating step, quantization parameter reconstructing information for reconstructing the input quantization parameter from the re-quantization parameter may be coded with the prediction error coefficient sequence to the prediction error coded sequence.
Furthermore, in the basic extended hierarchy separating step in the separator separating step, a re-quantization parameter deriving constant for calculating the re-quantization parameter that is appropriate based on a re-quantization property may be generated, the re-quantization parameter may be calculated from the input quantization parameter according to the re-quantization parameter deriving constant, and the basic hierarchy coefficient values and the extended hierarchy coefficient values may be separated, and in the prediction error coefficient sequence coding step in the basic coded signal generating step, information of the re-quantization parameter deriving constant may be coded as the quantization parameter reconstructing information with the prediction error coefficient sequence to the predicted error coded sequence.
Furthermore, in the prediction error coefficient sequence coding step, differential values between macroblocks of the re-quantization parameter deriving constant may be coded as the quantization parameter reconstructing information with the prediction error coefficient sequence to the prediction error coded sequence.
Furthermore, in the prediction error coefficient sequence coding step, the prediction error coefficient sequence may be coded by a coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, in the extended coded signal generating step in the separator separating step, the extended quantization coefficient sequence may be coded by the coding table in accordance with the re-quantization parameter deriving constant.
Furthermore, the present invention may include a separating information input step of inputting extended hierarchy separating pattern information defining a method of separating the extended hierarchy coefficient values into the plurality of extended quantization coefficient sequences, and in the extended quantization coefficient separating step in the separator separating step, the extended hierarchy coefficient values separated in the basic extended hierarchy separating step may be separated into the plurality of extended quantization coefficient sequences according to the extended hierarchy separating pattern information inputted in the separating information inputting step.
The present invention has an advantage that scalable transmission of images can be realized by generating independent, separated streams and performing transmitting path selection according to a network status in a router or the like having a band selecting function.
Incidentally, when the basic coded signal and the extended coded signal are each transmitted as a separated transmission coded signal, in other words, when the basic coded signal is transmitted independently, this basic coded signal becomes the same kind of signal as conventional transcoded signals, and thus the receiving side can receive the basic coded signal in a conventional manner by a regular receiving device, and can receive a compressed coded signal having a smaller code amount. Further, when the basic coded signal is multiplexed with the extended coded signals respectively and transmitted as transmission coded signals, the basic coded signal can be obtained from any one of the transmission coded signals, so that even when a communication error occurs, the basic coded signal can be obtained from one transmission coded signal. Therefore, images for reproduction can be safely and surely obtained.
Furthermore, when quantization parameter reconstructing information for reconstructing an input quantization parameter is coded within a transmission coded signal, a third coded signal that is the same as the first coded signal can be reconstructed including the input quantization parameter just by transmitting and receiving a small amount of information.
Further, when extended hierarchy coefficient values are separated into two or more according to the extended hierarchy separating pattern information, a manner of separating the extended hierarchy coefficient values can be easily changed by changing the extended hierarchy separating pattern information, and thus the separating manner can be changed for every transmission and reception.
Furthermore, by separately transmitting and receiving the contents of a program product to be transmitted such as images and sounds and the extended hierarchy separating pattern information, or by transmitting and receiving the extended hierarchy separating pattern information to/from a particular person, an unauthorized receiver cannot decode a transmission signal when it is read out in an unauthorized manner, so that the contents thereof cannot be viewed, and thus unauthorized accesses can be prevented as if the transmission signal is scrambled.
Hereinafter, a coded signal separating and merging system according to embodiments of the present invention will be described using the drawings.
A coded signal separating and merging system in each embodiment described below realizes scalable transmission of images by generating independent, separated streams in a transcoder having a stream separating function and carrying out a transmitting path selecting function according to a network status in a router or the like having a band selecting function. Priorities of respective separated streams are substantially equal to each other, so that the scalability can be realized by selecting an optional transmitting path in a router or the like.
A coded signal separating and merging system according to a first embodiment of the present invention is shown in
As shown in
Here, a stream inputted to the server 1000 is a stream generated in a standard coder, which is, for example, an image captured by a camera 600 and coded by a coder 700, or a stream of contents stored in a contents storage 800. Further, the server 1000 has a separator 1010, and the router 2000 has a transmitting path selector 2010.
Further, the receivers 3000a, 3000b . . . , 3000n respectively have merging devices 3010a, 3010b . . . , 3010n and decoders 3030a, 3030b . . . , 3030n, and the decoders 3030a, 3030b . . . , 3030n configured to perform reproduction in the receivers 3000a, 3000b . . . , 3000n comply with standard ones.
Here, since the receivers 3000a, 3000b . . . , 3000n, the merging devices 3010a, 3010b . . . , 3010n and the decoders 3030a, 3030b . . . , 3030n are the same ones respectively, one of each of them will be described as a receiver 3000, a merging device 3010, and a decoder 3030 below.
Next, the separator 1010 of the server 1000 is shown in
As shown in
As a method of generating separated streams by the separator 1010, there are two methods described below depending on processes in the multiplexing unit 1600, and process diagrams of these methods are shown in
A method (1) performs transmission of images in a network with a robustness transmitting path.
A method (2) performs transmission of images in a network without a robustness transmitting path.
The separating unit (separate) 1100 separates an inputted coded stream into one basic stream B and M number of extended streams E(m) (0≦m≦M−1).
The multiplexing unit (MUX) 1600 multiplexes the basic stream B and the extended stream E(m) (0≦m≦M−1) outputted from the separating unit 1100 according to the property of a transmitting path. As a method thereof, there are method (1) and method (2) described below.
Method (1)
In the method (1), St(B) and L number of streams St(1) (0≦1≦L−1) are outputted as separated streams. The basic stream B is transmitted as a separated stream St(B) to a robustness transmitting path in which transmission errors do not occur, and extended streams E(m) (0≦m≦M−1) are transmitted respectively as separated streams St(1) (0≦1≦L−1). The St(1) is shown by equation (1).
St(1)=E(1) (L=M, 0≦1≦M−1) equation (1)
The bit rate Rate[St(1)] of a first separated stream St(1) can be expressed as equation (2).
Rate[St(1)]=Rate[E(1)](L=M, 0≦1≦M−1) equation (2)
Method (2)
In the method (2), L number of streams St(1) (0≦1≦L−1) are outputted. The basic stream B is copied and multiplexed to all the extended streams E(m). Thus, by transmitting the basic stream B using plural transmitting paths, reproduction that does not depend on a network status is assured. In this case, the separated streams St(1) can be expressed as equation (3). Here, Multiplex [*] represents multiplexing.
St(1)=Multiplex [B, E(1)](L=M, 0≦1≦M−1) equation (3)
The bit rate Rate[St(1)] of the separated streams St(1) is represented by equation (4).
Rate[St(1)]=Rate[B+E(1)](L=M, 0≦1≦M−1) equation (4)
Next, the transmitting path selector 2010 of the router 2000 is shown in
The router 2000 and the transmission path selector 2010 inputs L number of separated streams St(1) outputted from the server 1000 and controls the rate of a stream to be transmitted by transmitting path selection (on/off of a transmitting path) according to a network status to realize scalability of images. Here, the total rate Rate[St out] of streams outputted from the router 2000 is Rate[St out]=Σ1 Rate[St(1)]. When the target total rate of output streams of the router 2000 is Rtarget, transmitting path switches of St(1), which fulfills Rate[St out]≦Rtarget, are turned on.
In
Next, the merging device 3010 of the receiver 3000 will be described. The merging device 3010 merges inputted L′ number of streams St(1) to reproduce an image. An image quality thereof depends only on the total rate of receiving streams and not on the types of receiving streams. The total rate of receiving streams is determined by transmitting path selection in the router 2000.
A block diagram of the merging device 3010 of the receiver 3000 is shown in
As shown in
Further, block diagrams of the merging device 3010 corresponding to the above-described method (1) and method (2) are shown in
The demultiplexing unit (DEMUX) 3100 divides inputted L′ number of streams St(1) into α number of basic streams B and M′ number of extended streams E(m). Here, α represents the total number of basic streams B multiplexed in all the separated streams St(1) inputted to the demultiplexing unit 3100, and the number thereof is different depending on the above described method (1) and method (2), which is shown by equation (5).
Further, information of errors or the like, which occur in a stream and are detected when the stream is divided, is passed to the merging controller 3020.
The basic stream selector (B-Selector) 3200 receives control from the merging controller 3020, selects one stream having less bit errors from the inputted α number of B streams, and outputs this stream. At this time, (α−1) number of basic streams B are discarded.
The merging unit (merge) 3300 receives control from the merging controller 3020, merges the inputted basic streams B and the extended streams E(m) (0≦m≦M′−1), and outputs a merged stream. Therefore, not all of the extended streams E(m) inputted to the merging unit 3300 are merged.
The merging controller (merge controller) 3020 receives error correction information from an error detector on the network and receives error information detected during processing in the demultiplexing unit 3100 and the merging unit 3300, controls the basic stream selector 3200 with respect to information regarding the basic stream B and controls the merging unit 3300 with respect to information regarding the extended stream E(m), and carries out a process to discard streams for which error correction is determined to be impossible in the merging controller 3020.
(Coding Syntax)
Hereinafter, separation performed in block unit constituting a macroblock, processing of a merging signal, and a data structure constituting a stream will be described.
First, the configuration and the format of the stream will be described.
The mode of a bit stream in this embodiment is such that an MPEG-2 bit stream (MPEG-2 prior to separation) complying the main profile is inputted and separated into a basic stream and M number of extended streams. MPEG-2 will be described below as an example, but regarding other DCT moving picture coding methods with motion compensation and prediction (ITU-T H. 261, ITU-T H. 263, ISO/IEC 14496-2 (commonly called “MPEG-4”), ITU-T H. 264, and the like), the same configuration can be used for the portion of the DCT coefficient coding therein.
The basic stream is reduced in bit rate and outputted as an MPEG-2 bit stream. The extended stream includes a prediction error stream constituted of prediction error information generated in a re-quantization process when reducing a rate, and a differential bit stream constituted of differential information between before and after reducing a rate.
The bit stream format of the extended stream is shown in
As shown in
In this embodiment, bit fields of the sequence hearer, GOP header, picture header, and slice header are equal to MPEG-2. However, in the slice layer, user data is defined between the slice header and the MB data, and in the user data, an attribute, which is original to this method, in the slice layer shown in
Next, synchronization between streams will be described.
The sequence header, the picture header, and the slice header are used for synchronization with an MPEG-2 bit stream to be outputted in GOP unit, picture unit, and slice unit, respectively. In the slice unit which is the smallest unit for synchronization, the synchronization is performed so that SSC (Slice Start Code) in each slice header becomes equal. Here, SSC in the slice header is a synchronization code denoting the start point of a slice layer, in which one last byte of a code represents the vertical position of the slice.
Next, principles of generating a stream will be described.
Re-quantization is performed, and variation difference information of quantization coefficient values before and after the re-quantization is coded in the block layer data. By merging the variation difference information and coded re-quantization output coefficients, coefficient information before the re-quantization can be completely reconstructed. At this time, the re-quantization output coefficients are coded as a basic hierarchy and becomes a basic stream B. When coefficient differential information before and after the re-quantization is an extended hierarchy, the extended hierarchy is separated sequentially into N number of extended quantization coefficient sequences. Prediction error information generated by the re-quantization becomes a prediction error coefficient sequence, and the prediction error coefficient sequence and the extended quantization coefficient sequences are coded respectively to generated M number of extended streams E(m). At this time, the rates of the M number of extended streams E(m) can be generated substantially equal to each other.
Next, the separating unit (separate) 1100 of the separator 1010 will be described.
A block diagram of a separating unit (separate) 1100a that is an embodiment of the separating unit (separate) 1100 is shown in
Each of the following items will be described:
Item 1. 1: Method of Generating a Basic Hierarchy and an Extended Hierarchy
Item 1. 2: Method of Generating a Basic Quantization Coefficient Sequence and a Prediction Error Coefficient Sequence by Re-Quantizing the Basic Hierarchy
Item 1. 3: Method of Generating a Basic Stream and Coding of the Basic Quantization Coefficient Sequence
Item 1. 4: Method of Generating an Extended Quantization Coefficient Sequence by Separating the Extended Hierarchy
Item 1. 5: Method of Generating Extended Streams
Coding of the Prediction Error Coefficient Sequence
Coding of the Extended Quantization Coefficient Sequence
However, in the aforementioned processing procedure, for the item 1. 2 and the item 1. 3, the same methods as those described in Japanese Patent Application Laid-open No. 2002-135130 “Device For Separating and Merging Coded Signal, Differential Coded Signal Generator, Method of Separating and Merging the Coded Signal, Method of Generating Differential Coded Signal, Medium In Which Program Product For Separating and Combining Coded Signals Is Stored, And Medium In Which Program Product For Generating the Differential Coded Signal Is Stored,” which was filed previously by the applicant of the present application, are used. Further, an integer h in this description is equal to “m” in the equation (8), specifically
in the paragraph [0324] of the aforementioned Japanese Patent Application Laid-open No. 2002-135130 in the case of intra-picture, and is equal to “m” in the equation (9), specifically
in the paragraph [0324] of the aforementioned Japanese Patent Application Laid-open No. 2002-135130 in the case of inter-picture. Further, a quantization parameter MQ1 prior to re-quantization and a re-quantization parameter MQ2 at the time of re-quantization, which will be described later, are the same as MQ1 and MQ2 in the aforementioned Japanese Patent Application Laid-open No. 2002-135130.
Item 1. 1: Method of Generating a Basic Hierarchy and an Extended Hierarchy
An input quantization coefficient sequence is separated to generate a basic hierarchy and an extended hierarchy. An input quantization coefficient sequence X is defined as equation (6).
X={x0, x1, x2 . . . , xi . . . }(0≦i≦63) equation (6)
Here, i (0≦i≦63) represents a coefficient position index number when a reference order in a block is the order of a zigzag scan. The basic hierarchy is a sequence of coefficients which are larger than an integer h among input quantization coefficients, and coefficients equal to or smaller than h are coefficients constituting the extended hierarchy.
From the above, the basic hierarchy B and the extended hierarchy E are expressed by equation (7).
Item 1. 2: Method of Generating a Basic Quantization Coefficient Sequence and a Prediction Error Coefficient Sequence by Re-Quantizing the Basic Hierarchy
The basic hierarchy B is defined as equation (8).
B={b0, b1 . . . , bi . . . }(0≦i≦63) equation (8)
Further, when the basic hierarchy B is re-quantized by the re-quantization parameter MQ2, a basic quantization coefficient sequence C which is a re-quantization output coefficient and a prediction error coefficient sequence D are outputted, which are defined respectively as equation (9).
Here, values ci produced by re-quantizing the basic hierarchy coefficients bi using a re-quantization parameter MQ2 are represented by equation (10) based on the equation (8) and the equation (9).
ci=func1(bi) (0≦i≦63) equation (10)
At this time, the re-quantization operation func1 is equal to the description in the paragraph [0393], specifically, QF2=QF1×MQ1/MQ2+sign(QF1)/2, in the above-described Japanese Patent Application Laid-open No. 2002-135130 in the case of intra-picture, and to the description in the paragraph [0395], specifically, QF2=(QF1+sign(QF1)×½)×MQ1/MQ2, in the above-described Japanese Patent Application Laid-open No. 2002-135130 in the case of inter-picture. Further, QF1 and QF2 in the paragraphs [0393] and [0395] in the above-described Japanese Patent Application Laid-open No. 2002-135130 are changed to bi and ci in this equation.
Prediction error coefficients di are obtained by equation (11) based on the equation (9).
di=bi−func2(ci) (0≦i≦63) equation (11)
Here, the equation (11) is equal to the process shown by equation (15),
ΔQ F[w]=Q Fnonzero-in [w]
specifically
in the paragraph [0429] in the above-described Japanese Patent Application Laid-open No. 2002-135130. Further, ΔQF[w], QFnonzero-in[w], and QFnonzero-out[w] of the equation (15) in the paragraph [0429] in the above-described Japanese Patent Application Laid-open No. 2002-135130 are changed to di, bi, and ci in this equation.
Item 1. 3: Method of Generating a Basic Stream
A method of coding the basic quantization coefficient sequence C generated in the item 1. 2 is the run length coding prescribed in MPEG-2.
Item 1. 4: Method of Generating an Extended Quantization Coefficient Sequence by Separating the Extended Hierarchy
An extended hierarchy E is defined as equation (12), and a process of generating an extended quantization coefficient sequence Fn (0≦n≦N−1) is shown below.
E={e0, e1, e2, . . . , es−1, . . . }(0≦S−1) equation (12)
Coefficients in the extended hierarchy E is distributed to N number of Fn (0≦n≦N−1) in order as shown in equation (13).
Item 1. 5: Method of Generating Extended Streams
Method of Coding the Prediction Error Coefficient Sequence
The prediction error coefficient sequence D generated in the item 1. 2 is scanned, and becomes a one-dimensional coefficient sequence constituted of non-zero coefficients only, which is coded to variable length codes to be an extended stream E(0).
b. Method of Coding the Extended Quantization Coefficient Sequence
The extended quantization coefficient sequences Fn (0≦n≦N−1) generated in Item 1. 4 are scanned respectively to be expressed in run length, and thereafter coded to extended streams E(m) (1≦m≦M−1) using a CBP (coded block pattern) and a VLC table in compliance with MPEG-2. At this time, the extended stream number m is incorporated in user data in the slice layer in respective extended streams. Further, the relationship between the extended stream number m and the index number n of the extended quantization coefficient sequence Fn is m=n+1.
The procedure of coding is shown below.
Next, the merging unit (merge) 3300 of the merging device 3010 will be described.
A block diagram of a merging unit (merge) 3300a that is an embodiment of the merging unit (merge) 3300 is shown in
Each of the following items will be described:
Item 2. 1: Decoding of a Basic Stream and Decoding of a Basic Quantization Coefficient Sequence
Item 2. 2: Decoding of Extended Streams
a. Decoding of a Prediction Error Coefficient Sequence
b. Decoding of Extended Quantization Coefficient Sequences
Item 2. 3: Method of Generating a Basic Hierarchy by Merging the Basic Quantization Coefficient Sequence and the Prediction Error Coefficient Sequence
Item 2. 4: Method of Generating an Extended Hierarchy by Merging the Extended Quantization Coefficient Sequences
Item 2. 5: Method of Generating a Merged Stream by Merging the Basic Hierarchy with the Extended Hierarchy
Item 2. 1: Decoding of a Basic Stream
As a method of decoding the basic stream B, a method prescribed in MPEG-2 is used. The basic stream is decoded from variable length codes up to the expression (run, level). From decoded run values, the order of zigzag scan of signals in a block is calculated, and into the values thereof, level values are inserted to generate a one-dimensional basic quantization coefficient sequence C*.
Item 2. 2: Decoding of Extended Streams
a. Decoding of a Prediction Error Coefficient Sequence
An extended stream E(0) is decoded from variable length codes to a prediction error coefficient sequence D*. A method of decoding the extended stream E(0) constituted of the prediction error coefficient sequence D* is the same as a method described in the above-described Japanese Patent Application Laid-open No. 2002-135130. Corresponding patents in respective countries corresponding to this Japanese Patent Application Laid-open No. 2002-135130 are European Patent Laid-open (EP) 1180900, U.S. Patent Application Laid-open 09/931038, Chinese Patent Application 01135747-9, and Korean Patent Application 2001-49741.
b. Decoding of Extended Quantization Coefficient Sequences
Extended streams E(m) (1≦m≦M′−1) are decoded from variable length codes to be expressed in combinations (run, level) using the CBP (coded block pattern) and the VLC table in compliance with MPEG-2. From decoded run values, the order of zigzag scan is calculated, and level values are inserted sequentially into positions in a block where a basic hierarchy is not inserted to generate one-dimensional extended quantization coefficient sequences F*n (0≦n≦N′−1).
Item 2. 3: Method of Generating a Basic Hierarchy by Merging the Basic Quantization Coefficient Sequence and the Prediction Error Coefficient Sequence
The basic quantization coefficient sequence C* and the prediction error coefficient sequence D* are merged with each other to generate a basic hierarchy B*. A method of generating the basic hierarchy B* will be shown below.
B* is defined as equation (14)
B*={b*0, b*1 . . . , b*i . . . }(0≦i≦63) equation (14)
The basic quantization coefficient sequence C* and the prediction error coefficient sequence D* are defined as equation (15).
From the equation (14) and the equation (15), basic hierarchy coefficients b*i* are generated by adding prediction error coefficients d*i to values produced by subjecting basic quantization coefficients c*i to the function func3, as shown in equation (16).
b*i =func3(c*i)+d*i equation (16)
Here, the equation (16) is equal to a process shown by equation (16), specifically
in the paragraph [0455] in the above-described Japanese Patent Application Laid-open No. 2002-135130. Further, ΔQF[w], QFnonzero-rec[w], and QFnonzero-out[w] of the equation (16) in the paragraph [0455] in the above-described Japanese Patent Application Laid-open No. 2002-135130 are changed to d*i, b*i, and c*i in this equation.
Item 2. 4: Method of Generating an Extended Hierarchy by Merging the Extended Quantization Coefficient Sequences
An index number n of the extended quantization coefficient sequence is obtained from the extended stream number m defined in the user data (
A method of generating the extended hierarchy E* will be described below. The extended hierarchy E* is defined as equation (17).
E*={e*0, e*1, e*2 . . . e*s . . . }(0≦s≦S−1) equation (17)
Further, the extended hierarchy quantization coefficient sequences F*n (0≦n≦N′−1) are defined as equation (18)
The index number n attached to each extended quantization coefficient sequence is used to perform merging as shown in equation (19) to generate the extended hierarchy E*.
E*={f*0(0), f*1(0) . . . , f*n(0) . . . , f*N′−1(0), f*0(1), f*1(1) . . . , f*n(1) . . . , f*N′−1(1) . . . , f*0(j), f*1(j) . . . , f*n(j) . . . , f*N′−1(j) . . . (j=0, 1, 2 . . . ) equation (19)
Item 2. 5: Method of Generating a Merged Stream by Merging the Basic Hierarchy with the Extended Hierarchy
Coefficient values of the generated basic hierarchy B* are inserted respectively into corresponding positions in the block according to the index number i. Next, coefficients of the extended hierarchy E* are inserted sequentially to the index number where the basic hierarchy is not inserted in the block to generate a merged quantization coefficient sequence X*. A method of generating the merged quantization coefficient sequence X* will be described below.
The merged quantization coefficient sequence is defined as equation (20).
X*={x*0, x*1, x*2 . . . , x*i . . . }(0≦i≦63) equation (20)
First, insertion of coefficients of the basic hierarchy B* into the merged quantization coefficient sequence X* provides equation (21).
x*i=b*i equation (21)
At this point of time, the value of a signal x*i in the merged quantization coefficient sequence X* into which no coefficient of the basic hierarchy is inserted is 0 (zero). Then, only x*i into which no coefficient of the basic hierarchy is inserted is extracted and numbered an index number k in order, thereby satisfying equation (22).
x*k=0 equation (22)
Finally, as shown in equation (23), by inserting coefficients of the extended hierarchy E* reconstructed in the item 2. 4 and shown by the equation (17) into x*k, the merged quantization coefficient sequence X* is generated.
x*k=e*k equation (23)
Furthermore, a resistance property to loss of streams will be described.
Merging operation in the case where any of the extended streams E(m) is lost while merging coefficients in a block, for example, in the case where an extended stream E(3) is lost while merging coefficients in a block is shown in
As shown in
Next, regarding a process in the separating unit 1100 in the separator 1010, a process in the case where a prediction error coefficient sequence D is multiplexed with a basic quantization coefficient sequence C to generate a basic stream B to thereby substantially equalize the rates of separated streams St(1) (0≦1≦L−1) will be described. In this embodiment, the relationship between an extended stream number m and an extended quantization coefficient sequence index number n is m =n.
A separating unit for multiplexing the basic quantization coefficient sequence C and the prediction error coefficient sequence D to generate the basic stream B is shown in
As shown in
The basic quantization coefficient multiplexer 1240 inputs a basic quantization coefficient sequence C outputted from a VLC basic 1210 and a prediction error coefficient sequence D outputted from a VLC prediction error 1220 and multiplexes the prediction error coefficient sequence D with the basic quantization coefficient sequence C to generate and outputs the basic stream B.
By the configuration as such, the separating unit 1100b multiplexes the prediction error coefficient sequence D with the basic quantization coefficient sequence C, so that the basic stream B can be generated.
Further, a merging unit 3300 of a merging device 3010 for the separating unit 1100b configured to generate such a basic stream B in which the basic quantization coefficient sequence C and the prediction error coefficient sequence D are multiplexed is shown in
As shown in
The basic quantization coefficient separating section 3430 separates an inputted basic stream B into a basic quantization coefficient sequence C and a prediction error coefficient sequence D, and outputs them to a VLD basic 3310 and a VLD prediction error 3320, respectively.
By the configuration as such, the merging unit 3300b can separate the basic quantization coefficient sequence C and the prediction error coefficient sequence D from the basic stream B.
Next, a case will be described in which the fact that the rates of separated streams St(1) are substantially even is utilized, and as a selecting method in the transmitting path selector 2010 in the router 2000, an optional transmitting path switch, by which the total output rate Rate[St out] of merged streams becomes equal to Rtarget or lower according to the target rate Rtarget, is turned on to enable reproduction of streams at the target rate.
In processing in the separator 1010, the rates Rate[E(m)] of extended streams E(m) are substantially even, and Rate[E(m)]≈q is defined. Here, Rate[A] represents the rate of A. At this time, if the rate of a basic stream B is adequately low as compared to the rates of extended streams E(m), the rates of separated streams St(1) become substantially even, so that the rates become Rate[St(1)]≈q. When the target rate Rtarget is Rtarget=4q, rates Rate[St out] of output streams become Rate[St out]≦4q by selecting any four separated streams St(1) (for example, selecting as shown in
Next, a case will be described in which, in a multiplexing unit 1600 in a separator 1010, extended streams E(m) are multiplexed to generate separated streams St(1) (0≦1≦L−1) each having a different rate.
By multiplexing extended streams E(m) in the multiplexing unit 1600, L number of separated streams according to the number of transmitting paths can be generated without changing the number M of extended streams outputted from a separating unit 1100, and the difference in rates of the separated streams St(1) is utilized so that the number of types of selectable rates becomes larger than the number of streams.
Block diagrams of a separator in this embodiment are shown in
As shown in
A concrete example of transmitting path selection in the transmitting path selector in the case where separated streams St(1) each having a different rate are generated as described above is shown in
As shown in
Here, operation of selecting an output rate and transmitting paths (changeover of switches) with L=3 and G1=1+1 other than the above-described example of selection of transmitting paths is shown in
As block diagrams of the merging device 3010 in this embodiment, a merging device for the method (1) of multiplexing a basic stream is shown in
Next, a case of merging extended streams E(m) so that the multiplexing parameter G1 becomes G1=21 and generating separated streams St(1) will be described.
Block diagrams of a separator in this embodiment is shown in
As shown in
A concrete example of transmitting path selection in a transmitting path selector in the case where separated streams St(1) each having a different rate are generated as described above is shown in
As shown in
Here, operation of selecting an output rate and transmitting paths (changeover of switches) with L=3 and G1=21 other than the above-described example of selection of transmitting paths is shown in
As block diagrams of the merging device 3010 in this embodiment, a merging device 3015 for the method (1) of multiplexing a basic stream is shown in
As above described method (1), when the basic coded signal and the extended coded signal are each transmitted as a separated transmission coded signal, in other words, when the basic coded signal is transmitted independently, this basic coded signal becomes the same kind of signal as conventional transcoded signals, and thus the basic coded signal can be received in a conventional manner by a regular receiving device on the receiving side. Therefore, a coded signal with a smaller, compressed code amount can be received.
Further, as above described method (2), when the basic coded signal is multiplexed with the extended coded signals respectively and transmitted as transmission coded signals, the basic coded signal can be obtained from any one of the transmission coded signals, so that even when a communication error occurs, the basic coded signal can be obtained from one transmission coded signal. Therefore, reproduced images can be safely and surely obtained.
Furthermore, a coded signal separating and merging system for coding quantization parameter reconstructing information within extended coded signals E(m) will be described in detail.
In the coded signal separating and merging system of this embodiment, there is generated a re-quantization parameter deriving constant to calculate an appropriate re-quantization parameter based on a re-quantization property, and an optimum re-quantization parameter is calculated by the re-quantization parameter deriving constant and an input quantization parameter. By coding information of the re-quantization parameter deriving constant used here as the quantization parameter reconstructing information within the extended coded signals E(m), the input quantization parameter is reconstructed from the basic coded signal B and the extended coded signal E(m).
Further, in this embodiment, extended hierarchy separating pattern information which defines a method of separating extended hierarchy coefficient values is used to separate coefficient values of an extended hierarchy into plural extended hierarchy coefficient sequences.
First, the re-quantization parameter deriving constant will be described. When inputted coefficient values are re-quantized (transcoded), the re-quantized coefficient values do not always become coefficient values of a constant percentage in accordance with the value of the re-quantization parameter. Accordingly, the value of the re-quantization parameter is restricted so that the coefficient values after re-quantization are efficiently quantized.
Here, the input quantization parameter is MQ1, and a re-quantization parameter obtained by calculation is MQ′2, and MQ1 and MQ′2 are used to calculate an integer h with the following equation (24) and equation (25).
Intra-picture:
Inter-picture:
Here, └ ┘ represents rounding operation to an integer.
The above-described h and the input quantization parameter MQ1 are used to calculate the re-quantization parameter MQ2 with the following equation (26) and equation (27).
Intra-picture:
Inter-picture:
MQ2=(h 301)×MQ1 equation (27)
Here, since the equation (26) and the equation (27) are integer operation, the re-quantization parameter MQ2 to be obtained is limited to a specific value. This h is defined as the re-quantization parameter deriving constant.
Hereinafter, the coded signal separating and merging system for separating the basic hierarchy and the extended hierarchy using the re-quantization parameter deriving constant h to code respective coefficient sequences will be described.
A separating unit of a separator configured to perform a coding process using the re-quantization parameter deriving constant h is shown in
As shown in
The basic extended hierarchy separator 1130c calculates the re-quantization parameter deriving constant h and uses this re-quantization parameter deriving constant h to separate a quantization coefficient sequence X converted by the run-level coefficient converter 1120 into a basic hierarchy coefficient sequence B and an extended hierarchy coefficient sequence E. Here, a coefficient larger than the re-quantization parameter deriving constant h becomes a basic hierarchy coefficient value and a coefficient equal to or smaller than the re-quantization parameter deriving constant h becomes an extended hierarchy coefficient value.
The basic quantization coefficient converter 1140c inputs the basic hierarchy coefficient sequence B separated by the basic extended hierarchy separator 1130c, inputs the re-quantization parameter deriving constant h as well, calculates a re-quantization parameter MQ2 by the re-quantization parameter deriving constant h and the input quantization parameter MQ1, re-quantizes the inputted basic hierarchy coefficient sequence B using the calculated re-quantization parameter MQ2, and converts it into a basic quantization coefficient sequence C and a surplus coefficient sequence (prediction error coefficient sequence) D, which is differential information between the basic hierarchy coefficient sequence B and the basic quantization coefficient sequence C.
The extended quantization coefficient separator 1150c inputs the extended hierarchy coefficient sequence E separated by the basic extended hierarchy separator 1130c and separates it into plural extended quantization coefficient sequences F0, F1 . . . , FN−1 according to extended hierarchy separating pattern information which defines a method of separating the extended hierarchy.
Here, the extended hierarchy separating pattern information may be prepared internally in advance or inputted from the outside. Further, the extended hierarchy separating pattern information is, for example, information such as “an extended hierarchy coefficient sequence E is separated into four and assigned one by one in order from the first one of coefficient values.”
The prediction error variable length coder (VLC prediction error) 1220c inputs the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130c, codes the surplus coefficient sequence (prediction error coefficient sequence) D, which is re-scanned by the re-scanner 1170, to variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h to generate a surplus coefficient (prediction error coefficient) coded signal, and outputs it as an extended coded signal E(0).
Further, at this time, quantization parameter reconstructing information is coded together in the extended coded signal E(0). As the quantization parameter reconstructing information, information of the re-quantization parameter deriving constant h is coded. Specifically, the re-quantization parameter deriving constant h is added to macroblock attribute information in the macroblock layer. In
Further, the quantization parameter reconstructing information may be the re-quantization parameter deriving constant h as it is as described above, but differential values of the re-quantization parameter deriving constant h between macroblocks can be used as the quantization parameter reconstructing information to further reduce the code amount.
The extended variable length coders (VLC/F0) 1231c, (VLC/F1) 1232c . . . , (VLC/FN−1) 1233c input the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130c and respectively codes the extended quantization coefficient sequences F0, F1 . . . FN−1 respectively re-scanned by the re-scanner 1181, 1182 . . . , 1183 to variable length codes by the coding table in accordance with the re-quantization parameter deriving constant h to thereby generate and output extended coded signals E(1), E(2) . . . , E(M−1).
Here, a coding table of coefficient values (level values) in the case of using original variable length codewords according to the re-quantization parameter deriving constant h is shown in
Further, at this time, similarly to the prediction error variable length coder (VLC prediction error) 1220c, the quantization parameter reconstructing information is coded together in the extended coded signals E(1), E(2) . . . , E (M−1). Contents of the quantization parameter reconstructing information are the same as those for the prediction error variable length coder (VLC prediction error) 1220c.
Further, the coefficient information separating section 1260, variable length decoder (VLD) 1110, run-level coefficient converter 1120, basic re-scanner 1160, prediction error re-scanner 1170, extended re-scanners 1181, 1182 . . . , 1183, basic variable length coder (VLC basic) 1210, and coefficient information multiplexing section 1270a of the separating unit 1100c are the same as those represented by the same numerals in the separating unit 1100a.
By the above configuration, in the separating unit 1100c, when a coded stream (first coded signal) is inputted, the quantization coefficient sequence X converted via the coefficient information separating section 1260, the variable length decoder (VLD) 1110, and the run-level coefficient converter 1120 is inputted to the basic extended hierarchy separator 1130c. In the basic extended hierarchy separator 1130c, the re-quantization parameter deriving constant h is calculated, and this re-quantization parameter deriving constant h is used to separate the inputted quantization coefficient sequence X into the basic hierarchy coefficient sequence B and the extended hierarchy coefficient sequence E.
The basic hierarchy coefficient sequence B separated by the basic extended hierarchy separator 1130c is inputted to the basic quantization coefficient converter 1140c. In the basic quantization coefficient converter 1140c, the inputted basic hierarchy coefficient sequence B is re-quantized using the re-quantization parameter MQ2 calculated with the re-quantization parameter deriving constant h, and converted into the basic quantization coefficient sequence C and the surplus coefficient sequence (prediction error coefficient sequence) D.
The basic quantization coefficient sequence C converted by the basic quantization coefficient converter 1140c is converted via the basic re-scanner 1160, the basic variable length coder (VLC basic) 1210, and the coefficient information multiplexing section 1270a into the basic coded signal B and outputted.
Further, the surplus coefficient sequence (prediction error coefficient sequence) D converted by the basic quantization coefficient converter 1140c is inputted via the prediction error re-scanner 1170 to the prediction error variable length coder (VLC prediction error) 1220c. The prediction error variable length coder (VLC prediction error) 1220c inputs the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130c, codes the surplus coefficient sequence (prediction error coefficient sequence) D to variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h, generates the surplus coefficient (prediction error coefficient) coded signal, and outputs it as the extended coded signal E(0).
On the other hand, the extended hierarchy coefficient sequence E separated by the basic extended hierarchy separator 1130c is inputted to the extended quantization coefficient separator 1150c. In the extended quantization coefficient separator 1150c, the inputted extended hierarchy coefficient sequence E is separated into plural extended quantization coefficient sequences F0, F1 . . . , FN−1 according to the extended hierarchy separating pattern information.
The plural extended quantization coefficient sequences F0, F1 . . . , FN−1 separated by the extended quantization coefficient separator 1150c are inputted via the extended re-scanners 1181, 1182, . . . , 1183 to the extended variable length coders (VLC/F0) 1231c, (VLC/F1) 1232c . . . , (VLC/FN−1) 1233c, respectively. The extended variable length coders (VLC/F0) 1231c, (VLC/F1) 1232c . . . , (VLC/FN−1) 1233c respectively input the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130c, codes the extended quantization coefficient sequences F0, F1 . . . , FN−1 to variable length codes by the coding table in accordance with the re-quantization parameter deriving constant h to thereby generate and output extended coded signals E(1), E(2) . . . , E(M−1).
Next, a merging unit of a merging device for performing decoding processing using the re-quantization parameter deriving constant h is shown in
As shown in
The prediction error variable length decoder (VLD prediction error) 3320c inputs an extended coded signal E*(0), obtains the re-quantization parameter deriving constant h from quantization parameter reconstructing information inside the extended coded signal E*(0), decodes surplus coefficient (prediction error coefficient) coded signal from variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h, and outputs surplus coefficient (prediction error coefficient) information (run-level information).
Here, the coding table according to the re-quantization parameter deriving constant h is the same as the coding table used in the prediction error variable length coder (VLC prediction error) 1220c of the separating unit 1100c of the above-described separator.
The extended variable length decoders (VLD/E(0)) 3331c, (VLD/E(1)) 3332c . . . , (VLD/E(N′−1)) 3333c input extended coded signals E*(1), E*(2) . . . , E*(M′−1) respectively, obtain the re-quantization parameter deriving constant h from quantization parameter reconstructing information in the respective extended coded signals E*(1), E*(2) . . . , E*(M′−1), decode the extended coded signals E*(1), E*(2) . . . , E*(M′−1) from variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h, and output extended quantization coefficient information (run-level information).
Further, the coding table according to the re-quantization parameter deriving constant h is also the same as the coding table used in the extended variable length coders (VLC/F0) 1231c, (VLC/F1) 1232c . . . , (VLC/FN−1) 1233c of the separating unit 1100c of the above-described separator.
The basic quantization coefficient merging section 3370c inputs, along with the basic quantization coefficient sequence C* converted by the basic coefficient converter 3340 and the prediction error coefficient sequence D* converted by the prediction error coefficient converter 3350, the re-quantization parameter deriving constant h from one of the prediction error variable length decoders (VLD prediction error) 3320c and the extended variable length decoders (VLD/E(0)) 3331c, (VLD/E(1)) 3332c . . . , (VLD/E(N′−1)) 3333c. Then, the input quantization parameter MQ1 is reconstructed from the re-quantization parameter MQ2 and the re-quantization parameter deriving constant h. Furthermore, the reconstructed input quantization parameter MQ1 and the re-quantization parameter MQ2 are used to merge the inputted basic quantization coefficient sequence C* and the prediction error coefficient sequence D* to generate the basic hierarchy coefficient sequence B*.
The extended quantization coefficient merging section 3380c inputs extended quantization coefficient sequences F*0, F*1 . . . , F*N−1 converted by the extended coefficient converters 3361, 3362 . . . , 3363 and internally or externally inputs extended hierarchy separating pattern information, and merges the extended quantization coefficient sequences F*0, F*1 . . . , F*N−1 into one extended hierarchy coefficient sequence E* according to the extended hierarchy separating pattern information.
The coefficient information separating section 3450a, basic variable length decoder (VLD basic) 3310, basic coefficient converter 3340, prediction error coefficient converter 3350, extended coefficient converters 3361, 3362 . . . , 3363, basic extended hierarchy merging section 3390, run-level coefficient converter 3410, variable length coder (VLC) 3420, and coefficient information multiplexing section 3460 of the merging unit 3300c are the same as those represented by the same numerals in the merging unit 3300a.
By the above configuration, in the merging unit 3300c, when the basic coded signal B* is inputted, the basic quantization coefficient sequence C* converted via the coefficient information separating section 3450a, the basic variable length decoder (VLD basic) 3310, and the basic coefficient converter 3340 is inputted to the basic quantization coefficient merging section 3370.
Further, when the extended coded signal E*(0) is inputted, it is inputted to the prediction error variable length decoder (VLD prediction error) 3320c, the re-quantization parameter deriving constant h is obtained from quantization parameter reconstructing information inside the extended coded signal E*(0), and by the coding table in accordance with the re-quantization parameter deriving constant h, the extended coded signal E*(0) is decoded from variable length codes to the surplus coefficient (prediction error coefficient) information (run-level information). This decoded surplus coefficient (prediction error coefficient) information is converted into a prediction error coefficient sequence D* in the prediction error coefficient converter 3350 and inputted to the basic quantization coefficient merging section 3370c.
The basic quantization coefficient merging section 3370c inputs the re-quantization parameter deriving constant h with the basic quantization coefficient sequence C* and the prediction error coefficient sequence D*, and calculates the input quantization parameter MQ1 from the re-quantization parameter MQ2. Furthermore, the input quantization parameter MQ1 is used to merge the inputted basic quantization coefficient sequence C* and the prediction error coefficient sequence D* to generate a basic hierarchy coefficient sequence B*, which is outputted to the basic extended hierarchy merging section 3390.
On the other hand, when the extended coded signals E*(1), E*(2) E*(M′−1) are inputted, they are inputted respectively to the extended variable length decoders (VLD/E(0)) 3331c, (VLD/E(1)) 3332c . . . , (VLD/E(N′−1)) 3333c, the re-quantization parameter deriving constant h is obtained from the quantization parameter reconstructing information in the respective extended coded signals E*(1), E*(2) E*(M′−1), and by a coding table in accordance with the re-quantization parameter deriving constant h, the extended coded signals are decoded from variable length codes to the extended quantization coefficient information (run-level information). The decoded respective extended quantization coefficient information are -converted in the extended coefficient converters 3361, 3362 . . . , 3363 into the extended quantization coefficient sequences F*0, F*1 . . . , F*N−1 and inputted to the extended quantization coefficient merging section 3380c. In the extended quantization coefficient merging section 3380c, the inputted extended quantization coefficient sequences F*0, F*1 . . . , F*N′−1 are merged into one extended hierarchy coefficient sequence E* according to the extended hierarchy separating pattern information and outputted to the basic extended hierarchy merging section 3390.
The basic extended hierarchy merging section 3390 inputs the basic hierarchy coefficient sequence B* and the extended hierarchy coefficient sequence E* and merges them to generate a merged quantization coefficient sequence (third quantization coefficient sequence) X*. The merged quantization coefficient sequence (third quantization coefficient sequence) X* is converted into a merged stream (third coded signal) via the run-level coefficient converter 3410, variable length coder (VLC) 3420, and coefficient information multiplexing section 3460 to be outputted.
As has been described above, in the coded signal separating and merging system in this embodiment, by coding the quantization parameter reconstructing information within the extended coded signal E(m), an original coded stream (first coded signal) including the input quantization parameter MQ1 can be completely reconstructed on the merging device side even from a conventional compressed coded signal (normal MPEG-2 bit stream) as it is, without incorporating information of the input quantization parameter MQ1 in the basic coded signal B, and the code amount of the extended coded signal E(m) can be made smaller as compared to the case where the input quantization parameter MQ1 itself is coded.
Next, there will be described a process in the case where a prediction error coefficient sequence D is multiplexed with a basic quantization coefficient sequence C to generate a basic stream B in a coded signal separating and merging system for coding quantization parameter reconstructing information within an extended coded signal E(m).
Hereinafter, a coded signal separating and merging system for separating a basic hierarchy and an extended hierarchy using a re-quantization parameter deriving constant h to code respective coefficient sequences will be described.
A separating unit of a separator configured to perform a coding process using the re-quantization parameter deriving constant h and multiplexing the prediction error coefficient sequence D with the basic quantization coefficient sequence C to generate the basic stream B is shown in
As shown in
The basic extended hierarchy separator 1130d calculates the re-quantization parameter deriving constant h and uses this re-quantization parameter deriving constant h to separate a quantization coefficient sequence X converted by the run-level coefficient converter 1120 into a basic hierarchy coefficient sequence B and an extended hierarchy coefficient sequence E, and is the same as the basic extended hierarchy separator 1130c of the above-described separating unit 1100c.
The basic quantization coefficient converter 1140d inputs the basic hierarchy coefficient sequence B separated by the basic extended hierarchy separator 1130d, inputs the re-quantization parameter deriving constant h as well, calculates a re-quantization parameter MQ2 by the re-quantization parameter deriving constant h and the input quantization parameter MQ1, re-quantizes the inputted basic hierarchy coefficient sequence B using the calculated re-quantization parameter MQ2, and converts it into a basic quantization coefficient sequence C and a surplus coefficient sequence (prediction error coefficient sequence) D. The basic quantization coefficient converter 1140d is the same as the basic quantization coefficient converter 1140c of the above-described separating unit 1100c.
The extended quantization coefficient separator 1150d inputs the extended hierarchy coefficient sequence E separated by the basic extended hierarchy separator 1130d and separates it into plural extended quantization coefficient sequences F0, F1 . . . , FN−1 according to extended hierarchy separating pattern information which defines a method of separating the extended hierarchy, and is the same as the extended quantization coefficient separator 1150c of the above-described separating unit 1100c. Further, the extended hierarchy separating pattern information used in this extended quantization coefficient separator 1150d is the same as that in the description of the extended quantization coefficient separator 1150c of the above-described separating unit 1100c, which may be prepared internally in advance or inputted from the outside, similarly to the above description.
The prediction error variable length coder (VLC prediction error) 1220d inputs the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130d, and codes the surplus coefficient sequence (prediction error coefficient sequence) D, which is re-scanned by the re-scanner 1170, to variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h to generate a surplus coefficient (prediction error coefficient) coded signal D.
Further, at this time, quantization parameter reconstructing information is coded together in the surplus coefficient (prediction error coefficient) coded signal D, which is similar to that in the prediction error variable length coder (VLC prediction error) 1220c of the above-described separating unit 1100c. Also, the quantization parameter reconstructing information is the same as that in the description of the prediction error variable length coder (VLC prediction error) 1220c of the above-described separating unit
The extended variable length coders (VLC/F0) 1231d, (VLC/F1) 1232d . . . , (VLC/FN−1) 1233d respectively code extended quantization coefficient sequences F0, F1 . . . FN−1, which are respectively re-scanned by the re-scanners 1181, 1182 . . . , 1183, to variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h to generate the extended coded signals E(0), E(1) . . . , E(M−1) and outputs them. The re-quantization parameter deriving constant h may be inputted from the basic extended hierarchy separator 1130d, or from the prediction error variable length coder (VLC prediction error) 1220d.
However, different from the prediction error variable length coder (VLC prediction error) 1220d, the quantization parameter reconstructing information is not coded within the extended coded signals E(0), E(1) . . . , E(M−1). This is because the surplus coefficient sequence (prediction error coefficient sequence) D in which the quantization parameter reconstructing information is coded is included in the basic coded signal B that is constantly received, and the quantization parameter reconstructing information coded in the surplus coefficient sequence (prediction error coefficient sequence) D can be used.
Further, the coefficient information separating section 1260, variable length decoder (VLD) 1110, run-level coefficient converter 1120, basic re-scanner 1160, prediction error re-scanner 1170, extended re-scanners 1181, 1182 . . . , 1183, basic variable length coder (VLC basic) 1210, basic quantization coefficient multiplexer 1240, and coefficient information multiplexing section 1270b of the separating unit 1100d are the same as those represented by the same numerals in the separating unit 1100b.
By the above configuration, in the separating unit 1100d, when a coded stream (first coded signal) is inputted, the quantization coefficient sequence X converted via the coefficient information separating section 1260, the variable length decoder (VLD) 1110, and the run-level coefficient converter 1120 is inputted to the basic extended hierarchy separator 1130d. In the basic extended hierarchy separator 1130d, the re-quantization parameter deriving constant h is calculated, and this re-quantization parameter deriving constant h is used to separate the inputted quantization coefficient sequence X into the basic hierarchy coefficient sequence B and the extended hierarchy coefficient sequence E.
The basic hierarchy coefficient sequence B separated by the basic extended hierarchy separator 1130d is inputted to the basic quantization coefficient converter 1140d. In the basic quantization coefficient converter 1140d, the inputted basic hierarchy coefficient sequence B is re-quantized using the re-quantization parameter MQ2 calculated with the re-quantization parameter deriving constant h, and converted into the basic quantization coefficient sequence C and the surplus coefficient sequence (prediction error coefficient sequence) D.
The basic quantization coefficient sequence C converted by the basic quantization coefficient converter 1140d is converted via the basic re-scanner 1160 and the basic variable length coder (VLC basic) 1210 into the basic quantization coefficient coded signal and outputted to the basic quantization coefficient multiplexer 1240.
Further, the surplus coefficient sequence (prediction error coefficient sequence) D converted by the basic quantization coefficient converter 1140d is inputted via the prediction error re-scanner 1170 to the prediction error variable length coder (VLC prediction error) 1220d. The prediction error variable length coder (VLC prediction error) 1220d inputs the re-quantization parameter deriving constant h from the basic extended hierarchy separator 1130d, codes the surplus coefficient sequence (prediction error coefficient sequence) D to variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h, generates the surplus coefficient (prediction error coefficient) coded signal, and outputs it to the basic quantization coefficient multiplexer 1240.
Subsequently, in the basic quantization coefficient multiplexer 1240, the inputted basic quantization coefficient coded signal and the surplus coefficient (prediction error coefficient) coded signal are multiplexed, and a basic coded signal B is generated in the coefficient information multiplexing section 1270b and outputted.
On the other hand, the extended hierarchy coefficient sequence E separated by the basic extended hierarchy separator 1130d is inputted to the extended quantization coefficient separator 1150d, and in the extended quantization coefficient separator 1150d, the inputted extended hierarchy coefficient sequence E is separated according to the aforementioned extended hierarchy separating pattern information into plural extended quantization coefficient sequences F0, F1 . . . , FN−1.
The plural extended quantization coefficient sequences F0, F1 . . . , FN−1 separated by the extended quantization coefficient separator 1150d are inputted via the extended re-scanners 1181, 1182 . . . , 1183 to the extended variable length coders (VLC/F0) 1231d, (VLC/F1) 1232d . . . , (VLC/FN−1) 1233d respectively. The extended variable length coders (VLC/F0) 1231d, (VLC/F1) 1232d . . . , (VLC/FN−1) 1233d respectively code the extended quantization coefficient sequences F0, F1 . . . , FN−1 to variable length codes by the coding table in accordance with the re-quantization parameter deriving constant h to generate and output the extended coded signals E(0), E(1) . . . , E(M−1).
Next, a merging unit of a merging device configured to separate the basic quantization coefficient sequence C and the surplus coefficient sequence (prediction error coefficient sequence) D from the basic coded signal B and perform a decoding process using the re-quantization parameter deriving constant h is shown in
As shown in
The prediction error variable length decoder (VLD prediction error) 3320d inputs the surplus coefficient (prediction error coefficient) coded signal separated by the basic quantization coefficient separating section 3430, obtains the re-quantization parameter deriving constant h from the quantization parameter reconstructing information in the surplus coefficient (prediction error coefficient) coded signal, and decodes the surplus coefficient (prediction error coefficient) coded signal from variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h to output surplus coefficient (prediction error coefficient) information (run-level information).
Here, the coding table according to the re-quantization parameter deriving constant h is the same as the coding table used in the prediction error variable length coder (VLC prediction error) 1220d of the separating unit 1100d of the above-described separator.
The extended variable length decoders (VLD/E(0)) 3331d, (VLD/E(1)) 3332d . . . , (VLD/E(N′−1)) 3333d input extended coded signals E*(0), E*(1) . . . , E*(M′−1) respectively, input the re-quantization parameter deriving constant h from the prediction error variable length decoder (VLD prediction error) 3320d, decode the extended coded signals E*(0), E*(1) . . . , E*(M′−1) from variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h, and respectively output extended quantization coefficient information (run-level information).
Further, the coding table according to the re-quantization parameter deriving constant h is the same as the coding table used in the extended variable length coders (VLC/F0) 1231c, (VLC/F1) 1232c . . . , (VLC/FN−1) 1233c of the separating unit 1100d of the above-described separator.
The basic quantization coefficient merging section 3370d inputs, along with the basic quantization coefficient sequence C* converted by the basic coefficient converter 3340 and the prediction error coefficient sequence D* converted by the prediction error coefficient converter 3350, the re-quantization parameter deriving constant h from the prediction error variable length decoder (VLD prediction error) 3320d. Then, the input quantization parameter MQ1 is reconstructed from the re-quantization parameter MQ2 and the re-quantization parameter deriving constant h. Further, the reconstructed input quantization parameter MQ1 and the re-quantization parameter MQ2 are used to merge the inputted basic quantization coefficient sequence C* and the prediction error coefficient sequence D* to generate the basic hierarchy coefficient sequence B*.
The extended quantization coefficient merging section 3380d inputs extended quantization coefficient sequences F*0, F*1. . . , F*N′−1 converted by the extended coefficient converters 3361, 3362 . . . , 3363 and merges the extended quantization coefficient sequences F*0, F*1 . . . , F*N−1 into one extended hierarchy coefficient E* according to the extended hierarchy separating pattern information, and is the same as the extended quantization coefficient merging section 3380c of the above-described merging unit 3300c. Further, the extended hierarchy separating pattern information may be provided internally or inputted from the outside, similarly to the extended quantization coefficient merging section 3380c of the merging unit 3300c.
The coefficient information separating section 3450a, basic quantization coefficient separating section 3430, basic variable length decoder (VLD basic) 3310, basic coefficient converter 3340, prediction error coefficient converter 3350, extended coefficient converters 3361, 3362 . . . , 3363, basic extended hierarchy merging section 3390, run-level coefficient converter 3410, variable length coder (VLC) 3420 and coefficient information multiplexing section 3460 of the merging unit 3300d are the same as those represented by the same numerals in the merging unit 3300b.
By the above configuration, in the merging unit 3300d, when the basic coded signal B* is inputted, coefficient information of the basic hierarchy is inputted via the coefficient information separating section 3450a to the basic quantization coefficient separating section 3430 and separated in the basic quantization coefficient separating section 3430 into the basic quantization coefficient coded signal and the surplus coefficient (prediction error coefficient) coded signal.
The basic quantization coefficient coded signal separated in the basic quantization coefficient separating section 3430 is converted via the basic variable length decoder (VLD basic) 3310 and the basic coefficient converter 3340 into the basic quantization coefficient sequence C* and inputted to the basic quantization coefficient merging section 3370d.
The surplus coefficient (prediction error coefficient) coded signal separated in the basic quantization coefficient separating section 3430 is inputted to the prediction error variable length decoder (VLD prediction error) 3320d, the re-quantization parameter deriving constant h is obtained from the quantization parameter reconstructing information in the surplus coefficient (prediction error coefficient) coded signal, and the surplus coefficient (prediction error coefficient) coded signal is decoded from variable length codes by a coding table in accordance with the re-quantization parameter deriving constant h to surplus coefficient (prediction error coefficient) information (run-level information). This decoded surplus coefficient (prediction error coefficient) information is converted into a prediction error coefficient sequence D* in the prediction error coefficient converter 3350 and inputted to the basic quantization coefficient merging section 3370d.
The basic quantization coefficient merging section 3370d inputs the re-quantization parameter deriving constant h with the basic quantization coefficient sequence C* and the prediction error coefficient sequence D*, and calculates the input quantization parameter MQ1 from the re-quantization parameter MQ2. Further, the input quantization parameter MQ1 is used to merge the inputted basic quantization coefficient sequence C* and the prediction error coefficient sequence D* to generate a basic hierarchy coefficient sequence B*, which is outputted to the basic extended hierarchy merging section 3390.
On the other hand, when the extended coded signals E*(0), E*(1) . . . , E*(M′−1) are inputted, they are inputted respectively to the extended variable length decoders (VLD/E(0)) 3331d, (VLD/E(1)) 3332d . . . , (VLD/E (N′−1)) 3333d, the re-quantization parameter deriving constant h is inputted from the prediction error variable length decoder (VLD prediction error) 3320d, and by a coding table in accordance with the re-quantization parameter deriving constant h, the extended coded signals are decoded from variable length codes to the extended quantization coefficient information (run-level information). The decoded respective extended quantization coefficient information are converted in the extended coefficient converters 3361, 3362 . . . , 3363 respectively into the extended quantization coefficient sequences F*0, F*1 . . . , F*N−1 and inputted to the extended quantization coefficient merging section 3380d. In the extended quantization coefficient merging section 3380d, the inputted extended quantization coefficient sequences F*0, F*1 . . . , F*N′−1 are merged into one extended hierarchy coefficient sequence E* according to the extended hierarchy separating pattern information and outputted to the basic extended hierarchy merging section 3390.
The basic extended hierarchy merging section 3390 inputs the basic hierarchy coefficient sequence B* and the extended hierarchy coefficient sequence E* and merges them to generate a merged quantization coefficient sequence (third quantization coefficient sequence) X*. The merged quantization coefficient sequence (third quantization coefficient sequence) X* is converted into a merged stream (third coded signal) via the run-level coefficient converter 3410, variable length coder (VLC) 3420, and coefficient information multiplexing section 3460 to be outputted.
As has been described above, in the coded signal separating and merging system of this embodiment, since the quantization parameter reconstructing information is handled as the surplus coefficient sequence (prediction error coefficient sequence) D and multiplexed to the basic coded signal B, it is unnecessary to include any information for reconstructing the quantization parameter in the extended coded signals E(m), so that the code amounts of many extended coded signals E(m) can be reduced, and also an original quantization parameter MQ1 can be obtained by receiving only the basic coded signal B.
Although the separator input means is not illustrated in those embodiments, it is arranged in the separator 1010, 1010a, 1010b, 1011 and 1015, or it is arranged besides each separator, and is arranged anyway at the preceding of the separating unit 1100, 1100a, 1100b, 1101, and 1105. This separator input means stores for example, coded streams temporarily, or carries out operation which reads the function of a processing program. Moreover, the separator separating means is equivalent to the separating unit 1100, 1100a, 1100b, 1101, and 1105 in each figure, and the separator multiplexing means is equivalent to the multiplexing unit 1600, 1600a, 1600b, 1601, and 1605. Moreover, although the separator output means is not illustrated, it is arranged in the separator, or it is arranged out of the separator, and is arranged anyway at the rear of the multiplexing unit. This the separator output means outputs for example, transmission coded signals for every lump, or carries out operation which reads the function which performs to output.
It is desirable that the basic stream B used as the basic coded signal has {fraction (1/10)}-{fraction (1/20)} of the amount of a first coded signal when it is taken into consideration the quality and the access speed of pictures. Moreover, although it is made for the extended stream used as extended coded signals to serve as the respectively equal amount of data with the form of above-mentioned embodiments, there is no necessity of making it division into equal amount.
Moreover, although the first coefficient converting means consists of VLD1100 and the run level coefficient converter 1120, it is good also as equipment which summarized both functions to one. Furthermore, as for the basic extended hierarchy separating means, the basic extended hierarchy separator 1130 corresponds, and, in the composition of
Although the merging device input means is not illustrated, it is arranged in the merging device 3010, 3010a, 3010b, 3010n, 3011, 3015 and 3016, or it is arranged besides each merging device , and is arranged anyway at the preceding of the demultiplexing unit 3100, 3100a, 3100b, 3101, 3102, 3105 and 3106. This merging device input means stores two or more streams temporarily, or carries out operation which reads the function of a processing program. Moreover, the merging device separating means is equivalent to the demultiplexing unit 3100, 3100a, 3100b, 3101, 3102, 3105, and 3106 in each figure, and the merging device merging means is equivalent to the merging unit 3300, 3300a, 3300b, 3301, 3302, 3305, and 3306. Moreover, although the merging device output means is not illustrated, it is arranged in the merging device , or it is arranged out of the merging device, and is arranged anyway at the rear of the merging unit. This merging device output means outputs for example, the third coded signal for every lump, or carries out operation which reads the function which performs to output.
For example, the basic coded signal converting means consists of coefficient information separating section 3450a, VLD basic 3310, the coefficient converter 3340, and the basic quantization coefficient merging section 3370 in
As described above, the coded signal separating and merging system according to the present invention achieves an advantage that the scalable transmission of images can be realized, and is useful as a server, transcoder, and the like for transmitting streams.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-381859 | Nov 2003 | JP | national |
| 2004-100419 | Mar 2004 | JP | national |
