The present disclosure relates to technology for carrying out byte alignment on bit streams that have been acquired by variable length encoding of a block coefficient sequence.
In recent years, editing processing of high quality video information has been carried out in various environments. Editing processing of video information includes, for example, recognition processing for video information during transmission, editing processing of a single video, and combination processing for a plurality of videos, etc. It has been envisaged to carry out this type of editing processing of video information not only on servers of data centers or individual personal computers, but also using mobile terminals owned by individuals, networked home electric appliances, audio-visual devices mounted in vehicles, routers within networks, etc. Here, editing processing of video information is normally carried out in units of macroblock (“MB”) groups (specifically, MB lines made up of MBs, independent MBs, groups of arbitrary MBs, etc.). Accordingly, editing processing for video information is carried out in units of video bit streams corresponding to MB groups.
On the other hand, video bit streams are generated by subjecting quantized block coefficient sequences to variable length encoding, in the case of H.264. Specifically, a video bit stream corresponding to an MB group is generated as encoded information in bit units by variable length encoding of a block coefficient sequence belonging to an MB group. It is known that a bit stream that has been generated in this way is not always segmented into bytes, even if video signal information that was the object of encoding (coefficient sequence) was byte segmented information.
In the case where a video bit stream has not been segmented into bytes, in data processing operations, called copy and seek, of the video bit stream, bit shift operation processing becomes necessary, and there is a problem with increased load on the device.
WO 2009/09367 describes a method of subjecting a bit stream that is in frame units to byte alignment using a slice header that is used with H.264/AVC. However, with the technology of WO 2009/09367, since bit quantity adjustment of a slice header that has the entire bit stream of a frame constituting an object is assumed, it is not possible to carry out byte alignment for arbitrary MB groups, resulting in lack of flexibility in use. Also, since there is an upper limit on the number of slice headers that can be inserted into a single frame, there is a problem in that carrying out byte alignment for MBs of arbitrary locations is difficult.
The present disclosure has been conceived in view of the above-described situation. One aspect of the present disclosure is to provide technology that is capable of avoiding shift operation processing in bit stream extraction processing, and suppressing processing load. A specific aspect of the present disclosure is to provide technology to adjust the bit quantity of a bit stream encoded in MB group units by adjusting coefficient sequences of blocks within MBs, and in this way carry out byte alignment of a bit stream created from arbitrary MB groups.
Means for solving the above-described problems can be described as in the following aspects.
Aspect 1
A byte alignment device, for carrying out byte alignment on bit streams that have been acquired by variable length encoding of a block coefficient sequence, comprising:
a base coefficient sequence insertion section, a desired increased bit quantity calculation section, an adjustment coefficient sequence detection section, and an adjustment coefficient sequence insertion section, wherein:
the base coefficient sequence insertion section is configured to replace adjustment coefficient sequences, that are high-frequency components within quantized block coefficient sequences, with predetermined base coefficient sequences,
the desired increased bit quantity calculation section is configured to calculate a bit quantity that is to be increased for byte alignment based on code amount of a bit stream that has been acquired by subjecting the block coefficient sequences, in which the adjustment coefficient sequences have been replaced with the base coefficient sequences, to variable length encoding,
the adjustment coefficient sequence detection section is configured to detect adjustment coefficient sequences that will replace the base coefficient sequences, based on the bit quantity that is to be increased, and
the adjustment coefficient sequence insertion section is configured to insert the adjustment coefficient sequences that have been detected into high-frequency components corresponding to the base coefficient sequences.
Aspect 2
The byte alignment device of aspect 1, further comprising a guard coefficient insertion section, wherein:
the guard coefficient insertion section is configured to replace coefficients, that are positioned one coefficient closer to a low frequency side than the high-frequency component coefficient sequence of the block coefficient sequences, with guard coefficients, and
the guard coefficients restrict influence on the code amount of the bit stream due to a value of a coefficient that is more to a low-frequency side than a guard coefficient.
Aspect 3
The byte alignment device of aspect 1 or aspect 2, wherein the adjustment coefficient sequence detection section is configured to detect the appropriate adjustment coefficient sequence based on factors that influence the code amount of the bit stream that has been subjected to variable length encoding, in addition to the bit quantity to be increased.
Aspect 4
The byte alignment device of any one of aspects 1-3, further comprising a search DB, and wherein the search DB is configured so that it is possible to detect the appropriate adjustment coefficient sequence based on the bit quantity to be increased and/or factors that influence the code amount of the bit stream that has been subjected to variable length encoding.
Aspect 5
A bit stream generating device, comprising the byte alignment device of any one of aspects 1-4 and a variable length encoding section, wherein:
the variable length encoding section, when subjecting coefficient sequences of adjacent blocks that are adjacent to object blocks that are the object of byte alignment by the byte alignment device to variable length encoding, is configured to carry out restriction of a prediction mode of the encoding of the object blocks, and/or fix coefficient values used in encoding.
Aspect 6
A bit stream generating method used in order to carry out byte alignment on bit streams that have been acquired by variable length encoding of a block coefficient sequence, comprising:
a step of replacing adjustment coefficient sequences, that are high-frequency components within quantized block coefficient sequences, with predetermined base coefficient sequences,
a step of calculating a bit quantity that is to be increased for byte alignment, based on code amount of a bit stream that has been acquired by subjecting the block coefficient sequences, in which the adjustment coefficient sequences have been replaced with base coefficient sequences, to variable length encoding,
a step of detecting appropriate adjustment coefficient sequences that will replace the base coefficient sequences, based on bit quantity that is to be increased, and
a step of inserting the detected adjustment coefficient sequences into high-frequency components corresponding to the base coefficient sequences.
Aspect 7
A computer program for causing execution of the method of aspect 6 on a computer.
Aspect 8
A data structure made up of block coefficient sequences, wherein:
high-frequency component coefficients of the block coefficient sequences are made adjustment coefficient sequences for carrying out byte alignment of a bit stream that has been subjected to variable length encoding,
guard coefficients are inserted into coefficients that are one coefficient closer to the low-frequency side than the adjustment coefficient sequences, and
the guard coefficients suppress influence on variable length encoding conditions due to values of low-frequency component coefficients that are closer to the low-frequency side than this guard coefficient.
Regarding the computer program and/or the data structure described above, they can be utilized on a computer by being stored in an appropriate non-transitory storage medium such as, for example, an electrical, magnetic, or optical medium. Also, this storage medium can be utilized by the Internet; for example, it may be a storage medium on a cloud computing system.
According to the present disclosure, it is possible to provide technology to carry out byte alignment on a bit stream that has been created from arbitrary MB groups.
The foregoing aspects and many of the attendant advantages of this disclosure will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
An encoding system of a first embodiment of the present disclosure will be described in the following with reference to the attached drawings.
First, the overall schematic structure of an encoding device that uses the encoding system of this embodiment will be described with reference to
This system comprises an integer precision conversion section 1, a quantization section 2, a byte alignment processing section 3, and a variable length encoding section 4. Each element will be described below.
Integer Precision Conversion Section
The integer precision conversion section 1 is input with a block coefficient sequence (in the case of intra frame encoding) or a block coefficient sequence residual signal (in the case of inter frame encoding). In this specification, both a block coefficient sequence and a residual signal for a block coefficient sequence will be referred to as a block coefficient sequence. The integer precision conversion section 1 converts real precision input values to integer precision, and since this can be configured using the same implementation as with conventional H.264, a detailed description will be omitted.
Quantization Section
The quantization section 2 is a functional element for subjecting a block coefficient sequence that has been subjected to integer precision conversion to quantization. This quantization section 2 can also be configured with the same implementation as for conventional H.264, so a detailed description is omitted.
Byte Alignment Processing Section
The byte alignment processing section 3 is a functional element corresponding to one embodiment of a byte alignment device of the present disclosure.
The byte alignment processing section 3 of this embodiment comprises a base coefficient sequence insertion section 31, a desired increased bit quantity calculation section 32, an adjustment coefficient sequence detection section 33, an adjustment coefficient sequence insertion section 34, a guard coefficient insertion section 35, and a search DB 36.
The base coefficient sequence insertion section 31 is configured to replace coefficient sequences for adjustment (hereafter “TailCoeff”), which are high-frequency components within quantized block coefficient sequences, with predetermined base coefficient sequences.
The desired increased bit quantity calculation section 32 is configured to calculate a bit quantity that is to be increased for byte alignment, based on code amount of a bit stream that has been acquired by subjecting the block coefficient sequences, in which the coefficient sequences for adjustment have been replaced with base coefficient sequences, to variable length encoding.
The adjustment coefficient sequence detection section 33 is configured to detect appropriate adjustment coefficient sequences that will replace base coefficient sequences (high-frequency component coefficient sequences), based on the bit quantity that is to be increased. Further, the adjustment coefficient sequence detection section 33 of this embodiment is configured to detect appropriate adjustment coefficient sequences based on factors (that will be described later) that influence code amounts of bit streams that have been subjected to variable length encoding, in addition to the bit quantity that is to be increased.
The adjustment coefficient sequence insertion section 34 is configured to insert the adjustment coefficient sequences that have been detected into high-frequency components corresponding to the base coefficient sequences.
The guard coefficient insertion section 35 is configured to replace coefficients that are one position closer to the lower frequency side than a high-frequency component coefficient sequence of a block coefficient sequence with guard coefficients (hereafter “GuardCoeff”). A fixed coefficient value (with this example, a coefficient of an absolute value of 2 or greater) is inserted into a GuardCoeff. Here, the guard coefficients restrict influence on code amount of the bit stream due to a value of a coefficient that is more to a low-frequency side than a guard coefficient (described later). Position of the guard coefficients shall be already known at the time of insertion of the guard coefficients. Search of guard coefficient position will be described later.
The search DB 36 is configured so that it is possible to detect the appropriate adjustment coefficient sequence based on the bit quantity to be increased and/or factors that influence code amount of the bit stream that has been subjected to variable length encoding. The search DB 36 of this embodiment is specifically a table structure (refer to
The detailed structure of the byte alignment processing section 3 of this embodiment will be described later as operation of this embodiment.
Variable Length Encoding Section
The variable length encoding section 4 is a functional element for carrying out variable length encoding on the coefficient sequences of each block.
The variable length encoding section 4 of this embodiment is “configured to carry out restriction of prediction modes for encoding of object blocks and/or fixing of coefficient values used in encoding, when carrying out variable length encoding of coefficient sequences of adjacent blocks that are adjacent to object blocks constituting the object of byte alignment processing by the byte alignment device.” Apart from this point, the variable length encoding section 4 of this embodiment can be configured in the same manner as for variable length encoding using CAVLC (Context Adaptive Variable Length Coding) of H.264, for example.
The detailed structure of the variable length encoding section 4 of this embodiment will be described later as the operation of this embodiment.
The bit stream generating procedure of this embodiment will be described in the following with further reference to
Steps SA-1 to 3 in
First, input of a video signal before encoding is received, and a coefficient sequence of that video signal, is converted with integer precision. Further, the coefficient sequence that has been converted with integer precision is subjected to quantization. Processing up to that point may be the same as for normal H.264, so detailed description is omitted.
Steps SA-4 to 5 in
Next, a procedure for byte alignment processing by the byte alignment processing section 3 of this embodiment will be described with further reference to
Step SB-0 in
The variable length encoding section 4 carries out variable length encoding for “each block, except for blocks that were subjected to adjustment of coefficient sequence (object blocks).” From a total number of bits of bit streams obtained as a result, it is possible to acquire bit quantity for bit streams of MB lines except for the object blocks. It is possible to acquire code amount for bit streams of all the MB line by adding this bit quantity and code amount of bit streams due to encoding of object block coefficient sequences. This code amount acts as a benchmark for byte alignment processing.
Step SB-1 in
First, the byte alignment processing section 3 acquires block coefficient sequences in the object block that have been subjected to quantization processing from the quantization section 2. Here, an example of a block coefficient sequence will be supplementarily described with reference to
Step SB-2 in
Next, the base coefficient sequence insertion section 31 substitutes a base coefficient sequence that has been prepared in advance (hereafter sometimes referred to as “BaseCoeff”) for the previously described TailCoeff. With this embodiment, a value of (0, 0, 0, 0) is used as the base coefficient sequence. This is in order, in CAVLC, to be able to minimize encoded bit quantity when using a coefficient sequence of (0, 0, 0, 0).
Step SB-3 in
Next, it is confirmed whether or not encoding of the block coefficient sequence is the first after the initial TailCoeff has been replaced. Specifically, variable length encoding of the block coefficient sequence is carried out twice for byte alignment. The first time a coefficient sequence having a TailCoeff replaced with a base coefficient sequence is subjected to variable length encoding, and the second time a coefficient sequence having a TailCoeff replaced with an adjustment coefficient sequence is subjected to variable length encoding. If the determination here is Yes, processing advances to step SB-4.
Steps SB-4 and SB-5 in
Next, a TailCoeff is replaced with a BaseCoeff, and variable length encoding is carried out by the variable length encoding section 4 using the coefficient sequence after replacement.
Steps SB-6 and SB-7 in
Next, the desired increased bit quantity calculation section 32 calculates an X modulo 8 residual amount, namely, X mod 8, with a code amount that has been obtained by summing code amount of an object block obtained by variable length encoding and a code amount acquired in step SB-0 (namely, code amount for all MB lines) as X. A difference between this residual amount and 8 is then calculated. The result is the desired increase amount for the number of bits in the bit stream.
Step SB-8 in
Next, an adjustment coefficient sequence that should be inserted into the TailCoeff is searched for and acquired from a search table 36, using the acquired desired increase amount and other coefficient characteristics. Specifically, in the search table 36, it is possible to search for an appropriate adjustment coefficient sequence based on respective factors that affect increased bit quantity (in this example, the desired increases bit quantity, LeftPartTotalCoeff and NeighborTotalCoeff). This search will later be further described.
The position of a guard coefficient (namely, the GuardIndex) is acquired at the time of generating the search table shown in
Terminology used in the description of the coefficient sequence is shown in Table 1 below.
Also, parameters constituting selective factors of the encoding table for variable length encoding, and measures taken in this embodiment, are shown in Table 2 below.
Steps SB-3, SB-9, and SB-5 in
Next, the procedure returns once more to step SB-3, and after that advances to step SB-9. Further, the adjustment coefficient sequence insertion section 34 inserts an adjustment coefficient sequence that has been obtained from the search table into the TailCoeff. Variable length encoding is carried out on the block coefficient sequence that has had the adjustment coefficient sequence inserted, and the obtained bit stream is output. By constructing a bit stream of the MB line using the bit stream that has been acquired in this way, it is possible to implement byte alignment.
With this embodiment, as has been described, it is possible to realize byte alignment with a simple procedure. It is also possible to execute processing for the previously described byte alignment with blocks belonging to an arbitrary MB as the object.
Also, with this embodiment, since a guard coefficient has been inserted into the block coefficient sequence, there is the advantage that it is possible to suppress any effect on encoding conditions, even if the TailCoeff is replaced with an adjustment coefficient sequence. However, if CAVLC is assumed, the value of a guard coefficient becomes 2 or 3. This point will be described later.
Here, the advantage of guard coefficient insertion will be supplementarily described.
At the time of actual variable length encoding, since a CoeffToken, which is a combination of TotalCoeff and TrailingOnes, is also encoded, that code amount affects the code amount of the overall bit stream. As a result of guard coefficient insertion, it is possible to suppress the effect on code amount of the CoeffToken parameter caused by change to the TailCoeff. However, since the encoding table for CoeffToken is determined using an average value for the number of nonzero coefficients of adjacent blocks (NeighborTotalCoeff), the code amount for CoeffToken will differ depending on NeighborTotalCoeff even if all coefficient sequences are the same. Accordingly, even in the event that a TailCoeff exists for which a particular bit quantity increase has been executed for a particular LeftPartCoeff, if NeighborTotalCoeff is different, it is not possible to execute a desired bit quantity increase even with the same TailCoeff. For the above-mentioned reasons, as long as NeighborTotalCoeff is the same (the table for encoding CoeffToken is the same), then using the above parameter settings, it becomes possible to handle LeftPartCoeff and TailCoeff independently, and with coefficient sequence adjustment of the TailCoeff, it becomes possible to control and increase the bit amount between 1 to 7 without affecting encoding of coefficient sequences other than this (LeftPartCoeff, GuardCoeff).
Step SA-6 in
Next, a bit stream that has been generated by variable length encoding of a block coefficient sequence that has been subjected to byte alignment processing is output. These processes themselves can be the same as conventional processes, so detailed description is omitted.
Step SA-7 in
Next, a procedure for generating a search table for searching for adjustment coefficient sequences will be described with reference to
Step SC-1 in
First, as a premise, a GuardCoeff value is set to 2 or 3, and an absolute value A used in the following is made 3 or less. Also, the position of the guard coefficient (GuardIndex) is an arbitrary value in this step, but with this example, a GuardIndex that can handle a bit quantity increase or decrease value between 1-7 bits (Index 11 in this example) is searched for by sequentially examining from a high-frequency side value (Index 16 with this example). The reasons for these settings will be additionally described below.
Setting of a value of absolute value A for numerical domain of TailCoeff to 3 or less:
(Reason) Since an encoding table used in Level of LeftPartCoeff varies in accordance with the magnitude of the absolute value of TailCoeff, this variation can be suppressed by this parameter. Since the encoding table for Level varies with the magnitude of absolute value of the level value set to a threshold of 0, 3, 6, 12, 24, 48, then in order to set the encoding table used for Level of the LeftPartCoeff independently of TailCoeff adjustment, it is necessary to make magnitudes of all absolute values of coefficient values of TailCoeff 3 or less, among the previously described thresholds.
Setting of a value of GuardCoeff to a coefficient of an absolute value of 2 or greater:
(Reason) With this setting code, amounts for parameters TrailingOnes, TrailingOnesFlag, and RunBefore can be made independent in LeftPartCoeff and TailCoeff. In the case of an absolute value 0 or 1, there is a possibility of a coefficient of LeftPartCoeff also being made an object at the time of determining TrailingOnes, TrailingOnesFlag, and RunBefore. This can be restricted by placing a coefficient of an absolute value of 2 or greater in the GuardCoeff.
The fact that GuardIndex value is 11 (case where a lowest frequency coefficient index is made 1):
(Reason) This parameter is obtained in association with generation of a search table that will be described below. The detail is as shown in
Also, the FixedTailTotalCoeff of this example is set to 2.
In the following, LeftPartTotalCoeff (refer to
Steps SC-2 and SC-3 of
For all cases where LeftPartTotalCoeff value is “from 0 to a maximum value (depending on the GuardIndex position) that LeftPartTotalCoeff can take,” in order to carry out generation of an adjustment coefficient sequence, in the event that LeftPartTotalCoeff is smaller than GuardIndex, LeftPartCoeff is appropriately set such that the number of nonzero coefficients becomes LeftPartTotalCoeff, and a predetermined CAVLC table index is set to 0. The maximum value for this index is 3 with CAVLC.
Steps SC-4 and SC-5 of
When the CAVLC table index is smaller than 4, BaseCoeff is set for TailCoeff, and variable length encoding is carried out. Then, a bit quantity after this encoding is made BaseCodeSize.
Step SC-6 of
For all combinations where integer values acting as absolute value A constitute respective elements of TailCoeff, block coefficient sequences are subjected to variable length encoding. Specifically, variable length encoding is carried out for all combinations from (−3, −3, −3, −3) to (3, 3, 3, 3), and a code amount at that time is made ArrangeCodeSize. A value of (ArrangeCodeSize−BaseCodeSize) mod 8 is made IncreaseCodeSize.
Step SC-7 in
Next, a group made up of IncreaseCodeSize and TailCoeff is output.
Step SC-8 of
It is determined whether TailCoeff exists in which TailTotalCoeff is the same as FixedTailTotalCoeff, and all of 1 to 7 bits of IncreaseCodeSize are satisfied. If the determination is Yes, processing advances to step SC-9. With this example, the value of FixedTailTotalCoeff is fixed to 2. That is, adjustment coefficient sequences with a number of nonzero coefficients of other than 2 are canceled. Here, fixing TailTotalCoeff to FixedTailTotalCoeff is performed for the following reason. Specifically, a table for variable length encoding (VLC table) changes depending on TailCoeff (also depending on other factors). TotalCoeff is the sum of LeftPartTotalCoeff, GuardTotalCoeff, and TailTotalCoeff, and GuardTotalCoeff becomes 1 in this example. Accordingly, by fixing TailTotalCoeff, all that has to be taken into account is the effect of LeftPartTotalCoeff on TotalCoeff. LeftPartTotalCoeff serves as an index of the search table.
Step SC-9 in
Next, if there are a plurality of groups made up of IncreaseCodeSize and TailCoeff (adjustment coefficient sequence candidates), for respective 1-7 bits of IncreaseCodeSize, a single adjustment coefficient sequence is selected based on the following criteria, taking into consideration influence on image quality degradation. Adopting these criteria is done in order to reduce visual effects as much as possible:
Step SC-10 of
Next, groups made up of LeftPartTotalCoeff, VLCTableIndex, IncreaseCodeSize, and TailCoeff are registered in a search table.
Step SC-11 in
Next, 1 is added to the VLC TableIndex, and processing returns to step SC-4.
Step SC-12 of
If the decision in step SC-8 is No, the search table is nullified since conditions required to generate a TailCoeff for which adjustment of all of 1-7 bits is possible are not satisfied for any input combinations, and any of GuardIndex or FixedTailTotalCoeff are set again. In the case of setting GuardIndex again, with this embodiment, the value of GuardIndex is reduced by one. By repeating this operation, it is possible to search for an appropriate value of GuardIndex.
Step SC-13 in
If the decision in step SC-4 is No, 1 is added to LeftPartTotalCoeff, and the operation returns to step SC-2. By repeating this operation, it is possible to generate corresponding search tables for all LeftPartTotalCoeff.
With the above-described processing, it is possible to create the search table shown in
Next, a byte alignment procedure of a second embodiment will be described with further reference to
With the previously described first embodiment, a block for carrying out byte alignment (namely, an object block) is a final block within an MB (refer to
However, conversely, in the event that a block other than the final block is made the object of byte alignment, since there are blocks that reference this block, it is necessary to restrict or fix reference information. This restriction or fixing is only required in the event that the encoding mode for these blocks is intra 4×4.
In the following description, it is assumed that the block b11 shown in
b12 prediction mode: intra—4×4_Vertical,
b15 prediction mode: intra—4×4_Horizontal, and
b16 prediction mode: prediction mode other than the following:
LeftPartTotalCoeff+1+TailTotalCoeff.
The present invention is not limited to the above-described embodiments, and various modifications can additionally be obtained within a scope that does not depart from the spirit of the invention.
For example, each of the above-described structural elements can exist as a functional block, and can also exist as independent hardware. Also, as a method of implementation, it is possible to use hardware or use computer software. Further, a single functional element of the present invention may be realized as a set of a plurality of functional elements, and a plurality of functional elements of the present invention may be implemented by a single functional element.
It is also possible for each functional element constituting the present invention to exist separately. In the case of existing separately, necessary data can be exchanged by means of a network, for example. Similarly, it is also possible for each function of an internal part of each section to exist separately. For example, it is possible to implement each functional element, or some of the functional elements, of this embodiment using grid computing or cloud computing.
Number | Date | Country | Kind |
---|---|---|---|
2011-236672 | Oct 2011 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2012/076815 | 10/17/2012 | WO | 00 | 4/28/2014 |