The present application is based on and claims priority of Japanese Patent Application No. 2014-019044 filed on Feb. 4, 2014 and Japanese Patent Application No. 2014-221646 filed on Oct. 30, 2014. The entire disclosure of the above-identified application, including the specification, drawings and claims is incorporated herein by reference in its entirety.
The present disclosure relates to a video coding apparatus and a video coding method.
In the High efficiency video coding (HEVC) that is an international video coding standard, coding is performed for each coding unit (CU) as a unit of coding. In the conventional H.264 etc., 16 pixels high and 16 pixels wide are used as a macroblock size. On the other hand, four kinds of sizes can be used as CU sizes. In addition, various sizes can be selected for prediction units (PUs) used for intra prediction in CUs and transform units (TUs) which are units of orthogonal transform (for example, see Non-patent Literature 1).
[NPL 1]
Such video coding apparatuses are desired to be modified to reduce complexity in control processing and prevent circuit scales from being increased.
The present disclosure provides a video coding apparatus and a video coding method for reducing complexity in control processing and preventing circuit scales from being increased.
A video coding apparatus in the present disclosure is a video coding apparatus which codes a coding target video based on a coding standard, and includes: a dividing unit configured to divide an image included in the coding target video into a plurality of control blocks; a prediction image generating unit configured to divide each of the control blocks into a plurality of prediction blocks, and generate, for each of the prediction blocks, a prediction image using one of inter prediction and intra prediction; a subtracter which generates a residual component by calculating, for each of the prediction blocks, a difference between the prediction image and the image included in the coding target video; and a signal processing unit configured to generate a compressed video by performing orthogonal transform, quantization, and variable length coding on the residual component, wherein the coding standard allows each of the control blocks to be divided into the prediction blocks such that the prediction blocks have different sizes, and the prediction image generating unit is configured to divide each of the control blocks into the prediction blocks such that the prediction blocks all have an identical size, rather than dividing each of the control blocks into the prediction blocks such that the prediction blocks have different sizes.
The video coding apparatus in the present disclosure is capable of reducing complexity in control processing and preventing circuit scales from being increased.
These and other objects, advantages and features of the disclosure will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the present disclosure.
Hereinafter, an embodiment is described referring to the drawings as necessary. It should be noted that unnecessarily detailed descriptions may be omitted below. For example, detailed descriptions about already well-known matters and overlapping descriptions for substantially identical configurations may be omitted. Such descriptions are omitted to prevent the descriptions below from being unnecessarily redundant and help a person skilled in the art to understand the present disclosure easily.
It should be noted that the inventors provide the attached drawings and descriptions below to allow the person skilled in the art to fully understand the present disclosure, and do not intend to restrict the subject matters of the Claims by the disclosure thereof.
Hereinafter, an embodiment is described with reference to
[Processing of Entire Coding Apparatus]
The video coding apparatus 100 includes: a picture memory 101; a block dividing unit 102; a block size determining unit 103; a residual coefficient encoder 104; a residual coefficient decoder 105; a picture buffer 106, an inter prediction unit 107; an intra prediction unit 108; a switch 109; a coded bitstream generating unit 110; an adder 111; and a subtracter 112.
First, the picture memory 101 obtains an input image (S101). More specifically, the picture memory 101 rearranges, in coding order, the input images input on a picture basis in display order. In response to a read instruction from the block dividing unit 102, the picture memory 101 outputs one of the input images which is specified by the read instruction to the block dividing unit 102 and the intra prediction unit 108.
Next, the block dividing unit 102 divides the input image (picture) into a plurality of control blocks (S102). More specifically, the block dividing unit 102 outputs pixels of a coding target picture, for each block to be output. The size of the output block is one of a plurality of block sizes which are preset.
More specifically, the block dividing unit 102 divides an input image input from the picture memory 101 into blocks called coding units (CUs) which are units of coding.
It is to be noted that the blocks that are output by the block dividing unit 102 may be a single CU, or may be a coding tree unit (CTU) made up of a plurality of CUs. In this embodiment, a CU or a CTU corresponds to a control block.
As illustrated in
In addition, the subsequent processes are performed on the basis of the CU or the CTU including the plurality of CUs which have been output by the block dividing unit 102. For the purpose of illustration, a configuration for performing processes on a CU basis is described below.
Next, the inter prediction unit 107 sets, for a CU, a block (a prediction unit (PU)) for prediction having a block size smaller than or equal to the size of the CU. More specifically, the inter prediction unit 107 sets the sizes of all of the PUs in a control block (CU or CTU) to an identical size (S103). This processing is described in detail later. Next, the inter prediction unit 107 generates a prediction image for each PU that has been set (S104).
More specifically, the inter prediction unit 107 performs inter prediction using a reconstructed image of a preceding picture that has already been coded and stored in the picture buffer 106, based on the CU input from the block dividing unit 102. More specifically, the inter prediction unit 107 searches for (by performing motion search) a reconstructed image area having a pixel configuration most similar to the pixel configuration of an input image, to generate (performs motion compensation on) the searched-out reconstructed image area as a prediction image. At this time, the inter prediction unit 107 performs motion compensation on the basis of a block (PU) obtained by further dividing a CU block.
As illustrated in
For each of CUs output by the block dividing unit 102, the intra prediction unit 108 obtains already coded pixels which are pixels neighboring the CU from the picture buffer 106. The intra prediction unit 108 generates an intra prediction image using the obtained pixels. It should be noted that the size identical to the size of the CU is basically used as the size of the prediction block (PU) for intra prediction. As an exception, only when the CU size is the minimum one (8×8 pixels), it may be allowed to use a prediction block (8×4 pixels or 4×8 pixels) obtained through horizontal or vertical division into two blocks.
The switch 109 selects a prediction image having a higher coding efficiency from among prediction images which have been generated by the inter prediction unit 107 or the intra prediction unit 108, and outputs the selected prediction image. More specifically, the switch 109 selects, from among the intra prediction and the inter prediction, the prediction method that yields the smaller sum of the amount of information required to code a prediction residual and the amount of side information such as a motion vector etc.
The subtracter 112 generates a difference image having pixel values presenting differences between the CU-based input image input from the block dividing unit 102 and the CU-based prediction image input from the switch 109 (S105). The subtracter 112 outputs the generated difference image to the block size determining unit 103.
The block size determining unit 103 determines the optimum block size in view of the coding efficiencies from among a plurality of orthogonal transform block sizes (TU sizes) illustrated in
The residual coefficient encoder 104 generates a residual coefficient by performing orthogonal transform and quantization for each of the TUs which has been set (S107).
More specifically, the residual coefficient encoder 104 generates orthogonal transform coefficients by performing orthogonal transform on the difference image generated in the subtracter 112 using, as a unit of processing, the TU having the block size determined in the block size determining unit 103. Furthermore, the residual coefficient encoder 104 generates residual coefficients by performing quantization on frequency components of the resulting orthogonal transform coefficients.
The residual coefficient decoder 105 generates the orthogonal transform coefficients by performing inverse quantization on the residual coefficients input from the residual coefficient encoder 104 using, as the unit of processing, the TU having the block size determined in the block size determining unit 103. Furthermore, the residual coefficient decoder 105 generates a reconstructed difference image by performing inverse orthogonal transform on the resulting orthogonal transform coefficients (S108).
The adder 111 generates the reconstructed image by adding the reconstructed difference image input from the residual coefficient decoder 105 and the prediction image input from the switch 109 on a CU basis (S109).
The picture buffer 106 stores the reconstructed image input from the adder 111. This reconstructed image is referred to in inter prediction of a picture to be coded temporally after a current coding target picture. In addition, this reconstructed image is referred to in intra prediction of a CU to be coded after in coding order than the current coding target CU.
The coded bitstream generating unit 110 generates a coded bitstream by performing variable length coding and arithmetic coding on the residual coefficients input from the residual coefficient encoder 104 and coding information to be required in decoding (S110). It is to be noted that the coded bitstream generating unit 110 may perform the processes at any time after the residual coefficients are generated (after Step S107).
In addition, whether or not the processing of all control blocks (CUs) has been completed is determined (S111). If the processing has not yet been completed (No in S111), the processes in Steps S103 and S110 are repeated on a control block basis.
Hereinafter, details of operations performed by the block size determining unit 103, the inter prediction unit 107, and the intra prediction unit 108 are described with reference to the drawings.
It is to be noted that the instruction 401 in
The vertical axis of
The horizontal axis of
[Details of Operation by Block Size Determining Unit 103]
For example, when the optimum TU size is determined considering coding efficiencies, the block size determining unit 103 sets the sizes of all of the TUs in a CTU to be output by the block dividing unit 102 to an identical size.
In the HEVC standard, several TU sizes can be selected based on
This operation is described using
In the HEVC standard, three kinds of TU sizes are allowed as illustrated in the case of 16×16 CU size in
On the other hand, the block size determining unit 103 in this embodiment sets, for each of CTUs to be output by the block dividing unit 102, the TU sizes of all of the CUs of the CTU to an identical size. More specifically, the block size determining unit 103 uses only configurations in which the sizes of TUs in all of the CUs are identical, as candidate TU sizes for the CUs of each CTU. Subsequently, the inter prediction unit 107 determines the optimum TU size considering coding efficiencies from among the candidate TU sizes. For example, when the TU size is 8×8, the TU size in all of the CUs is set to the identical size as illustrated in (b) of
It is to be noted that the block size determining unit 103 may determine a TU size for a single CU from among the CUs of a CTU, and apply the determined TU size to another CU.
With the configuration, the TU size used in orthogonal transform in the residual coefficient encoder 104 and the residual coefficient decoder 105 is set to be identical. As a result, variation in processing time required for orthogonal transform in the processing unit is reduced. In this way, as illustrated in (a) of
In addition, also in the case where a circuit for performing orthogonal transform is configured using a reconfigurable device, the identical TU size is used in the CTU. Thus, it is possible to reduce the number of times of occurrence of reconstruction. As a result, the disturbance in pipeline processing caused by time for reconstruction is reduced in the identical CTU. In addition, it is possible to reduce increase in power consumption for reconstruction.
[Operations Performed by Inter Prediction Unit 107 and Intra Prediction Unit 108]
The inter prediction unit 107 in this embodiment sets a PU size in a CU as indicated below. In other words, in this embodiment, the inter prediction unit 107 sets the sizes of all PUs belonging to the CU to an identical size. In addition, as for a plurality of CUs of a CTU to be output by the block dividing unit 102, the inter prediction unit 107 sets the sizes of all PUs of the plurality of CUs to an identical size.
In this case, for example, the operation is performed by restricting the PU sizes used when the inter prediction unit 107 performs motion search. Hereinafter, the above operation is described using
In
In the case of motion search in the HEVC standard, eight kinds of PU sizes are allowed as illustrated in the case of 16×16 CU size in
On the other hand, the inter prediction unit 107 in this embodiment, motion search is performed to set, for each of CTUs to be output by the block dividing unit 102, the PU sizes of all CUs of the CTU to an identical size. More specifically, the inter prediction unit 107 employs only the configurations in which the sizes of PUs in all of the CUs are identical, as candidate PU sizes for the CUs of the CTU. Subsequently, the inter prediction unit 107 selects the optimum one of the results of motion search performed for each of candidate PU sizes, and generates a prediction image using the selected optimum result of motion search. For example, when the PU size is 16×8, the PU size in all of the CUs is set to the identical size as illustrated in (b) of
The inter prediction unit 107 may determine a PU size for a single CU from among the CUs of a CTU, and apply the determined PU size to the other CUs.
When an output from the block dividing unit 102 is a single CU of 16 pixels high and 16 pixels wide, the inter prediction unit 107 sets the sizes of all PUs in the CU to an identical size. For example, the inter prediction unit 107 performs motion search regarding, as candidates, four configurations illustrated in (a) to (d) of
Normally in intra prediction, CU sizes and PU sizes which are blocks for prediction are basically identical. For this reason, even when the block dividing unit 102 outputs a CTU including a plurality of CUs, the PU sizes of the plurality of CUs are all identical. However, one of the PU sizes obtained by horizontally or vertically dividing a CU into two blocks may be used only when the CU size is the minimum size. In this case, the intra prediction unit 108 operates to apply, to all of the CUs, either a configuration in which a CU is divided into two PUs for use in intra prediction or a configuration in which a CU is not divided into two PUs for use in intra prediction.
With the configuration stated above, the video coding apparatus 100 according to this embodiment sets the PU sizes of all of the CUs of the CTU to the identical size. This results in reduction of variation in processing time for each CU in either the inter prediction unit 107 or the intra prediction unit 108. In this way, as illustrated in (a) of
[Modifications to Prediction Operations]
Hereinafter, modifications to prediction operations in the inter prediction unit 107 and the intra prediction unit 108 are described with reference to the drawings.
When the block dividing unit 102 outputs a CTU including a plurality of CUs, the switch 109 outputs a CTU having a corresponding size. For this reason, the inter prediction unit 107 and the intra prediction unit 108 which are sources of inputs to the switch 109 also output a corresponding CTU that is a plurality of CUs. In this case, the switch 109 selects an identical one of prediction methods which are intra prediction and inter prediction, for every one of CUs in each of CTUs, as illustrated in (a) and (d) of
Here, advantages of switching intra prediction and inter prediction on a CTU basis as described above are described with reference to
In (a) of
On the other hand, in the case of (b) of
Likewise, in the case of (c) of
In this way, pipeline operations and the number of pipeline stages vary depending on (i) whether or not intra prediction and inter prediction are performed in an identical CTU, and (ii) the number of times of intra prediction. In order to perform pipeline operations in all of such cases, extremely complex circuit configurations are required. On the other hand, the switch 109 according to this embodiment uses only one kind of prediction method in a CTU. In this case, the above disadvantages can be cleared.
In addition, in order to enable prediction using different PU sizes, it is conceivable to configure the inter prediction unit 107 or the intra prediction unit 108 using a reconfigurable device. In this case, according to this embodiment, since a configuration in which the sizes of all PUs are identical in a CTU is used, it is possible to reduce the number of times of occurrence of reconstruction processing in a device in processing time for the CTU. In this way, disturbance in pipeline processing is reduced. Furthermore, it is also possible to reduce increase in power consumption for frequent reconstruction.
The video coding apparatus 100 according to this embodiment is an apparatus which codes a coding target video, based on a coding standard, and includes: a picture memory 101 which stores a video; a block dividing unit 102 which divides an image included in the video into a plurality of control blocks (CUs or CTUs); an inter prediction unit 107 or an intra prediction unit 108 which divides each of the control blocks into a plurality of prediction blocks (PUs), and generates, for each prediction block, a prediction image using a corresponding one of inter prediction and intra prediction; a subtracter 112 which generates a residual component by calculating a difference between the generated prediction image and the image included in the video for each prediction block; a block size determining unit 103 which generates a compressed video by performing orthogonal transform, quantization, and variable length coding on the residual component; a residual coefficient encoder 104; and a coded bitstream generating unit 110. In the predetermined coding standard, it is allowed to divide each of the control blocks into the prediction blocks such that the prediction blocks have different sizes. However, it is allowed to set the sizes of the prediction blocks of the control block to different sizes. Either the inter prediction unit 107 or the intra prediction unit 108 divides each control block into the prediction blocks such that the prediction blocks all have an identical size, rather than dividing each control block into the prediction blocks such that the prediction blocks have different sizes. In other words, either the inter prediction unit 107 or the intra prediction unit 108 sets the sizes of all of the prediction blocks of the control block to an identical size, rather than setting the sizes thereof to different sizes. Stated differently, either the inter prediction unit 107 or the intra prediction unit 108 prohibits the sizes of all of the prediction blocks of the control block from being set to different sizes. In short, either the inter prediction unit 107 or the intra prediction unit 108 always sets the sizes of all of the prediction blocks of the control block to an identical size.
For example, in the HEVC, various sizes can be selected as sizes of prediction blocks, and a coding efficiency is increased. On the other hand, in order to realize hardware which can support processes on all of the block sizes illustrated in
However, in the pipeline configuration, in order to cause the processing elements to function efficiently, there is a need to reduce waiting time which occurs due to variation in the processing time by the processing elements or a dependency of data. In addition, when such variation requiring waiting time cannot be prevented, for example, there is a need to design a circuit considering either increase in circuit scale due to addition of a circuit for exception processing or a margin for variation.
In view of this, with the configuration, PU sizes used when generating a prediction image are all set to be identical in the control block. This results in reduction of variation in processing time for each CU in either the inter prediction unit 107 or the intra prediction unit 108. In this way, as illustrated in (a) of
In addition, the inter prediction unit 107 for inter prediction and the intra prediction unit 108 for intra prediction in this embodiment are switched, for each CU or CTU, to generate a prediction image using a corresponding one of inter prediction and intra prediction for all of the PUs in a CU or a CTU. The coding standard allows a combination of inter prediction and intra prediction to be used for the prediction blocks (PUs) obtained through the division of a given one (a CU or CTU) of the control blocks. Either the inter prediction unit 107 for inter prediction or the intra prediction unit 108 for intra prediction generates the prediction image using the corresponding one of inter prediction and intra prediction for every one of the prediction blocks obtained through the division of the given one of the control blocks, without using the combination of inter prediction and intra prediction for the prediction blocks obtained through the division of the given one of the control blocks. In other words, the inter prediction unit 107 and the intra prediction unit 108 prohibit different prediction methods from being used for the prediction blocks in the control block. Stated differently, each of the inter prediction unit 107 and the intra prediction unit 108 always uses an identical prediction method that is the corresponding one of inter prediction and intra prediction for all of the prediction blocks in the control block.
In this way, as illustrated in (a) of
In addition, the block size determining unit 103 divides each control block (a CU or CTU) into a plurality of orthogonal transform blocks (TUs), and performs orthogonal transform on the residual component for each orthogonal transform block. In the coding standard, it is allowed to divide each of the control blocks into the orthogonal transform blocks such that the orthogonal transform blocks have different sizes. The block size determining unit 103 divides each control block into the orthogonal transform blocks such that the orthogonal transform blocks all have the identical size without dividing each control block into the orthogonal transform blocks such that the orthogonal transform blocks have different sizes. In other words, although, in the coding standard, it is allowed to set the sizes of the orthogonal transform blocks in the control block to different sizes, the block size determining unit 103 sets the sizes of all of the orthogonal transform blocks in the control block to the identical size, rather than setting the sizes thereof to different sizes. Stated differently, the block size determining unit 103 prohibits the sizes of the orthogonal transform blocks in the control block to different sizes. In other words, the block size determining unit 103 always sets the sizes of the orthogonal transform blocks in the control block to the identical size.
In this way, the residual coefficient encoder 104 and the residual coefficient decoder 105 reduce variation in processing time on CUs.
In addition, the size of each control block (CU) is 16 pixels high and 16 pixels wide. In this way, it is possible to reduce modifications from video coding apparatuses conforming to the conventional H.264 etc.
These general and specific aspects may be implemented using an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of apparatuses, systems, methods, integrated circuits, computer programs, or computer-readable recording media.
Each of the constituent elements in the above-described embodiment may be configured in the form of an exclusive hardware product, or may be realized by executing a software program suitable for the constituent element. Each of the constituent elements may be realized by means of a program executing unit, such as a CPU and a processor, reading and executing the software program recorded on a recording medium such as a hard disk or a semiconductor memory.
In addition, all of the numerals above are used for the purpose of specifically explaining the present disclosure, and thus the present disclosure is not limited to the exemplary numerals.
In addition, functional blocks in the block diagram are divided as an example. Thus, some of the functional blocks may be integrated into a single functional block, a single functional block may be divided into a plurality of functional blocks, or part of functions of a functional block may be transferred to another one of the functional blocks. Furthermore, similar functions of some of the functional blocks may be processed by hardware or software in parallel or in time division.
In addition, the execution order of a plurality of steps illustrated in the flowchart is provided for the purpose of specifically explaining the present disclosure, and thus other execution orders are possible. Some of the steps may be executed at the same time (in parallel) with any of the other steps.
Although only an exemplary embodiment of the present disclosure has been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiment without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of the present disclosure.
The video coding apparatus according to the present disclosure is applicable to apparatuses for broadcasting services, consumer recorders, digital video cameras, personal computers, mobile phones, etc. used to store and transmit coded streams generated by coding video signals.
Number | Date | Country | Kind |
---|---|---|---|
2014-019044 | Feb 2014 | JP | national |
2014-221646 | Oct 2014 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
6084908 | Chiang et al. | Jul 2000 | A |
8855203 | Yamamoto et al. | Oct 2014 | B2 |
9288490 | Sasai | Mar 2016 | B2 |
20120033731 | Yamamoto | Feb 2012 | A1 |
20130163664 | Guo | Jun 2013 | A1 |
Number | Date | Country |
---|---|---|
2004-112825 | Apr 2004 | JP |
2012-138773 | Jul 2012 | JP |
2014-13975 | Jan 2014 | JP |
2010-116869 | Oct 2010 | WO |
Entry |
---|
Benjamin Bross et al., “High Efficiency Video Coding (HEVC) text specification draft 10 (for FDIS & Last Call)”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCTVCL1003—v34, 12th Meeting: Geneva, CH, Jan. 14-23, 2013, pp. 1-298. |
Ali Tabatabai, Toby Walker, “USNB Comments on the MPEG-4 Part 10 AVC FCD”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 6th Meeting: Awaji, Island, JP, Dec. 5-13, 2002, Dec. 5, 2002, JVT-F089, pp. 1-5, URL,http://wflp3.itu.int/av-arch/jvt-site/2002—12—Awaji/JVT-F089-L.doc. |
Number | Date | Country | |
---|---|---|---|
20150222906 A1 | Aug 2015 | US |