This application relates to picture processing technologies, and in particular, to a transform method and an inverse transform method for a picture block, and an apparatus.
A picture block-based hybrid video coding scheme is used in a modern video coding standard, where an encoding process includes (intra and inter) prediction, transform, quantization, and entropy encoding on each picture block in a picture frame, and a decoding process includes entropy decoding, dequantization, inverse transform, and (intra and inter) prediction on a picture block in a picture frame. A derivative mode is a new block split mode. A plurality of block split modes may be derived based on one picture block, and a picture subblock is actually a split result of the picture block.
Currently, in a video coding process, transform and inverse transform are performed based on a picture subblock obtained through splitting. Consequently, coding efficiency is low.
This application provides a transform method and an inverse transform method for a picture block, and an apparatus, to improve picture coding efficiency.
According to a first aspect, this application provides a transform method for a picture block. For different to-be-encoded picture blocks in a to-be-processed picture, the to-be-encoded picture block is split into a plurality of picture subblocks, and at least one of the plurality of picture subblocks is transformed. In this way, transform performed on the to-be-encoded picture block is flexible and diversified, and complies with a picture feature of the to-be-encoded picture block. This improves picture coding efficiency.
A derivative mode is a new block split mode. A plurality of block split modes may be derived based on one picture block, and a picture subblock is actually a split result of the picture block. A picture block in a picture is an object on which encoding is performed. A plurality of block split methods may be used to split the picture block. A first split method is to split the picture block by using a horizontal line and/or a vertical line. In this split method, a plurality of split effects may be derived based on different combinations of four horizontal and vertical lines. A second split method is to split the picture block by using a horizontal line. In this split method, a plurality of split effects may be derived based on different combinations of a plurality of horizontal lines. A third split method is to split the picture block by using a vertical line. In this split method, a plurality of split effects may be derived based on different combinations of a plurality of vertical lines.
In an embodiment, an encoding apparatus splits the to-be-encoded picture block to obtain M picture subblocks, where M is an integer greater than 1; and transforms a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
In an embodiment, that the encoding apparatus transforms the to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined is any one of the following three methods: transforming each of the M picture subblocks; transforming a to-be-transformed picture block obtained after at least two of the M picture subblocks are combined; or transforming a to-be-transformed picture block obtained after all the M picture subblocks are combined.
In an embodiment, the encoding apparatus may further determine a block split method used for the to-be-encoded picture block; determine, from the three methods, a transform method used for the at least one of the picture subblocks; and add identification information in a bitstream of the picture, where the identification information is used to indicate the block split method used for the to-be-encoded picture block and the transform method used for the at least one of the picture subblocks.
In an embodiment, the encoding apparatus may vertically split the to-be-encoded picture block to obtain the plurality of picture subblocks, and/or horizontally split the to-be-encoded picture block to obtain the plurality of picture subblocks.
In an embodiment, the encoding apparatus determines a method with a lowest operation cost in the three methods as the transform method used for the at least one of the picture subblocks. Alternatively, the encoding apparatus determines, from the three methods and based on a result of comparison between a length and/or a width of the to-be-encoded picture block and a specified threshold, the transform method used for the at least one of the picture subblocks.
According to a second aspect, this application provides an inverse transform method for a picture block. For different to-be-decoded picture blocks in a to-be-processed picture, a block split method and a transform method that are used for the to-be-decoded picture block in the picture are determined based on identification information obtained from a bitstream. The to-be-decoded picture block is split according to the block split method to obtain a plurality of picture subblocks. Then, at least one of the picture subblocks is inversely transformed according to the transform method. In this way, inverse transform performed on the picture block is flexible and diversified, and complies with a picture feature of the picture block. This improves picture coding efficiency.
In an embodiment, a decoding apparatus splits the to-be-decoded picture block according to the block split method to obtain M picture subblocks, where M is an integer greater than 1; and inversely transforms, according to the transform method, a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
In an embodiment, that the decoding apparatus inversely transforms, according to the transform method, the to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined is any one of the following three methods: inversely transforming each of the M picture subblocks; inversely transforming a to-be-inversely transformed picture block obtained after at least two of the M picture subblocks are combined; or inversely transforming a to-be-inversely transformed picture block obtained after all the M picture subblocks are combined.
In an embodiment, that the decoding apparatus splits the to-be-decoded picture block according to the block split method to obtain the plurality of picture subblocks is any one of the following three methods: vertically splitting the to-be-decoded picture block to obtain the plurality of picture subblocks; horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks; or vertically and horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks.
In an embodiment, that the decoding apparatus inversely transforms the at least one of the picture subblocks according to the transform method is either of the following two methods: when the M picture subblocks are obtained by horizontally splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are vertically combined; or when the M picture subblocks are obtained by vertically splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are horizontally combined.
According to a third aspect, this application provides an encoding apparatus, including:
a first obtaining module, configured to obtain a to-be-encoded picture block in a to-be-processed picture, where the to-be-encoded picture block is an object on which encoding is performed;
a first split module, configured to split the to-be-encoded picture block to obtain a plurality of picture subblocks; and
a transform module, configured to transform at least one of the picture subblocks.
In an embodiment, the first split module is specifically configured to split the to-be-encoded picture block to obtain M picture subblocks, where M is an integer greater than 1.
The transform module is specifically configured to transform a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
In an embodiment, the transform module is specifically configured to perform any one of the following three methods:
transforming each of the M picture subblocks;
transforming a to-be-transformed picture block obtained after at least two of the M picture subblocks are combined; or
transforming a to-be-transformed picture block obtained after all the M picture subblocks are combined.
In an embodiment, the encoding apparatus further includes:
a first determining module, configured to determine a block split method used for the to-be-encoded picture block, and determine, from the three methods, a transform method used for the at least one of the picture subblocks.
In an embodiment, the first determining module is further configured to add identification information in a bitstream of the picture. The identification information is used to indicate the block split method used for the to-be-encoded picture block and the transform method used for the at least one of the picture subblocks.
In an embodiment, the first split module is specifically configured to vertically split the to-be-encoded picture block to obtain the plurality of picture subblocks, and/or horizontally split the to-be-encoded picture block to obtain the plurality of picture subblocks.
In an embodiment, the first determining module is specifically configured to determine a method with a lowest operation cost in the three methods as the transform method used for the at least one of the picture subblocks.
In an embodiment, the first determining module is specifically configured to determine, from the three methods and based on a result of comparison between a length and/or a width of the to-be-encoded picture block and a specified threshold, the transform method used for the at least one of the picture subblocks.
According to a fourth aspect, this application provides a decoding apparatus, including:
a second obtaining module, configured to obtain a bitstream of a to-be-processed picture, and obtain identification information from the bitstream;
a second determining module, configured to determine, based on the identification information, a block split method and a transform method that are used for a to-be-decoded picture block in the picture, where the to-be-decoded picture block is an object on which decoding is performed;
a second split module, configured to split the to-be-decoded picture block according to the block split method to obtain a plurality of picture subblocks; and
an inverse transform module, configured to inversely transform at least one of the picture subblocks according to the transform method.
In an embodiment, the second split module is specifically configured to split the to-be-decoded picture block according to the block split method to obtain M picture subblocks, where M is an integer greater than 1.
The inverse transform module is specifically configured to inversely transform, according to the transform method, a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
In an embodiment, the inverse transform module is specifically configured to perform any one of the following three methods:
inversely transforming each of the M picture subblocks;
inversely transforming a to-be-inversely transformed picture block obtained after at least two of the M picture subblocks are combined; or
inversely transforming a to-be-inversely transformed picture block obtained after all the M picture subblocks are combined.
In an embodiment, the second split module is specifically configured to perform any one of the following three methods:
vertically splitting the to-be-decoded picture block to obtain the plurality of picture subblocks;
horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks; or
vertically and horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks.
In an embodiment, the inverse transform module is specifically configured to perform either of the following two methods:
when the M picture subblocks are obtained by horizontally splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are vertically combined; or
when the M picture subblocks are obtained by vertically splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are horizontally combined.
According to a fifth aspect, this application provides a picture processing device, including:
one or more processors, and
a memory, configured to store one or more programs.
When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the transform method for a picture block according to the first aspect or the inverse transform method for a picture block according to the second aspect.
According to a sixth aspect, this application provides a computer-readable storage medium. The computer-readable storage medium stores instructions. When the instructions are run on a computer, the instructions are used to perform the transform method for a picture block according to the first aspect or the inverse transform method for a picture block according to the second aspect.
According to a seventh aspect, this application provides a computer program. When the computer program is executed by a computer, the computer program is used to perform the transform method for a picture block according to the first aspect or the inverse transform method for a picture block according to the second aspect.
To make the objectives, technical solutions, and advantages of this application clearer, the following clearly describes the technical solutions in this application with reference to the accompanying drawings in this application. Definitely, the described embodiments are merely a part rather than all of the embodiments of this application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of this application without creative efforts shall fall within the protection scope of this application.
A derivative mode is a new block split mode. A plurality of block split modes may be derived based on one picture block, and a picture subblock is actually a split result of the picture block.
Based on any split effect shown in the foregoing embodiments of the block split method, this application provides a transform or inverse transform method for a picture block. For different to-be-processed picture blocks (namely, to-be-encoded picture blocks or to-be-decoded picture blocks) in a to-be-processed picture, the to-be-processed picture block is split to obtain a plurality of picture subblocks, and at least one of the plurality of picture subblocks is transformed or inversely transformed. In this way, transform or inverse transform performed on the to-be-processed picture block is flexible and diversified, and complies with a picture feature of the to-be-processed picture block. This improves picture coding efficiency.
Step S101: Obtain a to-be-encoded picture block in a to-be-processed picture, where the to-be-encoded picture block is an object on which encoding is performed.
In this application, encoding is performed based on a to-be-encoded picture block. The to-be-encoded picture block may be a square or a non-square. The to-be-encoded picture block is a largest transform unit in this application.
Step S102: Split the to-be-encoded picture block to obtain a plurality of picture subblocks.
In this application, the encoding apparatus may obtain M picture subblocks (where M is an integer greater than 1) by splitting the to-be-encoded picture block. The picture subblocks may be obtained through a derivative split or a non-derivative split. The split effects shown in
Step S103: Transform at least one of the picture subblocks.
In this application, transform is performed on a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks obtained in step S102 are combined (where N is an integer less than or equal to M and greater than or equal to 1).
In an embodiment, the transform may be any one of the following three methods: transforming each of the M picture subblocks; transforming a to-be-transformed picture block obtained after at least two of the M picture subblocks are combined; or transforming a to-be-transformed picture block obtained after all the M picture subblocks are combined. To be specific, the encoding apparatus may separately transform each of the M picture subblocks. For example, in the picture block 206 in
In another embodiment, the transform may be either of the following two methods: if the M picture subblocks are obtained by horizontally splitting the to-be-encoded picture block, transforming a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are vertically combined; or if the M picture subblocks are obtained by vertically splitting the to-be-encoded picture block, transforming a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are horizontally combined. For example, if a block split method of the to-be-encoded picture block is a derivative split corresponding to the picture block 200, combining of the N picture subblocks is to combine neighboring picture subblocks in a vertical direction. If the block split method of the picture block is a derivative split corresponding to the picture block 300, combining of the N picture subblocks is to combine neighboring picture subblocks in a horizontal direction.
It can be learned that in this application, transform is performed on the at least one of the plurality of picture subblocks obtained by splitting the to-be-encoded picture block. In this way, transform performed on the to-be-encoded picture block is flexible and diversified, and complies with a picture feature of the to-be-encoded picture block. This improves picture encoding efficiency.
In an embodiment, the encoding apparatus needs to determine the block split method used for the to-be-encoded picture block; determine, from the three methods, a transform method used for the at least one of the picture subblocks; and add identification information in a bitstream of the picture. The identification information is used to indicate the block split method used for the to-be-encoded picture block and the transform method used for the at least one of the picture subblocks.
As described above, there may be three transform methods used for the at least one of the picture subblocks, and the encoding apparatus needs to determine one of the transform methods as the transform method used for the at least one of the picture subblocks. The determining process may be determining a transform method with a lowest operation cost in the three methods as the transform method used for the at least one of the picture subblocks, or determining, from the three methods and based on a result of comparison between a length and/or a width of the to-be-encoded picture block and a specified threshold, the transform method used for the at least one of the picture subblocks. In this application, the encoding apparatus calculates an operation cost of each of the three methods, and determines the transform method with the lowest operation cost as the transform method used for the at least one of the picture subblocks. For example, for the picture block 108, a cost function (the cost function may be set with reference to factors such as a pixel, a color level, and a grayscale in the picture block, and may also be set with reference to factors such as an amount of data and a quantity of pixels in the picture block; a derivative refinement method and a transform method that are selected based on these reference factors meet the picture feature in the picture block; and then encoding that best complies with the picture feature is implemented) is used to calculate operation costs of transforming the picture block by using the three methods, and the transform method with the lowest operation cost is determined as the transform method used for the at least one of the picture subblocks. Alternatively, the encoding apparatus may compare the length and/or the width of the picture block with the specified threshold. For example, the length or the width of the picture block is greater than M, or a ratio of the length to the width of the picture block is less than 1.5. A transform method corresponding to the determining condition is selected as the transform method used for the at least one of the picture subblocks. There may be another determining condition for comparing the length and/or the width of the to-be-encoded picture block with the specified threshold. This is not specifically limited. Based on the foregoing two methods for determining a transform method, a transform method that best complies with the picture feature of the to-be-encoded picture block can be determined, thereby improving picture encoding efficiency.
Because the encoding apparatus may determine different transform methods for different to-be-encoded picture blocks, the encoding apparatus needs to notify a decoding apparatus of the different transform methods, so that the decoding apparatus can determine an inverse transform method to be used for a corresponding to-be-decoded picture block. In this application, the decoding apparatus and the encoding apparatus may be located on a same picture processing device, or may be located on different picture processing devices. For example, the encoding device is located on a photographing device, the decoding device is located on a video playback device, and the two devices are connected by a network or a cable. Based on this, the encoding apparatus may be synchronized with the decoding apparatus, and transmit an encoded picture bitstream to the decoding apparatus while performing encoding in real time, or transmit an encoded picture bitstream to the decoding apparatus when necessary (for example, when a user requests to watch a video).
Step S201: Obtain a bitstream of a to-be-processed picture, and obtain identification information from the bitstream.
The decoding apparatus may obtain the bitstream of the to-be-processed picture in a plurality of manners. The bitstream may be obtained in real time or may be extracted at one time.
Step S202: Determine, based on the identification information, a block split method and a transform method that are used for a to-be-decoded picture block in the picture, where the to-be-decoded picture block is an object on which decoding is performed.
Step S203: Split the to-be-decoded picture block according to the block split method to obtain a plurality of picture subblocks.
The block split method performed by the decoding apparatus on the to-be-decoded picture block is determined based on the identification information, and may be any one of block split effects shown in
Step S204: Inversely transform at least one of the picture subblocks according to the transform method.
Inverse transform performed by the decoding apparatus on the to-be-decoded picture block corresponds to transform performed by the encoding apparatus on the to-be-encoded picture block. A principle of an inverse transform process of the decoding apparatus is similar to a principle of a transform process of the encoding apparatus, and details are not described herein again.
In this application, inverse transform is performed on the at least one of the plurality of picture subblocks obtained by splitting the to-be-decoded picture block. In this way, inverse transform performed on the to-be-decoded picture block is flexible and diversified, and complies with a picture feature of the to-be-decoded picture block. This improves picture decoding efficiency.
Based on the foregoing technical solution, the first split module 12 is specifically configured to split the to-be-encoded picture block to obtain M picture subblocks, where M is an integer greater than 1. The transform module 13 is specifically configured to transform a to-be-transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
Based on the foregoing technical solution, the transform module 13 is specifically configured to perform any one of the following three methods: transforming each of the M picture subblocks; transforming a to-be-transformed picture block obtained after at least two of the M picture subblocks are combined; or transforming a to-be-transformed picture block obtained after all the M picture subblocks are combined.
Based on the foregoing technical solution,
Based on the foregoing technical solution, the first determining module 14 is further configured to add identification information in a bitstream of the picture. The identification information is used to indicate the block split method used for the to-be-encoded picture block and the transform method used for the at least one of the picture subblocks.
Based on the foregoing technical solution, the first split module 12 is specifically configured to vertically split the to-be-encoded picture block to obtain the plurality of picture subblocks, and/or horizontally split the to-be-encoded picture block to obtain the plurality of picture subblocks.
Based on the foregoing technical solution, the first determining module 14 is specifically configured to determine a method with a lowest operation cost in the three methods as the transform method used for the at least one of the picture subblocks.
Based on the foregoing technical solution, the first determining module 14 is specifically configured to determine, from the three methods and based on a result of comparison between a length and/or a width of the to-be-encoded picture block and a specified threshold, the transform method used for the at least one of the picture subblocks.
The apparatus in this embodiment may be used to execute the technical solutions of the method embodiment shown in
Based on the foregoing technical solution, the second split module 23 is specifically configured to split the to-be-decoded picture block according to the block split method to obtain M picture subblocks, where M is an integer greater than 1; and the inverse transform module 24 is specifically configured to inversely transform, according to the transform method, a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are combined, where N is an integer less than or equal to M and greater than or equal to 1.
Based on the foregoing technical solution, the inverse transform module 24 is specifically configured to perform any one of the following three methods: inversely transforming each of the M picture subblocks; inversely transforming a to-be-inversely transformed picture block obtained after at least two of the M picture subblocks are combined; or inversely transforming a to-be-inversely transformed picture block obtained after all the M picture subblocks are combined.
Based on the foregoing technical solution, the second split module 23 is specifically configured to perform any one of the following three methods: vertically splitting the to-be-decoded picture block to obtain the plurality of picture subblocks; horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks; or vertically and horizontally splitting the to-be-decoded picture block to obtain the plurality of picture subblocks.
Based on the foregoing technical solution, the inverse transform module 24 is specifically configured to perform either of the following two methods: when the M picture subblocks are obtained by horizontally splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are vertically combined; or when the M picture subblocks are obtained by vertically splitting the picture block, inversely transforming a to-be-inversely transformed picture block obtained after N neighboring picture subblocks in the M picture subblocks are horizontally combined.
The apparatus in this embodiment may be used to execute the technical solutions of the method embodiment shown in
As a computer-readable storage medium, the memory 31 may be configured to store a software program, a computer-executable program, and a module, for example, a program instruction/module corresponding to the method in the embodiment shown in
The memory 31 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and an application program required by at least one function. The data storage area may store data created based on use of a terminal, and the like. In addition, the memory 31 may include a high-speed random access memory, and may further include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or another non-volatile solid-state storage device. In some examples, the memory 31 may further include a memory that is remotely disposed for the processor 30, and the remote memory may be connected to the picture processing device through a network. Examples of the foregoing network include but are not limited to the internet, an intranet, a local area network, a mobile communications network, and a combination thereof.
The input apparatus 32 may be configured to: receive input digit or character information, and generate a key signal input related to a user setting and function control of the picture processing device. The output apparatus 33 may include a display device such as a display screen.
In an embodiment, this application provides a computer-readable storage medium. The computer-readable storage medium stores instructions, and when the instructions are run on a computer, the instructions are used to perform the method embodiment shown in
In an embodiment, this application provides a computer program. When the computer program is executed by a computer, the computer program is used to perform the method embodiment shown in
A person of ordinary skill in the art may understand that all or some of the steps in the foregoing method embodiments may be implemented by program-instructing related hardware. The program may be stored in a computer-readable storage medium. When the program is executed, the steps in the foregoing method embodiments are performed. The foregoing storage medium includes any medium that can store program code, such as a ROM, a RAM, a magnetic disk, or an optical disc.
Finally, it should be noted that the foregoing embodiments are merely intended for describing the technical solutions of this application other than limiting this application. Although this application is described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they may still make modifications to the technical solutions described in the foregoing embodiments or make equivalent replacements to some or all technical features thereof, without departing from the scope of the technical solutions of the embodiments of this application.
Number | Date | Country | Kind |
---|---|---|---|
201811198621.X | Oct 2018 | CN | national |
This application is a continuation of International Application No. PCT/CN2019/110355, filed on Oct. 10, 2019, which claims priority to Chinese Patent Application No. 201811198621.X, filed on Oct. 15, 2018. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2019/110355 | Oct 2019 | US |
Child | 17230699 | US |