This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2008-291220, filed on Nov. 13, 2008; the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an image decoding apparatus that includes a plurality of processors and a shared cache memory and applies various kinds of signal processing to an input image signal in parallel, an image decoding method, and an image data converting apparatus.
2. Description of the Related Art
As a method in the past for applying signal processing to an input image signal in parallel, for example, there is a technology disclosed in Japanese Patent Application Laid-Open No. 2007-318517. In a method of processing image data disclosed in Japanese Patent Application Laid-Open No. 2007-318517, slices are cyclically allocated to a plurality of arithmetic processing units in order to cause the arithmetic processing units to simultaneously process the slices in parallel. Parallel processing is set to simultaneously access a memory by making use of a characteristic that macro blocks referred to in the processing of the slices overlap. Consequently, a load on a bus caused when the arithmetic processing units access the memory is reduced. Therefore, usage of the memory is also reduced.
Japanese Patent Application Laid-Open No. 2008-118616 discloses a technology for using a plurality of threads, allocating an input bit stream (an image signal) to the threads for each of pictures, and collectively carrying out syntax analysis and decoding for each of the pictures in parallel.
However, the technologies in the past have problems that should be solved. Specifically, in the technology disclosed in Japanese Patent Application Laid-Open No. 2007-318517, when most of targets of arithmetic processing are data that do not overlap among slices as in a syntax analysis result of a bit stream, a sufficient effect cannot be obtained. Therefore, the technology cannot be effectively used. In the technology disclosed in Japanese Patent Application Laid-Open No. 2008-118616, syntax analysis and signal processing for slices are collectively performed in the same thread. Therefore, it is difficult to share reference to data overlapping among the slices. In particular, cache memory usage during writing in a picture substantially increases.
An image decoding apparatus according to an embodiment of the present invention comprises: a syntax-element compressing unit that executes compression processing on syntax elements extracted in syntax analysis processing on a bit stream and classifies obtained syntax elements after the compression based on types thereof; a plurality of syntax-element expanding units that correspond to any one of classified groups of syntax elements in a one to one relation and expand the syntax elements belonging to the corresponding group to restore the original syntax elements; and a plurality of signal processing units that correspond to any one of the syntax-element expanding units in a one to one relation and apply, to the syntax elements restored by the corresponding syntax-element expanding unit, signal processing corresponding to a type thereof.
An image decoding method according to an embodiment of the present invention comprises: applying compression processing to syntax elements obtained in syntax analysis processing for a bit stream, classifying the syntax elements after the compression based on types thereof, and storing the syntax elements in the inter-core shared cache memory; and allocating a different processor to each of the classified groups of syntax elements and executing respective kinds of signal processing corresponding to the respective groups of syntax elements.
An image data converting apparatus according to an embodiment of the present invention comprises: a syntax analyzing unit that executes syntax analysis processing on an input image bit stream; and a syntax-element compressing unit that converts a specific parameter included in syntax elements extracted in the syntax analysis processing by the syntax analyzing unit into a variable length code including a 3-bit type bit, a value of the type bit itself indicating an absolute value of the parameter before the compression when the value of the type bit is 0 to 5, a value obtained by adding 6 to a next value of 1 byte indicating an absolute value of the parameter before the compression when the value of the type bit is 6, and a value obtained by adding 262 to a next value of 2 bytes indicating an absolute value of the parameter before the compression when the value of the type bit is 7.
Exemplary embodiments of an image decoding apparatus, an image decoding method, and an image data converting apparatus according to the present invention will be explained below in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.
Such an image decoding apparatus is realized by, for example, executing software for image decoding on an apparatus including a plurality of pairs of processors and cache memories exclusively used by the processors and an inter-core shared cache memory usable by the processors shown in
The syntax elements compressed by the syntax-element compressing unit 2 are classified based on types thereof and stored in the queues 31a, 31b, and 31c.
The syntax-element expanding units 32a, 32b, and 32c extract the compressed syntax elements from the queues at the pre-stage and expands the syntax elements in procedures corresponding to the types thereof to restore the original syntax elements (before the compression).
The motion predicting unit 33a-1 performs motion prediction based on the syntax elements restored by the syntax-element expanding unit 32a. The luminance-motion compensating unit 33a-2 and the color-difference-motion compensating unit 33a-3 respectively perform motion compensation for luminance and motion compensation for color difference based on a result of the motion prediction by the motion predicting unit 33a-1.
The luminance-difference-image decoding unit 33b decodes differential image data concerning luminance based on the syntax elements restored by the syntax-element expanding unit 32b. The color-difference-differential-image decoding unit 33c decodes differential image data concerning color difference based on the syntax elements restored by the syntax-element expanding unit 32c.
A detailed operation of the image decoding apparatus having the configuration shown in
The syntax-element compressing unit 2 receives a syntax analysis result (syntax elements extracted from an input bit stream) from the syntax analyzing unit 1 and converts the syntax elements into variable length codes of a format explained later to thereby compress the syntax elements. The syntax-element compressing unit 2 classifies the syntax elements after the compression according to types thereof and stores the syntax elements in queues corresponding thereto. Specifically, the syntax-element compressing unit 2 classifies the syntax elements into syntax elements (syntax elements after the compression) concerning motion prediction, syntax elements concerning luminance differential image decoding, and syntax elements concerning color difference differential image decoding and stores the syntax elements in the queues corresponding thereto. For example, the syntax-element compressing unit 2 compresses the syntax elements concerning motion prediction and then stores the syntax elements in the queue 31a.
Details of a format of the syntax elements after the compression output from the syntax-element compressing unit 2 are explained. The syntax elements include one or more parameters. The syntax-element compressing unit 2 converts specific parameters into variable length codes among the parameters included in the syntax elements and compresses the parameters to thereby compress the syntax elements.
As compression performance for the variable length codes, Context-Adaptive Variable Length Coding (CAVLC) and Context-Adaptive Binary Arithmetic Coding (CABAC) specified in the H.264/MPEG-4 AVC standard or the like indicate overwhelmingly better performance. However, for decoding of these code words, context information around the code words is necessary, which prevents parallel signal processing. On the other hand, the variable length codes of the format shown in
The motion prediction information is a set (mvdx, mvdy) of differential values (parameters) of calculation results of motion prediction algorithms in the horizontal direction and the vertical direction and present values. Therefore, mvdx and mvdy are divided into signs (∓) and absolute values, the signs are represented by 1 bit (sign_of_mvdx, sign_of_mvdy), and the absolute values are represented by 3 bits (mm_format_type_of_mvdx, mm_format_type_of_mvdy) equivalent to type explained in the compression procedure (see
The information of the differential image is represented by parameters obtained by zigzag-scanning a coefficient matrix of DCT. Specifically, the information of the differential image is represented by a set of a parameter (zero_run) indicating the number of continuous 0's detected in the zigzag-scan and a parameter indicating coefficients (non-zero coefficients). When the coefficient matrix is treated as a matrix in a unit of 4×4, zero_run is 15 at the maximum. Therefore, this parameter is represented by 4 bits. On the other hand, the non-zero coefficient is divided into a sign (∓) and an absolute value, the sign is represented by 1 bit (sign_of_levelcode), and the absolute value is represented by 3 bits (coeff format type) equivalent to type (see
The operations of the image decoding apparatus having the configuration shown in
Similarly, the syntax-element expanding unit 32b reads out the syntax elements concerning the luminance differential image decoding after the compression stored in the queue 31b, executes processing opposite to the processing executed by the syntax-element compressing unit 2, and restores the original syntax elements. The syntax-element expanding unit 32c reads out the syntax elements concerning the color difference differential image decoding after the compression stored in the queue 31c, executes processing opposite to the processing executed by the syntax-element compressing unit 2, and restores the original syntax elements. The restored syntax elements are passed to the luminance-differential-image decoding unit 33b and the color-difference-differential-image decoding unit 33c at the post stage, respectively. The syntax elements are immediately discarded after being used in predetermined signal processing.
As explained above, the image decoding apparatus according to this embodiment converts the specific parameters in the various syntax elements obtained by executing the syntax analysis into the variable length codes and compresses the parameters to thereby compress the syntax elements. The image decoding apparatus classifies the syntax elements after the compression based on types thereof and stores the syntax elements in the queues. The image decoding apparatus expands the syntax elements immediately before using the syntax elements in the parallel signal processing at the post stage (restores the syntax elements to the state before the compression). After using the syntax elements in the signal processing, the image decoding apparatus immediately discards the syntax elements. This makes it possible to perform the various kinds of signal processing for decoding an image bit stream in parallel while preventing usage of the cache memory from increasing.
In the first embodiment, the image decoding apparatus that executes the various kinds of signal processing on the single image bit stream in parallel while holding down usage of the cache memory is explained. On the other hand, in a second embodiment of the present invention, an image processing apparatus that executes various kinds of signal processing on a plurality of image bit streams in parallel while holding down usage of a cache memory is explained.
As shown in the figure, the syntax-element compressing units 2-n (n=1 and 2) and the queues 31a-n, 31b-n, and 31c-n are associated with each other. The syntax-element compressing units classify syntax elements after compression based on types thereof and store the syntax elements in any ones of the queues associated with the syntax-element compressing units. Each of the switches 34a, 34b, and 34c extracts the compressed syntax elements from any one of the two queues at the pre-stage. A method of compressing the syntax elements is as explained in the first embodiment. Conditions for switching the switches are not particularly specified. For example, when a queue is about to overflow, the switches are switched to extract the compressed syntax elements from the queue or, when a connected queue is emptied, the switches are switched to another queue side.
In the image decoding apparatus having such a configuration, for example, an input image signal (video signal) is assigned to any ones of the syntax analyzing units for each of slices or pictures. The syntax analyzing units and the components at the post stage execute processing same as that of the components of the image decoding apparatus explained in the first embodiment to decode an image.
In
In the image decoding apparatus according to this embodiment, as shown in the upper part of
As shown in the middle part of the figure, the signal processing for the slices can be executed in parallel when the processing for the slice referred to is completed. For example, when the signal processing for a second slice shown in the figure is completed, the signal processing for third and fourth slices, which refer to the second slice, can be executed in parallel. Further, as shown in the lower part of the figure, in the signal processing, respective kinds of signal processing (MC: motion compensation, IQT: inverse quantization, and DBF: deblocking filter) for luma (luminance) and chroma (chromaticity) can be executed in parallel.
As explained above, the image decoding apparatus according to this embodiment includes a plurality of systems of the syntax analyzing units and the syntax-element compressing units and further includes a plurality of queues and switches corresponding to the systems. Consequently, effects same as those of the image decoding apparatus explained in the first embodiment can be obtained. Further, the signal processing can be performed in parallel for the processing concerning syntax analysis (the syntax analysis processing and the compression processing for a syntax analysis result).
Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
2008-291220 | Nov 2008 | JP | national |