Patent Application
20180109702
This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2016-0133595, filed on Oct. 14, 2016 in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.
The present disclosure relates generally to an image processing apparatus and a control method thereof, and for example to an image processing apparatus, which to efficiently compress a received image signal, converts the received image signal to another color gamut and then into a digital image signal, and a control method thereof.
As consumption for image contents grows, demands for efficient storage and efficient transmission and reception of images are rising. In particular, when the images are transmitted by wireless, to improve communication efficiency, a technique for efficiently compressing the images is required.
To efficiently compress and transmit the images, a high efficiency video codec (HEVC) as a related art for compression of image contents is known. The HEVC is also called a H.265 or a MPEG-H part2. The HEVC converts a color gamut for a received image signal and compresses the color gamut-converted image signal. If the color gamut of the image signal is converted before compressing the image signal, the image signal can not only be compressed with a high efficiency, but also converted/inverse-converted with more less loss and the signal to noise ratio is also improved.
However, in a specific image, if the color gamut of a corresponding image signal is converted before compressing the corresponding image signal according to the related art, there is a problem in that when the corresponding image signal is restored, due to a characteristic thereof, a color thereof is distorted or an unwanted artifact is magnified.
Example embodiments address at least the above problems and/or disadvantages and other disadvantages not described above.
The example embodiments may provide an image processing apparatus, which efficiently compresses an image signal without distorting and/or reducing distortion a color thereof, and a control method thereof.
In accordance with an example aspect of the disclosure, an image processing apparatus is provided, the apparatus including a signal processor configured to process an image signal including a plurality of color components, and a controller configured to control the signal processor to perform a color gamut conversion, a domain transform, a quantization processing and an encoding processing with respect to an input image signal, and in response to differences between the color components of the image signal being less than a first critical level, to not perform the color gamut conversion. Accordingly, the image processing apparatus may efficiently compress the input image signal without distorting or reducing a distortion of a color thereof.
The controller may be configured to, in response to a high frequency component of the image signal exceeding a second critical level, control the signal processor to not perform the color gamut conversion, and to perform the quantization processing.
The controller may be configured to, in response to the differences between the color components of the image signal being less than a third critical level higher than the first critical level and the high frequency component of the image signal exceeding a fourth critical level lower than the second critical level, control the signal processor to not perform the color gamut conversion, and to perform the quantization processing.
The controller may be configured to control the signal processor to, in response to the differences between the color components of the image signal exceeding the first critical level, perform a test for the color gamut conversion, and to determine whether to perform the color gamut conversion based on at least one of: a compression ratio and a signal to noise ratio of the image signal as a result of the performed test.
The apparatus may further include communicator comprising communication circuitry configured to communicate with an external apparatus, and the controller may be configured to control the communicator to transmit the image signal to which the encoding processing is performed, to the external apparatus.
In accordance with another example aspect of the disclosure, an image processing apparatus is provided, the apparatus including a signal processor configured to process an image signal including a plurality of color components, and a controller configured to control the signal processor to perform a color gamut conversion, a domain transform, a quantization processing and an encoding processing with respect to an input image signal, and, in response to a high frequency component of the image signal exceeding a first critical level, to not perform the color gamut conversion. With this, the image processing apparatus may efficiently compress the input image signal without distorting and/or reducing a distortion of a color thereof.
The controller may be configured to, in response to differences between the color components of the image signal being less than a second critical level, control the signal processor to not perform the color gamut conversion, and to perform the quantization processing.
The controller may be configured to, in response to the high frequency component of the image signal exceeding a third critical level lower than the first critical level and the difference between the color components of the image signal being less than a fourth critical level higher than the second critical level, control the signal processor to not perform the color gamut conversion, and to perform the quantization processing.
The controller may be configured to control the signal processor to, in response to the high frequency component of the image signal exceeding the first critical level, perform a test for the color gamut conversion, and to determine whether to perform the color gamut conversion based on at least one of: a compression ratio and a signal to noise ratio of the image signal as a result of the performed test.
The apparatus may further include a communicator comprising communication circuitry configured to communicate with an external apparatus, and the controller may be configured to control the communicator to transmit the image signal to which the encoding processing is performed, to the external apparatus.
In accordance with another example aspect of the disclosure, a method of controlling an image processing apparatus including a signal processor configured to process an image signal having a plurality of color components is provided, the method including performing, by the signal processor, a color gamut conversion, a domain transform, a quantization processing and an encoding processing with respect to an input image signal, and in response to a difference between the color components of the image signal being less than a first critical level, controlling the signal processor to not perform the color gamut conversion. Thus, the image processing apparatus may efficiently compress the input image signal without distorting and/or reducing a distortion of a color thereof.
The method may further include, in response to a high frequency component of the image signal exceeding a second critical level, performing the quantization processing without performing the color gamut conversion.
The method may further include, in response to the differences between the color components of the image signal being less than a third critical level higher than the first critical level and the high frequency component of the image signal exceeding a fourth critical level lower than the second critical level, performing the quantization processing without performing the color gamut conversion.
The method may further include, in response to the differences between the color components of the image signal exceeding the first critical level, performing a test for the color gamut conversion, and determining whether to perform the color gamut conversion based on at least one of a compression ratio and a signal to noise ratio of the image signal as a result of the performed test.
The method may further include transmitting the image signal to which the encoding processing is performed, to an external apparatus.
In accordance with further example aspect of the disclosure, a control method of an image processing apparatus including a signal processor configured to process an image signal having a plurality of color components is provided, the method including performing, by the signal processor, a color gamut conversion, a domain transform, a quantization processing and an encoding processing with respect to an input image signal, and in response to a high frequency component of the image signal exceeding a first critical level, controlling the signal processor to not perform the color gamut conversion. According to this, the image processing apparatus may efficiently compress the input image signal without distorting and/or reducing a distortion of a color thereof.
The method may further include, in response to differences between the color components of the image signal being less than a second critical level, performing the quantization processing without performing the color gamut conversion.
The method may further include, in response to the high frequency component of the image signal exceeding a third critical level lower than the first critical level and the differences between the color components of the image signal being less than a fourth critical level higher than the second critical level, performing the quantization processing without performing the color gamut conversion.
The method may further include, in response to the high frequency component of the image signal exceeding the first critical level, performing a test for the color gamut conversion, and determining whether to perform the color gamut conversion based on at least one of a compression ratio and a signal to noise ratio of the image signal as a result of the performed test.
The method may further include transmitting the image signal to which the encoding processing is performed, to an external apparatus.
As described above, according to various example embodiments, the image processing apparatus may selectively convert the color gamut of the image signal, thereby efficiently compressing the image signal without distorting and/or reducing a distortion of the color thereof.
The above and/or other aspects, features and attendant advantages of the present disclosure will become apparent and more readily appreciated from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, and wherein:
Below, various example embodiments will be described in greater detail with reference to accompanying drawings. The following descriptions of the example embodiments are made by referring to elements illustrated in the accompanying drawings, in which like numerals refer to like elements having substantively the same functions.
Hereinafter, in the description of the example embodiments, a ‘color gamut’, which represents bounds of colors displayable by a certain output device may also be referred to a color space. Types of the color gamut includes, for example, a RGB (red, green, blue) to represent colors in an additive color mixture method that if the colors are mixed, the brightness is increased, a CMYK (cyan, magenta, yellow, black) using a subtractive color mixture method, a HSV (hue, saturation, value), a CIELAB, a CIELUV, a YCoCg (Y: luminance, Co: orange chrominance, Cg: green chrominance), a YCbCr (Y: luminance, Cb: blue chrominance, Cr: red chrominance), etc. Components representing pixel values, such R, G and B in the RGB and C, M, Y and K in the CMYK, may be referred, for example, to as color components of an image signal.
As illustrated in
The image processing apparatus 1 according to an example embodiment performs a color gamut conversion, a domain transform, a quantization processing and an encoding processing with respect to the input image signal, and compresses the input image signal by encoding the quantized signal in a digital code. Here, the image processing apparatus 1 may analyze the input image signal to determine whether when performing the compression after the color gamut conversion, the input image signal will be distorted in color and based on the determination result, and may not perform the color gamut conversion, but the quantization processing and the encoding processing. In other words, the image processing apparatus 1 may perform quantization processing and encoding processing without performing color gamut conversion. As an example, if differences between color components in a specific pixel or region is less than a critical level, the image processing apparatus 1 determines that the image signal may be distorted in color and does not perform the color gamut conversion, but the domain transform, the quantization processing and the encoding processing. In other words, to compare the differences between the color components in a RGB color gamut, the image processing apparatus 1 may compare a difference between an R value and a B value, a difference between a B value and a G value and a difference between an R value and a G value, which are color components in the specific pixel or region. As another example, if a frequency component of the image signal exceeds a critical level, the image processing apparatus 1 may not perform the color gamut conversion, but the quantization processing and the encoding processing. With this, the image processing apparatus may efficiently compress the image signal without distorting and/or reducing a distortion of the color thereof.
Hereinafter, an example configuration of the image processing apparatus 1 will be described with reference to a block diagram of the image processing apparatus.
The image processing apparatus 1 may include a signal receiver 200 including various circuitry for receiving the image signal. The signal receiver 200 may be provided with, for example, and without limitation, a tuner. The tuner receives tuning a broadcasting signal of any one selected by a user from among a plurality of channels. The signal receiver 200 may also receive an image signal from a server via an electronic device, such as a digital versatile disk (DVD), a PC or the like, a mobile device such as a smart phone, or an internet.
The signal processor 201 may include various signal processing circuitry and performs an image processing with respect to the image signal received via the signal receiver 200, and displays an image on a display based on the processed image signal, transmits the processed image signal to an external apparatus 2 via the communicator 205, or stores the processed image signal in the storage 207. Image processing processes performed by the circuitry of the signal processor 201 may include, for example, demultiplexing for separating a transmission stream including the image signal into sub-streams, such as an image signal, an audio signal, and addition data, de-interlacing for converting the image signal from an interlace form to a progressive form, scaling for adjusting the resolution of the image signal, noise reduction for improving image quality, detail enhancement, frame refresh rate conversion, etc. The signal processor 201 may perform a compression processing with respect to the received image signal to output an encoded digital image signal. The image compression may be performed by block units, dividing an image into a plurality of macroblocks. The macroblocks may have various sizes, for example, 16*16, 32*32, 64*64, etc. The sizes of the macroblocks are adaptively selected for compression efficiency.
The image processing apparatus 1 may further include a display (not shown) for outputting an image based on the image signal processed by the signal processor 201. Achieved types of the display are not limited. For instance, the display may be achieved in various display types, such as a liquid crystal display (LCD) , a plasma display panel (PDP), a light-emitting diode (LED) display, an organic light emitting diodes (OLED) display, a surface-conduction electron-emitter, a carbon nano-tube, a nano-crystal display, or the like, but is not limited thereto.
If the display is a LCD type, the display includes a LCD panel, a backlight unit to supply light to the LCD panel, a panel driving board to drive the LCD panel, etc. The display may be also achieved by an OLED panel, which is a spontaneous emission element, without the backlight unit.
The image processing apparatus 1 may further include a communicator 205 which may include various communication circuitry for transmitting the compressed image signal to the outside. The communicator 205 is configured to communicate with the external apparatus 2. The communicator 205 is achieved in various types according to achieved types of the external apparatus or the image processing apparatus 1. For instance, the communicator 205 may include a connecting part for wired communication. The connecting part may transmit/receive signals/data based on standards, such as a high definition multimedia interface (HDMI), a HDMI-consumer electronics control (CEC), a universal serial bus (USB), a component and so on, and include more than at least one connector or terminal corresponding to the standards, respectively. The communicator 205 may communicate by wire with a plurality of servers via a wired local area network (LAN).
According to design methods of the image processing apparatus 1, the communicator 205 may include various configurations besides the connecting part including the connector or terminals for wired connection. As a non-limiting example, the communicator 205 may include a radio frequency (RF) circuit for transmitting and receiving a RF signal to perform a wireless communication with the external apparatus, and may be configured to perform communication via at least one from among wireless fidelity (Wi-Fi), Bluetooth, Zigbee, ultra-wide band (UWB), wireless USB, and near field communication (NFC).
The image processing apparatus 1 may further include storage 207 for storing the compressed image signal. The storage 207 includes a non-volatile memory (writable ROM), which retains data regardless of whether the image processing apparatus 1 is turned on or off and which is writable to reflect changes. In other words, the storage 207 may be provided with any one of a flash memory, an EPROM and an EEPROM. The storage 207 may be further provided with a volatile memory, such as a DRAM or a SRAM, which has a reading or writing speed faster than the non-volatile memory.
The controller 209 performs controls needed for operating all the elements of the image processing apparatus 1. The controller 209 may include a control program for controlling to perform the control operation as described above, a non-volatile memory in which the control program is installed, a volatile memory in which at least a portion of the installed control program is loaded, and at least one microprocessor or central processing unit (CPU) for executing the loaded control program.
The controller 209 may analyze differences between the color components, frequency characteristic and the like of the image signal, and perform a color conversion test to determine whether to execute the color gamut conversion. The controller 209 may control the signal processor 201 to selectively perform the color gamut conversion with respect to the received image signal based on the determination, and then to perform the quantization processing and the encoding processing.
Hereinafter, a more detailed configuration of the signal processor 201 for performing the compression processing with respect to the image signal will be described with reference to
The color gamut converter 300 may include various circuitry and/or program elements and converts color components of a received image signal to another color gamut. Although the HEVC as described above converts the input image signal of RGB color gamut into an image signal of YCoCg color gamut, which has a high compression ratio and which is inversely convertible without loss, and compresses the converted image signal, the idea of the disclosure is not limited thereto.
According to an example embodiment, the color gamut conversion may be selectively performed. The controller 209 may analyze the input image signal to determine whether to perform the color gamut conversion with respect to the image signal. For eyes of the human, the smaller the differences between the color components are and the higher the high frequency component is, the more a sensitivity capable of perceiving details of pixels is decreased and the easier a color distortion is perceived, thereby generating degradation with high subjective sensitivity. Accordingly, the analysis of the image signal include determining whether the respective differences between the color components of image signal are less than a critical level and/or the high frequency component of the image signal exceeds a critical level. The controller 209 may also determine whether to perform the color gamut conversion according to whether a loss does not occur in a compression ratio or a signal to noise ratio as a result of color gamut conversion test, i.e., based on a rate-distortion cost varying according to the color gamut conversion.
The controller 209 may control the signal processor 201 not to perform the color gamut conversion, based on the determination on whether to perform the color gamut conversion. As an example, as illustrated in
The image signal of which the color gamut is changed or not enters the domain transforming/quantizing part 301. The domain transforming/quantizing part 301 performs a domain transform and a quantization processing with respect to the image signal. The domain transform, which transforms the image signal of time domain into an image signal of frequency domain, may be performed for example, and without limitation, by any one of a discrete cosine transform (hereinafter, also referred to a ‘DCT transform’), a fast fourier transform (hereinafter, also referred to a ‘FFT transform’), and a karhunen loeve transform (hereinafter, also referred to a ‘KLV transform’), but is not limited thereto. The encoder 303 encodes the quantized image signal to convert into a digital image signal. The image signal is converted into the encoded image signal at the signal processer 201 and transmitted to the display, the communicator 205 and the storage 207.
The execution of the color gamut conversion may be determined by macroblock units. If all of differences between color components of each pixel included in every macroblock is less than the critical level, the controller 209 may determine to omit the color gamut conversion. However, the disclosure is not limited thereto.
At an operation S500, the signal processor 201 receives an image signal. At an operation S501, the controller 209 determines whether a high frequency component of the image signal exceeds a critical level. The controller 209 determines to omit performing a color gamut conversion if the high frequency component of the image signal exceeds the critical level, and to perform the color gamut conversion if the high frequency component of the image signal is less than the critical level. If it is determined to perform the color gamut conversion, at an operation S502, the signal processor 201 performs the color gamut conversion with respect to the image signal. At an operation S503, the signal processor 201 performs a domain transform, a quantization processing and an encoding processing with respect to the image signal to which the color gamut conversion is performed or not. Lastly, at an operation S504, the signal processor 201 outputs the encoded image signal.
At an operation S600, the signal processor 201 receives an image signal. At an operation S601, the controller 209 determines whether differences between color components of the image signal are less than a first critical level. If the differences between the color components are less than the first critical level, the controller 209 determines to omit performing a color gamut conversion. If the differences between the color components exceed the first critical level, at an operation S602, the controller 209 determines whether a high frequency component of the image signal exceeds a second critical level. The controller 209 determines to omit performing the color gamut conversion if the high frequency component of the image signal exceeds the second critical level, and to perform the color gamut conversion if the high frequency component of the image signal is less than the second critical level. If it is determined to perform the color gamut conversion, at an operation S603, the signal processor 201 performs the color gamut conversion with respect to the image signal. At an operation S604, the signal processor 201 performs a domain transform, a quantization processing and an encoding processing with respect to the image signal to which the color gamut conversion is performed or not. At an operation S605, the signal processor 201 outputs the encoded image signal.
In the example embodiment, the image processing apparatus 1 more specifically determines whether the color is distorted taking into account both the differences between the color components and the frequency characteristic of the image signal, and compresses the image signal. As another example, the controller 209 may determine whether the color is distorted synthetically taking account of the differences between the color components and the frequency characteristic of the image signal. For instance, the controller 209 may calculate (determine) a synthetic color-distortion cost by multiplying the differences between the color components and a ratio of the high frequency component by predetermined coefficients, respectively and adding them, and then determine whether the color will be distorted based the calculated synthetic color-distortion cost.
At an operation S700, the signal processor 201 receives an image signal. At an operation S701, the controller 209 determines whether differences between color components of the image signal are less than a first critical level. If the differences between the color components are less than the first critical level, the controller 209 determines to omit performing a color gamut conversion for the image signal. If the differences between the color components exceeds the first critical level, at an operation S703, the controller 209 determines whether the differences between the color components are less than a third critical level higher than the first critical level. If the differences between the color components exceed the third critical level, the controller 209 determines to perform the color gamut conversion. If the differences between the color components are less than the third critical level, as an operation S702, the controller 209 determines whether a high frequency component of the image signal exceeds a second critical level. If the high frequency component of the image signal exceeds the second critical level, the controller 209 determines to omit performing the color gamut conversion for the image signal. If the high frequency component of the image signal is less than the second critical level, at an operation S704, the controller 209 determines whether the high frequency component of the image signal exceeds a fourth critical level lower than the second critical level. If the high frequency component of the image signal exceeds the fourth critical level, the controller 209 determines to omit performing the color gamut conversion for the image signal. If the high frequency component of the image signal is less than the fourth critical level, the controller 209 determines to perform the color gamut conversion for the image signal.
If it is determined to perform the color gamut conversion for the image signal, at an operation S705, the signal processor 201 performs the color gamut conversion with respect to the image signal. At an operation S706, the signal processor 201 performs a domain transform, a quantization processing and an encoding processing with respect to the image signal to which the color gamut conversion is performed or not. At an operation S707, the signal processor 201 outputs the encoded image signal.
In the example embodiment, the image processing apparatus 1 may determine that the distortion of the color can occur if the differences between the color components of the image signal are higher than the first critical level, but less than the third critical level and the high frequency component of the image signal is equal to or more than the fourth critical level lower than the second critical level. In other words, the image processing apparatus 1 determines whether to perform the color gamut conversion synthetically taking account of the differences between the color components and the frequency characteristic of the image signal.
If an image signal is received by the signal processor 201, it enters the color gamut converter 800 and the second domain transforming/quantizing part 802. The color gamut converter 800 performs the color gamut conversion with respect to the image signal and transmits a first image signal to which the color gamut conversion is performed, to the first domain transforming/quantizing part 801. The first domain transforming/quantizing part 801 performs a domain transform from a time domain to a frequency domain with respect to the first image signal, and performs a quantization processing with respect to the domain transformed-first image signal to generate a first quantization coefficient. The second domain transforming/quantizing part 802 performs the domain transform from the time domain to the frequency domain with respect to a second image signal to which the color gamut conversion is not performed, and performs the quantization processing with respect to the domain transformed-second image signal to generate a second quantization coefficient. The first and second domain transforming/quantizing parts 801 and 802 transmit the generated first and second quantization coefficients to the rate-distortion cost calculator 807. The first and second inverse domain transforming/quantizing parts 803 and 804 perform an inverse domain transform from the frequency domain to the time domain and an inverse quantization processing with respect to the first and second image signal to which the domain transform and the quantization processing are performed. The first and second restorers 805 and 806 perform a restoration processing with respect to the inversely quantized first and second image signals of time domain. The first and second restorers 805 and 806 transmit the restored first and second restoration signals to the rate-distortion cost calculator 807. The rate-distortion cost calculator 807 calculates (determines) a rate-distortion cost using the generated first and second quantization coefficients and the first and second restoration signals.
The rate-distortion cost, which may be calculated using a compression ratio and a mean square error (MSE), may, for example, and without limitation, be expressed by the following mathematical formula.
J=D+λR [Mathematical formula 1]
Here, J is a rate-distortion cost, D is a mean square error (hereinafter, also referred to MSE) of an image calculated from the first and second restoration signals, R is a bit rate of a compression signal calculated from the first and second quantization coefficients and λ as a coefficient for calculating the rate-distortion cost is a separately calculated lagrange multiplier. The mathematical formula 1 is merely an example for calculating the rate-distortion cost and the disclosure is not limited thereto.
In the example embodiment, the controller 209 performs a color gamut conversion test with respect to the image signal to determine whether when the color gamut conversion is performed with respect to the image signal, a loss occurs in the compression ratio and the signal to noise ratio, as compared with when the color gamut conversion is not performed, and determines whether to perform the color gamut conversion with respect to the image signal, using the rate-distortion cost calculated based on the color gamut conversion test.
The controller 209 determines whether the color distortion will occur in the image signal and whether to perform the color gamut conversion based on the rate-distortion cost. For instance, the controller 209 determines whether the color distortion will occurs according to whether the differences between the color components of the image signal are less than the first critical level or the high frequency component of the image signal exceeds a second critical level. If it is determined that the color distortion will not occur, the controller 209 performs the color gamut conversion test to determine whether to perform the color gamut conversion. An encoder 1111 encodes and outputs the first residual signal or the second residual signal to which the domain transform and the quantization processing are performed, according to the determination of the controller 209 with respect to whether to perform the color gamut conversion.
As an additional example embodiment, as a result of the color gamut conversion test, if it is determined that the signal to noise ratio exceeds a predetermined critical level when the color gamut conversion is performed, the controller 209 may omit performing the color gamut conversion without the need to calculate the rate-distortion cost.
While various example embodiments have been illustrated and described with reference to various example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2016-0133595 | Oct 2016 | KR | national |
