This invention relates to video processing and more specifically to classifying and segmenting regions of pixels base upon characteristics such as color and texture.
A method and system for improving the quality of a video image segmented into a plurality of blocks of known size is disclosed. The method comprises the steps of associating a value to each of said blocks and altering said associated value corresponding to a selected one of said blocks when each of said associated values of blocks adjacent to said selected block is different than said selected block associated value.
In the drawings:
It is to be understood that these drawings are solely for purposes of illustrating the concepts of the invention and are not intended as a definition of the limits of the invention. The embodiments shown in
Segmentation of video images, such as television images, is the process wherein each frame of a sequence of images is subdivided into regions or segments. Each segment includes a cluster of pixels that encompass a region of the image with common properties or characteristics. For example, a segment may be distinguished by a common color, texture, shape, amplitude range or temporal variation. Several methods are known for image segmentation using a process wherein a binary decision determines how the pixels will be segmented. According to such a process, all pixels in a region either satisfy a common criteria for a segment and are therefore included in the segment, or they do not satisfy the criteria and are completely excluded: While these segmentation methods are satisfactory for some purposes, they are unacceptable for many others. In the case of moving image sequences, small changes in appearance, lighting or perspective may only cause small changes in the overall appearance of the image. However, application of a segmentation method such as that described above tends to allow regions of the image that should appear to be the same to satisfy the segmentation criteria in one frame, while failing to satisfy it in another. One of the main reasons for segmenting images is to conduct enhancement operations on the segmented portions. When the image is segmented according to a binary segmentation method such as that previously described, the subsequently applied enhancement operations often produce random variations in image enhancement, usually at the edges of the segmentation regions. Such random variations in moving sequences represent disturbing artifacts that are unacceptable to viewers. Image enhancement in the television setting includes both global and local methods. While local enhancement methods are known, they are currently controlled by global parameters. For example, an edge enhancement algorithm may adapt to the local edge characteristics, but the parameters that govern the algorithm (i.e., filter frequency characteristics) are global—the enhancement operations that are applied are the same for all regions of the image. The use of global parameters limits the most effective enhancement that can be applied to any given image. Improved enhancement would be available if the algorithm could be trained to recognize the features depicted in different segments of the image and could therefore allow the image enhancement algorithms and parameters that are optimum for each type of image feature to be chosen dynamically.
However, one of the principle problems with the current state of the art is that it is essentially pixel-based. As the characteristics such as color and luminance within a segment may vary significantly from pixel to pixel, the determined segment probability function may include significant “noise-like” indicators. When the input video signal also includes noise, the resultant segment probability function becomes even more noise-like. One method of reducing the noise-like indicators in the probability distribution is to process it using a low-pass filter. However, such processing has the undesirable side-effect of removing the texture in the segment of the image.
Hence, there a need for a method and system for reducing the effects of the noise in the determined segment probability function, while maintaining the image texture.
As is known, video images may have significant areas or segments that may be identified as having substantially the same characteristics, e.g., color, luminosity, texture. For example, a segment of an image may contain information related to a sky, i.e., blue color, smooth texture. Similarly, fields of grass may be identified by its green color and semi-smooth texture. Such identification of areas, or segments of video images are more fully discussed in commonly assigned, co-pending related patent application Ser. No. ______ and commonly assigned, co-pending related patent application Ser. No. ______, which disclose determining a probability function for each such segment identified.
Furthermore, it would be understood that the block size need not be symmetrical as shown, but may contain any number of pixel elements in either length or width. Only for the purposes of clearly illustrating and discussing the present invention, are the image pixel elements of the selected segment grouped into 8×8 blocks, represented as blocks 110-180.
However, if the answer at block 360 is in the affirmative, then a determination is made at block 370 whether the value of the block selected at block 310 is substantially similar to the value of the adjacent block selected at block 320. If the answer is in the affirmative, then processing on the block selected at block 310 is completed. However, if the answer is negative, then the value of the block selected at block 310 is altered to correspond to the value of the adjacent block selected at block 320. Accordingly, the anomaly value associated with the selected is removed and made comparable to the values of the adjacent blocks.
For example, a block associated with a logical zero value may have all of its associated adjacent pixel blocks having an opposite value of logical one. In this, case, the block associated with the anomalous logical zero value is “removed” by setting its associated value to a logical one value, similar to all the adjacent block associated value. Similarly, if a block with an isolated logical one value is surrounded by blocks associated with a logical zero value, the anomalous logical one value is removed by setting the value to a logic zero.
Returning to
In one aspect of the invention, the value associated with each block may then be used to control the processing that is to be done for each pixel within the block. For example, one form of pixel-level processing that may be performed is determine whether a noise filter must be turned on during the processing of each pixel in the block. This method is advantageous to strike a balance between reduced image noise and maintaining appropriate textual information. In another aspect, the values associated with each block may be used to control forms of processing such as modifying the edge sharpness or color of a region differently than other regions.
Input/output devices 402, processors 403 and memories 404 may communicate over a communication medium 406. Communication medium 406 may represent, for example, a bus, a communication network, one or more internal connections of a circuit, circuit card or other apparatus, as well as portions and combinations of these and other communication media. Input data from the sources 410 is processed in accordance with one or more software programs that may be stored in memories 404 and executed by processors 403. Processors 403 may be any means, such as general purpose or special purpose computing system, or may be a hardware configuration, such as a laptop computer, desktop computer, handheld computer, dedicated logic circuit, integrated circuit, Programmable Array Logic (PAL), Application Specific Integrated Circuit (ASIC), etc., that provides a known output in response to known inputs.
In one embodiment, the coding and decoding employing the principles of the present invention may be implemented by computer readable code executed by processor 403. The code may be stored in the memory 404 or read/downloaded from a memory medium such as a CD-ROM or floppy disk (not shown). In another and preferred embodiment, hardware circuitry may be used in place of, or in combination with, software instructions to implement the invention. For example, the elements illustrated herein may also be implemented as discrete hardware elements or as programmable devices operable to execute coed.
After processing the input data, processor 403 may cause the processed data to be transmitted to television display 480 or monitor 490 via network 470. As will be appreciated, networks 420 and 440 may be an internal network among the components, e.g., ISA bus, microchannel bus, PCMCIA bus, etc., or an external network, such as a Local Area Network, Wide Area Network, POTS network, or the Internet.
In one aspect of the invention, the term computer or computer system may represent one or more processing units in communication with one or more memory units and other devices, e.g., peripherals, connected electronically to and communicating with the at least one processing unit. Furthermore, the devices may be electronically connected to the one or more processing units via internal busses, e.g., ISA bus, microchannel bus, PCI bus, PCMCIA bus, etc., or one or more internal connections of a circuit, circuit card or other device, as well as portions and combinations of these and other communication media or an external network, e.g., the Internet and Intranet.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB03/05794 | 12/5/2003 | WO | 8/17/2005 |
Number | Date | Country | |
---|---|---|---|
60433310 | Dec 2002 | US |