Configurable and compact pixel processing apparatus

Abstract

An image processing apparatus for processing pixels is disclosed. The image processing apparatus comprises one or more functional blocks adapted to perform a corresponding functional task on the pixels. Further, the image processing apparatus includes one or more line-delay elements for delaying a horizontal scan line of the pixels. A desired processing task, which includes at least one functional task, is performed by configuring each functional block based on an actual number of the line-delay elements used for performing the desired processing task. Each functional block used for performing the desired processing task receives a group of pixels for processing from one or more horizontal scan lines such that the group overlaps another group of pixels for processing from one or more horizontal scan lines by another functional block.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments of the present invention generally relate to pixel processing. More particularly, embodiments of the present invention relate to configurable and compact pixel processing apparatus.

2. Related Art

The market for electronic products is divided into several sectors. Each market sector demands a specific range of performance within specific cost levels. As the market for electronic products evolves, it further divides into additional sectors. This gives rise to several issues. Since it is generally impractical to pursue all the market sectors, specific market sectors are selected. Once the specific market sectors are selected, electronic products are developed that meet the cost and performance demands of the selected market sectors.

In general, the electronic products are tailored for specific market sectors by its components. Low cost and low performance components are incorporated into electronic products for low cost and low performance market sectors. High cost and high performance components are incorporated into electronic products for high cost and high performance market sectors. Moreover, various versions of the same component may be manufactured for different market sectors.

SUMMARY OF THE INVENTION

An image processing apparatus for processing pixels is disclosed. The image processing apparatus comprises one or more functional blocks adapted to perform a corresponding functional task on the pixels. Further, the image processing apparatus includes one or more line-delay elements for delaying a horizontal scan line of the pixels. A desired processing task, which includes at least one functional task, is performed by configuring each functional block based on an actual number of the line-delay elements used for performing the desired processing task. Each functional block used for performing the desired processing task receives a group of pixels for processing from one or more horizontal scan lines such that the group overlaps another group of pixels for processing from one or more horizontal scan lines by another functional block.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the present invention.

FIG. 1 illustrates a device in accordance with an embodiment of the present invention.

FIG. 2A illustrates a processing arrangement of functional blocks in accordance with an embodiment of the present invention.

FIG. 2B illustrates a delay arrangement of line-delay elements for the processing arrangement of FIG. 2A in accordance with an embodiment of the present invention.

FIG. 3 illustrates pixel processing overlap arrangement in the processing arrangement of FIG. 2A in accordance with an embodiment of the present invention.

FIG. 4A illustrates an exemplary processing arrangement of functional blocks in accordance with an embodiment of the present invention.

FIG. 4B illustrates an exemplary delay arrangement of line-delay elements for the exemplary processing arrangement of FIG. 4A in accordance with an embodiment of the present invention.

FIG. 5 illustrates exemplary pixel processing overlap arrangement in the processing arrangement of FIG. 4A in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the preferred embodiments, it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention.

FIG. 1 illustrates a device 100 in accordance with an embodiment of the present invention. The device 100 includes an image capture unit 10 for generating an image comprising a plurality of horizontal scan lines of pixels and an image processing unit 20 for processing the pixels. The image capture unit 10 has a control unit 2, an image sensors unit 4, and a local processing unit 6. The image sensors unit 4 includes a plurality of image sensors. Examples of image sensors include CCD sensors and CMOS sensors. Under the control of the control unit 2, the image sensors unit 4 captures and outputs an image comprising a plurality of horizontal scan lines of pixels. The local processing unit 6 processes the pixels.

The device 100 may be a digital camera, a camera phone, a camera PDA (personal digital assistant), or any other type of device with imaging capability. Additionally, the device 100 may be a portable or handheld device.

As depicted in FIG. 1, the image processing unit 20 receives the pixels from the image capture unit 10. The image processing unit 20 processes the plurality of horizontal scan lines of pixels of the image. In particular, the image processing unit 20 is capable of performing various functional tasks on the pixels. Examples of functional tasks include defective pixel concealment, noise filtering, color interpolation, and edge enhancement. For real-time processing, the image processing unit 20 is able to utilize line-delay elements for delaying a horizontal scan line of pixels and pixel-delay elements. Since the cost of a line-delay element is much greater than the cost of a pixel-delay element and since performance is directly related to the number of line-delay elements utilized, the image processing unit 20 is configurable to meet various price and performance requirements. For example, the image processing unit 20 may have X number of line-delay elements. The image processing unit 20 may be configured to use X number of line-delay elements to provide high performance with a high price tag. Also, the image processing unit 20 may be configured to use any subset of the X number of line-delay elements to provide lower performance with a lower price tag.

Further, the image processing unit 20 provides a compact implementation by overlapping the various groups of pixels on which separate functional tasks are performed. This allows line-delay element to be shared by separate functional tasks.

FIG. 2A illustrates a processing arrangement 200 of functional blocks in accordance with an embodiment of the present invention. This processing arrangement 200 is implemented in the image processing unit 20. As shown in FIG. 2A, the processing arrangement 200 includes one or more functional blocks 210 and 220, wherein each functional block 210 and 220 is adapted to perform a corresponding functional task on the pixels. Examples of functional tasks include defective pixel concealment, noise filtering, color interpolation, and edge enhancement. Moreover, each functional block 210 and 220 comprises a configurable input and a configurable output.

Focusing on FIG. 2A, each symbol L represents a different horizontal scan line of pixels. The subscript (e.g., 1, k1, k2, k3, etc.) is used to enumerate the symbol L. Moreover, horizontal scan line L₁ to horizontal scan line L_k3 represent k3 consecutive horizontal scan lines. Generally, the image processed by the image processing unit 20 (FIG. 1) has more horizontal scan lines than the value k3. However, the image processing unit 20 concurrently processes k3 horizontal scan lines of the image.

In this processing arrangement 200, pixels from k3 consecutive horizontal scan lines are concurrently processed by functional blocks 210 and 220. In general, pixel processing is performed on a group of pixels, wherein the group includes the target pixel and pixels surrounding the target pixel. As input, the functional block 210 receives pixels from k2 consecutive horizontal scan lines, wherein the target pixel is in the input horizontal scan line L_k1. Moreover, the functional block 210 provides its processing result at the output 230 so that an updated pixel value Wk1 (FIG. 3) may be written to the corresponding input horizontal scan line L_k1 to form input horizontal scan line WL_k1 for use by the functional block 220.

In one embodiment, as input, the functional block 220 receives pixels from consecutive horizontal scan lines already processed by functional block 210. Thus, pixels from input horizontal scan line WL_k1 to input horizontal scan line L_k3 are received, wherein the target pixel is in the input horizontal scan line L_k2. Moreover, the functional block 220 provides its processing result at the output 240. Additionally, an updated pixel value may be written to another input horizontal scan line for use by another functional block.

In another embodiment, as input, the functional block 220 receives pixels from consecutive horizontal scan lines already processed by functional block 210 and not yet processed by functional block 210. For example, pixels from input horizontal scan line L_k0 (where k0 is greater than 1 and less than k3) to input horizontal scan line L_k3 may be received, wherein the target pixel is in the input horizontal scan line L_k2. Moreover, the functional block 220 provides its processing result at the output 240. Additionally, an updated pixel value may be written to another input horizontal scan line for use by another functional block.

It should be understood that a functional block may be divided into two or more functional blocks. Moreover, there may be more than two functional blocks. Further, a functional block may be bypassed by coupling an input horizontal scan line to a corresponding output of the functional block (e.g., coupling the input horizontal scan line L_k1 to output 230 of functional block 210 causes bypass of functional block 210). This makes possible multiple configurations for the processing arrangement 200.

FIG. 2B illustrates a delay arrangement 280 of line-delay elements 270 for the processing arrangement of FIG. 2A in accordance with an embodiment of the present invention. This delay arrangement 280 is implemented in the image processing unit 20. As depicted in FIG. 2B, delay arrangement 280 includes one or more line-delay elements 270 for delaying a horizontal scan line of the pixels. Since the processing arrangement 200 concurrently processes pixels from k3 (e.g., L₁ to L_k3) consecutive horizontal scan lines, the delay arrangement 280 has k3-1 line-delay elements 270. In an embodiment, each line-delay element 270 comprises a memory location in a RAM (random access memory). The output of functional block 210 is made available to functional block 220 by performing a write operation 275 to the input horizontal scan line L_k1 to form input horizontal scan line WL_k1 for use by the functional block 220.

FIG. 3 illustrates pixel processing overlap arrangement 300 in the processing arrangement 200 of FIG. 2A in accordance with an embodiment of the present invention. As depicted in FIG. 3, the group 310 of pixels from multiple horizontal scan lines processed by the functional block 210 overlaps the group 320 of pixels from multiple horizontal scan lines processed by the functional block 220. Moreover, the updated pixel value Wk1 is provided in the group 320 of pixels by performing a write operation as described above. This pixel processing overlap arrangement 300 leads to a compact implementation for the image processing unit 20 of FIG. 1. Moreover, the pixel processing overlap arrangement 300 facilitates efficient use of line-delay elements through sharing among the functional blocks 210 and 220 of FIG. 2A. Further, a subset of the line-delay elements may be utilized instead of all the line-delay elements 270. Thus, a desired processing task, which includes at least one functional task, is performed on pixels by configuring each functional block based on an actual number of line-delay elements used for performing the desired processing task.

FIG. 4A illustrates an exemplary processing arrangement 400 of functional blocks in accordance with an embodiment of the present invention. This exemplary processing arrangement 400 is implemented in the image processing unit 20 of FIG. 1. As shown in FIG. 4A, the exemplary processing arrangement 400 includes functional blocks 410 and 420, wherein functional block 410 is further divided into functional blocks 404 and 406. The functional block 404 performs defective pixel concealment while the functional block 406 performs noise filtering. Moreover, functional block 420 performs color interpolation. It should be understood that functional blocks 404, 406, and 420 may perform other functions.

In this exemplary processing arrangement 400, pixels from seven consecutive horizontal scan lines are concurrently processed by functional blocks 404, 406, and 420. In general, pixel processing is performed on a group of pixels, wherein the group includes the target pixel and pixels surrounding the target pixel. As input, the functional block 404 receives pixels from consecutive horizontal scan lines L₁ to L₅, wherein the target pixel is in the input horizontal scan line L₃. Moreover, the functional block 404 provides its processing result at the output 425 so that an updated pixel value A (FIG. 5) may be used by the functional block 406. As input, the functional block 406 receives pixels from consecutive horizontal scan lines L₁ to L₂ and L₄ to L₅ and output 425 of functional block 404, wherein the target pixel is in the input horizontal scan line L₃ which corresponds to the output 425 of functional block 404. Moreover, the functional block 406 provides its processing result at the output 430 so that an updated pixel value B (FIG. 5) may be written to the corresponding input horizontal scan line L₃ to form input horizontal scan line WL₃ for use by the functional block 420.

As input, the functional block 420 receives pixels from consecutive horizontal scan lines WL₃ to L₇, wherein the target pixel is in the input horizontal scan line L₅. Moreover, the functional block 420 provides its processing result at the output 440.

FIG. 4B illustrates an exemplary delay arrangement of line-delay elements for the exemplary processing arrangement of FIG. 4A in accordance with an embodiment of the present invention. This exemplary delay arrangement 480 is implemented in the image processing unit 20 of FIG. 1. As depicted in FIG. 4B, delay arrangement 480 includes six line-delay elements 470 for delaying a horizontal scan line of the pixels since the exemplary processing arrangement 400 concurrently processes pixels from seven (e.g., L₁ to L₇) consecutive horizontal scan lines. In an embodiment, each line-delay element 470 comprises a memory location in a RAM (random access memory). The output of functional block 406 is made available to functional block 420 by performing a write operation 475 to the input horizontal scan line L₃ to form input horizontal scan line WL₃ for use by the functional block 420.

FIG. 5 illustrates exemplary pixel processing overlap arrangement in the processing arrangement of FIG. 4A in accordance with an embodiment of the present invention. As depicted in FIG. 5, the group 504 of pixels from multiple horizontal scan lines processed by the functional block 404 overlaps the group 506 of pixels from multiple horizontal scan lines processed by the functional block 406. Moreover, the updated pixel value A is provided in the group 506 of pixels. Further, the group 506 of pixels from multiple horizontal scan lines processed by the functional block 406 overlaps the group 520 of pixels from multiple horizontal scan lines processed by the functional block 420. Moreover, the updated pixel value B is provided in the group 520 of pixels by performing a write operation as described above. This exemplary pixel processing overlap arrangement 500 leads to a compact implementation for the image processing unit 20 of FIG. 1. Moreover, the exemplary pixel processing overlap arrangement 500 facilitates efficient use of line-delay elements through sharing among the functional blocks 404, 406, and 420. Further, a subset of the line-delay elements may be utilized instead of all the line-delay elements 470. Thus, a desired processing task, which includes at least one functional task, is performed on pixels by configuring each functional block based on an actual number of line-delay elements used for performing the desired processing task. That is, an actual number of the line-delay elements are selected. Then, each functional block is configured based on the actual number of the line-delay elements so that to perform the desired processing task.

In an alternate configuration, the exemplary processing arrangement 400 may be changed so that the functional block 420 operates on three consecutive horizontal scan lines instead of five consecutive scan lines, eliminating the need for two of the line-delay elements and L₆ and L₇.

In yet another configuration, the exemplary processing arrangement 400 may be changed so that functional blocks 404 and 406 are bypassed (e.g., coupling the input horizontal scan line L₃ to output 425 of functional block 404 causes bypass of functional block 404, coupling the output 425 of functional block 404 to output 430 of functional block 404 causes bypass of functional block 406) and so that there is no need to perform a write operation by coupling inputs L₁ through L₅ to inputs L₃ through L₇ respectively, eliminating the need for two of the line-delay elements.

In still another configuration, the exemplary processing arrangement 400 may be changed so that functional blocks 404 and 406 are bypassed (e.g., coupling the input horizontal scan line L₃ to output 425 of functional block 404 causes bypass of functional block 404, coupling the output 425 of functional block 404 to output 430 of functional block 404 causes bypass of functional block 406), so that there is no need to perform a write operation, and so that the functional block 420 operates on three consecutive horizontal scan lines instead of five consecutive scan lines by coupling inputs L₁ through L₃ to inputs L₄ through L₆ respectively, eliminating the need for four of the line-delay elements.

As a result, either performance or price tag may be emphasized depending on a given application.

The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the Claims appended hereto and their equivalents.

Claims

1. An image processing apparatus for processing pixels, comprising: a plurality of functional blocks adapted to perform a corresponding functional task on said pixels arranged into horizontal scan lines, wherein said functional blocks operate concurrently on said pixels, wherein a first functional block of said functional blocks operates on a first set of consecutive horizontal scan lines while a second functional block of said functional blocks operates concurrently on a second set of consecutive horizontal scan lines, wherein said first set includes an overlapping portion of consecutive horizontal scan lines with respect to said second set and a non-overlapping portion of consecutive horizontal scan lines with respect to said second set, wherein said second functional block is configured to receive an output of said first functional block; anda plurality of line-delay elements for delaying a respective horizontal scan line of said pixels.

2. The image processing apparatus as recited in claim 1 wherein said functional task is one of defective pixel concealment, noise filtering, color interpolation, and edge enhancement.

3. The image processing apparatus as recited in claim 1 wherein each functional block further comprises a configurable output.

4. The image processing apparatus as recited in claim 1 wherein a write operation makes available an output of a functional block to another functional block.

5. The image processing apparatus as recited in claim 1 wherein a functional block is bypassed by coupling said configurable input that receives said target pixel to an output of said functional block.

6. The image processing apparatus as recited in claim 1 wherein said horizontal scan lines are consecutive horizontal scan lines, wherein said functional blocks are configured to perform said corresponding functional tasks by using a first subset of said line-delay elements instead of said plurality of line-delay elements, wherein said functional blocks are configured to perform said corresponding functional tasks by using a second subset of said line-delay elements instead of said plurality of line-delay elements, wherein each line-delay element comprises a memory location in a RAM (random acess memory), wherein each functional block comprises a plurality of configurable inputs, wherein for each functional block said configurable inputs are selectively coupled to a subset of said line-delay elements to receive said respective horizontal scan lines of said pixels, wherein for each functional block at least one of said configurable inputs receives said respective horizontal scan line including a target pixel, wherein for each functional block at least one of said configurable inputs receives said respective horizontal scan line including a pixel adjacent to said target pixel, and wherein each subset of said line-delay elements associated with said functional blocks overlaps another subset of said line-delay elements.

7. A method of processing pixels, said method comprising: providing a plurality of functional blocks, which operate concurrently on said pixels, in an image processing apparatus, each functional block adapted to perform a corresponding functional task on said pixels arranged into horizontal scan lines, wherein a first functional block of said functional blocks operates on a first set of consecutive horizontal scan lines while a second functional block of said functional blocks operates concurrently on a second set of consecutive horizontal scan lines, wherein said first set includes an overlapping portion of consecutive horizontal scan lines with respect to said second set and a non- overlapping portion of consecutive horizontal scan lines with respect to said second set, wherein said second functional block is configured to receive an output of said first functional block; andproviding a plurality of line-delay elements in said image processing apparatus, each line-delay element for delaying a respective horizontal scan line of pixels.

8. The method as recited in claim 7 wherein said functional task is one of defective pixel concealment, noise filtering, color interpolation, and edge enhancement.

9. The method as recited in claim 7 wherein each functional block further comprises a configurable output, wherein said functional blocks are configured to perform said corresponding functional tasks by using a first subset of said line-delay elements instead of said plurality of line-delay elements, wherein said functional blocks are configured to perform said corresponding functional tasks by using a second subset of said line-delay elements instead of said plurality of line-delay elements.

10. The method as recited in claim 7 further comprising: performing a write operation to make available an output of a functional block to another functional block.

11. The method as recited in claim 7 further comprising: bypassing a functional block by coupling said input that receives said target pixel to an output of said functional block.

12. The method as recited in claim 7 wherein said horizontal scan lines are consecutive horizontal scan lines, and further comprising: for each functional block, configuring at least one input to receive said respective horizontal scan line including a target pixel; andfor each functional block, configuring at least one input to receive said respective horizontal scan line including a pixel adjacent to said target pixel, wherein each subset of said line-delay elements associated with said functional blocks overlaps another subset of said line-delay elements.