Patent Application
20060158677
1. Field of the Invention
The present invention relates to image processing, and more specifically to enhancing image data processing by reducing the amount of header information generated for a stream of image data.
2. Description of the Related Art
Image data processing includes receiving image data and generating header information related to the image data. For example, the header information can include information regarding compression of the image data.
If a camera captures image data by frames, then an image data processing engine may encode one frame of image data and generate header information for the frame after the image data capture. However, if the image data processing engine does not encode the frame of image data fast enough, then this may cause a downstream delay with a processor in communication with the image data processing engine. For example, the processor can only decode the image data as fast as the image processing engine can encode the image data. On a desktop computer, such delays may be negligible because the desktop computer typically has one or more image processing engines and large memories that can buffer mass quantities of image data. However, in some devices, the delays are not negligible. Specifically, constraints such as small memories and limited battery power require minimal delays during the operation of some devices, such as mobile handheld devices. Additionally, devices without these constraints may have downstream processing delays because of the current encoding ratio.
Accordingly, what is needed is a method and apparatus for enhancing image data processing in devices having limited computing resources to improve processing times during image data encoding and decoding.
Broadly speaking, the present invention is a method and apparatus for enhanced image data processing to improve an encoding ratio of a stream of image data. It should be appreciated that the present invention can be implemented in numerous ways, such as a process, an apparatus, or a device, of which several inventive embodiments of the present invention are described below.
An embodiment of a method of processing image data includes generating header information of a first frame of image data. The header information is stored in a first buffer at a first location and a second buffer at a corresponding second location. The method also includes encoding the frame of image data for storage in the first buffer at a third location as compressed image data. Further, the method includes transmitting the stored header information and the compressed image data for decoding from the first buffer, such that subsequent compressed image data without generating header information is stored in the first buffer at the third location or the second buffer at a corresponding fourth location. The transmitting includes the stored header information and the subsequent compressed image data.
In an embodiment of a method of encoding image data, the method includes receiving a first frame of image data and generating header information of the first frame of image data. The method also includes storing both the first frame of image data as encoded image data and the header information in a first buffer. Further, the method includes disabling the generation of header information, thereby storing subsequent frames of image data without generating header information.
In an embodiment of a graphics controller for processing image data, the graphics controller includes an image data encoder configured to receive and compress a first frame of image data. The graphics controller also includes a memory coupled to the image data encoder. The memory has a first buffer and a second buffer, such that generated header information of the first frame of image data is stored in a first portion of the first buffer and the second buffer. Further, the graphics controller includes an offset portion of the first buffer and the second buffer. The offset portion of the first buffer is adjacent to the first portion of the first buffer and the offset portion of the second buffer is adjacent to the first portion of the second buffer. The offset portion includes a compressed subsequent frame of image data without generated header information of a subsequent frame of image data.
In an embodiment of a computer program embodied on a computer readable medium for processing image data, the computer program includes instructions for generating header information of a first frame of image data. The header information is stored in a first buffer at a first location and a second buffer at a corresponding second location. Further, the computer program includes instructions for encoding the frame of image data for storage in the first buffer at a third location as compressed image data. Moreover, the computer program includes instructions for transmitting the stored header information and the compressed image data for decoding from the first buffer. Subsequent compressed image data without generating header information is stored in the first buffer at the third location or the second buffer at a corresponding fourth location. The transmitting includes the stored header information and the subsequent compressed image data.
In an embodiment of a method for enabling an electronic device to more efficiently process image data, the method initiates with establishing compression parameters to be applied to compress the image data. The method includes transmitting the compression parameters simultaneously to an image capture sensor and a graphics engine of the device and transferring a data unit of compressed image data from the image capture sensor to the graphics engine without including a header defining the compression parameters. The method also includes decoding the data unit through the compression parameters of the graphics engine.
In a method for efficiently processing captured image data through a device, the method initiates with compressing the captured image data. A header is applied to a first data unit of the compressed captured image data through logic associated with an image capture sensor. The first data unit of the compressed captured image data is transmitted to a graphics controller and subsequent data units of the compressed captured image data are transmitted to the graphics controller without any header information. Compression parameters are extracted from the header information of the first data unit and stored in memory associated with the graphics controller. The first data unit and the subsequent data units are decoded with the compression parameters.
In an image capture device, the device includes an image capture sensor capable of compressing captured image data. A graphics controller configured to program the image capture sensor with compression parameters upon initialization of the device is included. The graphics controller is further configured to load the compression parameters into a decode module of the graphics controller, thereby enabling the image capture sensor to transmit the compressed captured image data without header information.
Other embodiments of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.
Exemplary embodiments of the invention may be understood by reference to the following description, taken in conjunction with the accompanying drawings in which:
The following embodiments describe a method and apparatus for enhanced image data processing. For example, mobile devices with limited resources such as battery powered devices having relatively small memories can reduce power consumption and processing delays by reducing the encoding ratio of image data through the embodiments described below. The embodiments described herein generate a single header and subsequently process multiple frames of image data without generating additional header information. Thus, by reducing the generated header information, exemplary embodiments can improve the encoding ratio for image data processing. Although the embodiments described herein are described in relation to mobile devices, other embodiments can include any device for performing enhanced data image processing. It will be obvious, however, to one skilled in the art, that the present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.
It should be appreciated that the compressed image data can have many suitable formats after encoding. Exemplary formats include joint photographic experts group (JPEG), graphics interchange format (GIF), portable network graphics (PNG), tagged image file format (TIFF), and bit-mapped graphics (BMP), etc. In accordance with an exemplary embodiment,
The device 300 includes a controller 310, a processor 370, and an image capture device 380. In other exemplary embodiments, the image capture device 380 may be external to the device 300. Exemplary image capture devices include a charged coupled device and a complementary metal oxide semiconductor sensor. The controller 310, the processor 370, and the image capture device 380 are coupled together to propagate image data from the image capture device 380 to the controller 310. However, the connections illustrated in
In yet another embodiment, the device 300 may not receive image data from the image capture device 380. For example, the device 300 may receive image data from a wireless connection to another device. Thus, one of ordinary skill in the art will recognize that the device can receive image data from any source as long as the device 300 is capable of achieving an encoding ratio of approximately 1:1.
Image data is processed by and stored in the memory 330 of the controller 310. Thereafter, the processor 370 decodes image data stored in the controller 310 by retrieving the encoded image data from the memory 330. To decode the stored image data, the processor 370 uses the header information, such as the header-1210 (
The memory 330 receives encoded image data generated by the codec 340. In one embodiment, the codec 340 is a coder/decoder including circuitry and logic capable of generating the header information of the frame of image data and encoding the frame of image data. In another embodiment, the codec 340 can also be a compression/decompression engine configured to compress and decompress a frame of image data. In yet other embodiments, the encoder/decoder includes the capability to produce a compressed frame of image data. Thus, in one exemplary embodiment, the codec 340 encodes frames of image data as JPEG. However, the codec 340 can produce any graphics format of encoded image data for storage in the memory 330. In other exemplary embodiments, the codec 340 can be an encoder without decoding capability, as long as the codec 340 generates the header information and encodes frames of image data.
The resizer 350 coupled to the codec 340 crops and scales image data. Thus, the resizer 350 provides frames of image data to the codec 340. The frame of image data is an array of pixels wherein the pixels can represent any color scheme. An exemplary color scheme is RGB888, which uses 8 bits to represent each color value of the pixel. In other exemplary embodiments, other color schemes are possible, as long as the pixels can be organized as frames of image data. The image capture device interface 360, coupled to the resizer 350, is capable of managing the stream of image data from the image capture device 380. As mentioned above, the image capture interface 360 can receive the stream of image data from any suitable source.
The stream of image data includes frames of image data that can be stored in the memory 330. The memory includes buffers, wherein the buffers have identifiable locations organized by memory addresses. The memory addresses identify starting and ending addresses of locations that can store the frames of image data. Thus, in an exemplary embodiment, when the codec 340 generates the header-1210 (
In an exemplary embodiment, the codec 340 ceases to generate further header information because a signal (not shown) to the codec 340 has been disabled. For example, the signal can indicate whether or not the codec 340 should generate header information. When enabled, the signal can indicate that the codec 340 should generate header information. When disabled, the lack of the signal indicates that the codec 340 should not generate header information. One of ordinary skill in the art will recognize that an alternative embodiment can include an enable signal that indicates that the header information should not be generated. Consequently, a disable signal indicates that the header should be generated. Thus, any suitable method of signaling to the codec 340 to cease generating header information is possible.
As soon as the header and the encoded image data are stored in the buffer-1410, the processor 370 of
In another exemplary embodiment, where only one header is generated for the stream of encoded data, both the header information and the encoded image data are stored in the buffers.
Still referring to
The codec 340 then generates the encoded image data-1, which is stored in the buffer-1510 at a third location 530 in
To permit the retrieval of the data, transmission logic of the codec 340 is capable of transmitting the header information and encoded image data to the processor 370 of
For subsequent frames of image data, the codec 340 avoids generating header information. Thus, when a second frame of image data arrives at the codec 340, the encoded image data-2 is stored in an available buffer. An example of storing subsequent frames of encoded image data is illustrated in
Specifically, the method of
In operation 650, the graphics controller receives another frame of image data when there are additional frames of image data to encode. Then, in operation 660, if header generation is disabled, the method returns to operation 620. However, if header generation is not disabled, then in operation 670, header generation is disabled in an encoder of the graphics controller. Thus, subsequent frames of image data after a first frame of image data will not have header information generated.
In operation 825, the graphics controller stores the encoded image data in a buffer. In one embodiment, to accomplish the storage of the encoded image data, the previously mentioned logic is capable of determining an offset address from a start address of the buffer. Consequently, the encoded image data is stored at an offset portion, such that the beginning of the offset portion is identified by the offset address, as illustrated with reference to
If in operation 830, the graphics controller determines that there is no further image data, then the method ends. Alternatively, in operation 835, the graphics controller determines if header information generation is disabled for subsequent frames of image data. If header information generation is disabled, then the method proceeds to operation 845. Otherwise, in operation 840, the graphics controller disables header information within the encoder of the graphics controller. In operation 845, the graphics controller decides whether an offset is determined. In one embodiment, the offset is equivalent to the header information size generated from the first frame of image data. If the offset is not determined, then in operation 850, the graphics controller determines the offset in order to identify the beginning address of an offset portion used to store encoded image data. Thereafter, the method proceeds to operation 855. If the offset is determined in operation 845, then the method proceeds to operation 855, where the graphics controller receives a frame of image data. In operation 860, the encoder encodes the frame of image data and then stores the encoded image data in operation 825. The method then repeats as described above.
In operation 910, the processor retrieves the header information and encoded image data from a buffer. In an exemplary embodiment, the header information is the header information generated from the first frame of image data received by the graphics controller in
It should be appreciated while the above embodiments discuss the encoding side of the image processing scheme, the invention may be extended to the decoding side also. As the image capture sensors are becoming capable of capturing more information, e.g., 3 megapixels and up, the image capture sensors may be compress the information. This allows for the information to be efficiently transferred from the image capture sensor to the graphics processor, especially in a device having limited resources.
For example, mobile communications manufacturers are developing hand-held electronic devices with camera capability in which the image capture sensors compress an image for transfer to a mobile graphics engine (MGE). One common compression format used for the compression of the image is the JPEG standard. The JPEG standard consists of a header and the actual encoded data for each block of data. Thus, a JPEG decoder of the graphics processor has to decode the header, update quantization and Huffman tables even though the values for those tables have not changed between each JPEG data unit. In order to more efficiently decode the compressed data the graphics processing chip may be programmed with the compression parameters for the quantization table and Huffman tables simultaneously with the programming of the image sensor.
In an alternative embodiment, rather than program the compression parameters when programming the image capture sensor 950, the compression parameters may be transferred with a first frame of image data sent by the image capture sensor to the graphics controller 952. Here, circuitry within image capture sensor 950 may be included to apply a header to a first data unit of image data sent by the image capture sensor to the graphics controller. Thereafter, the circuitry does not apply a header to subsequent blocks of data. Graphics controller 952 then extracts the compression parameters, i.e., populates the quantization tables and Huffman tables with the compression parameters from the first header, and utilizes these compression parameters to decode the compressed image data.
The method of
As an alternative to populating the tables of the graphics controller with the compression parameters when programming the image capture sensor, the data may be sent with a header applied to a first data unit of the JPEG data stream. Thereafter, the data is transmitted without headers unless the compression parameters are changed or the unit is powered off. As mentioned above with reference to
It should be appreciated that logic of the present invention includes software or hardware embodiments. Further, embodiments of the present invention may be practiced with various computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers and the like. The invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.
With the above embodiments in mind, it should be understood that the invention can employ various computer-implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared and otherwise manipulated.
Any of the operations described herein that form part of the invention are useful machine operations. The invention also relates to a device or an apparatus for performing these operations. The apparatus can be specially constructed for the required purpose, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
The invention can also be embodied as computer readable code on a computer readable medium. The computer readable medium is any data storage device that can store data, which can be thereafter read by a computer system. The computer readable medium also includes an electromagnetic carrier wave in which the computer code is embodied. Examples of the computer readable medium include hard drives, network attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications can be practiced within the scope of the appended claims. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
