Patent Application
20070177025
The invention relates generally to a mobile imaging device comprising at least two integrated cameras.
Dual-camera mobile devices, such as mobile telephones (also called mobile phones and cell phones), are known in the art and increasing in popularity. One camera functions as a still image camera, and the other as a video camera. The video camera is used for video conferencing applications, and would typically face the user during operation. The still image camera may be used for taking still images at typically higher resolution than the video camera. Many implementations use a separate processor for each camera, which have the disadvantage of requiring additional system cost and greater area/volume within the mobile device (i.e. mobile telephones).
Solid state imaging devices having pixel arrays, including charge coupled imaging devices (CCD) and complementary metal oxide semiconductor (CMOS) imaging devices, and other solid state imaging devices are commonly used in photo-imaging applications, such as the cameras described above.
One implementation of a dual-camera single-processor mobile telephone is discussed in U.S. Patent Application No. 2005/0036046 to Atsum, filed Feb. 17, 2005 (hereinafter “Atsum”). In Atsum, two cameras are connected to a single processor, but the images are taken contemporaneously, and the processing of the two images is interleaved in the processor by breaking down each image into “stripes” and alternately processing one stripe at a time from each image. The disadvantage of this type of processing is that it requires additional processing and decoding to separate and identify the different images. Atsum requires an additional file to be created to identify how large each “stripe” of data will be, as well as an additional steps of compressing the data, and then uncompressing the data before processing. The encoder and decoder of the Atsum device adds to the complexity, size, and cost of the mobile telephone.
As dual-camera mobile devices, such as telephones, increase in their popularity there is an obvious need and desire for simplified system solutions that reduce system cost and minimize the area/volume occupied by such a dual-camera solution within the mobile device.
In accordance with exemplary embodiments of the invention, a method and implementing apparatus minimizing die area and module size for a dual-camera mobile device includes a first image sensor, an image processor coupled to and residing on the same integrated circuit as the first image sensor, and a second image sensor, wherein the image processor is configured to substantially and separately process images from the first and second image sensors.
The foregoing and other advantages and features of the invention will be more readily understood from the following detailed description of the invention provided below with reference to the accompanying drawing, in which:
In the following detailed description, reference is made to the accompanying drawings, which form a part hereof and show by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized, and that structural, logical, and electrical changes may be made without departing from the spirit and scope of the present invention. The progression of processing steps described is exemplary of the embodiments of the invention; however, the sequence of steps is not limited to that set forth herein and may be changed as is known in the art, with the exception of steps necessarily occurring in a certain order.
In a dual-camera telephone implementation in accordance with the invention, only one camera needs to operate at a time. Since the image processing is similar for both camera image sensors, the image processing can be shared. This sharing is accomplished by using a first sensor having associated on-chip processing and a second sensor which does not have associated on-chip processing. Image data from the second camera is bridged via a serial or parallel interface to the first camera, which performs the processing for the second sensor.
The mobile device of the invention minimizes system cost by eliminating the replication of on-chip image processing or complicated simultaneous processing of images from two sensors. In a typical dual-camera telephone there are two on-chip processors, each of which contains the full suite of image processing algorithms to translate the raw, e.g. Bayer pattern output of the sensor, into, e.g., a quality luminance/chrominance (YUV) or a compressed image data output. In the proposed solution the on-chip processing is only present for the first sensor's chip but this processing is also selectively usable by the second sensor. This results in a die area savings for a two-sensor mobile device.
First camera module 110 is directed toward the back of mobile device 100, and may be used as a still image or video camera. First camera module 110 may typically range from 1.3 megapixels to 5 megapixels in resolution, although both higher and lower resolutions may be used to practice the invention. First camera module 110 is formed on chip 150. On-chip image processor 130 is also formed on chip 150. On-chip image processor 130 performs the requisite image and other processing on the image data stream from first camera 110.
Second camera module 120 is directed toward the front of mobile device 100, typically facing the user, and may be used as a video or still image camera. In a desired application, the second camera module 120 may be used for video conferencing. Second camera module 120 may be optimized for video conferencing applications by maximizing the low light performance of the module 120. VGA is typically the maximum resolution required to use second camera module 120 for video conferencing applications, although the invention is not intended to be limited to VGA output. Second camera module 120 is connected to on-chip image processor 130 via bridge interface 140, which may be either a serial or parallel connection, although other connections may also be used. On-chip image processor 130 also includes control mechanisms to receive pixel data from the second camera module 120 and to perform the requisite processing on the image data stream from second camera module 120.
On-chip image processor 130 typically controls the second camera module 120 using an optional bidirectional serial control interface 165. However, when bridge interface 140 is a high-speed serial interface, mobile device 100 may also use bridge interface 140 to control the second camera module 120, provided the bridge interface 140 supports bidirectional data transfers, in which case optional bidirectional serial control interface 165 would be unnecessary. The on-chip image processor 130 may output via a data interface 170 to the mobile device baseband processor 160 a processed image from first camera module 110 or the processed image or image stream relayed and processed from second camera module 120.
Baseband processor 160 is a communications control processor for a mobile device which performs the general functions of mobile device 100. For example, if mobile device 100 is a mobile telephone, baseband processor 160 may generate the display, process incoming and outgoing calls, maintain an address book, and other associated functions. Functions such as focus, zoom, and white balancing for camera modules 110, 120 may be controlled by baseband processor 160 or on-chip image processor 130, depending on the phone architecture.
For increased flexibility, the on-chip image processor 130 may be equipped with both high-speed serial and parallel interfaces such that if bridge interface 140 is a parallel interface, then data interface 170 is a high-speed serial interface, or vice versa. However, a preset serial or parallel interface for bridge interface 140 or data interface 170 may also be used.
Since camera module 120 does not need an associated processor, it has a much smaller size than chip 150, which comprises camera module 110 and associated on-chip image processor 130. For some flip-type mobile telephone applications this has the benefit of requiring fewer interface wires to run through the phone hinge, simplifying the flex cable and reducing electromagnetic interference. The interface of baseband processor 160 to the dual-camera unit 155 may be simplified by having only one interface that receives image data from the two cameras. This would also reduce the cost of baseband processor 160.
The invention further includes methods of forming and operating a multi-camera unit of the embodiment illustrated in the figures. The method of operating the dual-camera unit 155 for a mobile device 100 comprises acquiring a first image with a first image sensor 110, processing the first image with an image processor 130 residing on the same integrated circuit 150 as said first image sensor 110. The method further comprises acquiring a second image with a second image sensor 120, transmitting the second image to the image processor 130 over a bridge interface 140, and processing the second image with the processor 130. The method may further include sending the processed image from on-chip image processor 130 to baseband processor 160 for handling in accordance with a user preference, entered at keypad 182.
The method of forming the multi-camera unit 155 comprises providing a first image sensor 110, providing an image processor 130 connected to and residing on the same integrated circuit 150 as the first image sensor 110 for processing images, pro viding a bridge interface 140 connected to the image processor 130, and providing a second image sensor 120 connected to the bridge interface 140. The second image sensor 120 outputs images to the processor 130 over the bridge interface 140. The image processor 130 is configured to substantially and separately process images from the first and second image sensors 110, 120.
While the invention has been described in detail in connection with exemplary embodiments known at the time, it should be readily understood that the invention is not limited to such disclosed embodiments. Rather, the invention can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the invention. For example, the mobile device 100 may be a personal digital assistant (PDA), mobile telephone, e-mail device, or other mobile communication device, and the camera modules 110, 120 are not limited to a still and a video camera. Non-limiting examples of camera modules 110, 120 usable in accordance with the invention are, respectively, Micron Part Nos. MI-SOC3130 and MI-380. In addition, the invention is not limited to the type of connections presented in the description of bridge interface 140.
Thus, the invention is not to be seen as limited by the foregoing description, but is only limited by the scope of the appended claims.
