The present invention relates to a method and apparatus for streaming data from multiple devices over a single data bus. More particularly, the invention relates to graphics display systems comprising a graphics controller for interfacing multiple cameras to a display device.
Graphics display systems, such as mobile or cellular telephones, typically employ a graphics controller as an interface between one or more providers of image data and a graphics display device such as an LCD panel or panels. In a mobile telephone, the providers of image data are typically a host, such as a CPU, and a camera. The host and camera transmit image data to the graphics controller for ultimate display on the display device. The host also transmits control data to both the graphics controller and the camera to control the operation of these devices.
The graphics controller provides various processing options for processing image data received from the host and camera. For example, the graphics controller may compress or decompress, e.g., JPEG encode or decode, incoming or outgoing image data, crop the image data, resize the image data, scale the image data, and color convert the image data according to one of a number of alternative color conversion schemes. All these image processing functions provided by the graphics controller are responsive to and may be directed by control data provided by the host.
The host also transmits control data for controlling the camera to the graphics controller, the graphics controller in turn programming the camera to send one or more frames of image data acquired by the camera to the graphics controller. Where, as is most common, the graphics controller is a separate integrated circuit or “chip,” and the graphics controller, the host, and the camera are all remote from one another, instructions are provided to the camera, and image data from the camera are provided to the graphics controller for manipulation and ultimate display, through a camera interface in the graphics controller.
Often, cellular telephones include two cameras. For example, it may be desirable to use one camera to image the user of the telephone while a call is being placed, and to use another camera to image scenery or other objects of interest that the caller would like to transmit in addition to his or her own image. In such cellular telephones, two camera interfaces are provided in the graphics controller.
The graphics controller cannot process parallel streams of data from multiple cameras, so that only one camera interface can be active at a given time. However, even an inactive camera interface consumes power. Therefore, as the present inventors have recognized, there is a need for a method and apparatus for streaming data from multiple devices over a single data bus.
A method for streaming data from multiple devices over a single data bus comprises causing first and second data streams produced respectively by first and second devices to be synchronized, and inserting into each of the data streams a plurality of corresponding high impedance states to form respective modified data streams in such a manner that the data corresponding to one of the modified data streams is present at the same time that another of the modified data streams is in a high impedance state, and superimposing the modified data streams on the bus for selecting the data.
An apparatus for streaming data from multiple devices comprises a clock source for synchronizing first and second data streams produced respectively by two of the devices. The apparatus also includes a switching circuit for inserting into the first data stream a plurality of high impedance states to form a first modified data stream, and for inserting into the second data stream a plurality of high impedance states to form a second modified data stream. Additionally, the apparatus includes a controller for controlling the switching device in such manner that data corresponding to one of the first and second modified data streams is present at the same time that the other of the first and second modified data streams is in a high impedance state. Preferably, the apparatus also includes a bus for receiving the modified first and second data streams in superimposition.
Embodiments of the invention are also directed to systems which employ methods and apparatus for streaming data from multiple devices over a single data bus.
This summary is provided only for generally determining what follows in the drawings and detailed description. This summary is not intended to fully describe the invention. As such, it should not be used limit the scope of the invention. Objects, features, and advantages of the invention will be readily understood upon consideration of the following detailed description taken in conjunction with the accompanying drawings.
Embodiments of the invention relate generally to methods and apparatus for streaming data from multiple devices over a single data bus. Particular embodiments pertain more particularly to graphics display systems comprising a graphics controller for interfacing multiple cameras to a display device; however, it should be understood that the principles described have wider applicability. One preferred graphics display system is a mobile telephone, wherein the graphics controller is a separate integrated circuit from the remaining elements of the system, but it should be understood that graphics controllers according to the invention may be used in other systems, and may be integrated into such systems as desired without departing from the principles of the invention. Reference will now be made in detail to specific preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.
Referring to
The host 12 is preferably a microprocessor, but may be a digital signal processor, computer, or any other type of device adapted for controlling digital circuits. The host communicates with the graphics controller 10 over a bus 16 to a host interface 12a in the graphics controller.
The display device 14 has one or more display panels 14a with corresponding display areas 18. The one or more display panels 14a are adapted for displaying on their display areas pixels of image data (“pixel data”). The pixel data are typically 24 bit sets of three 8-bit color components but may have any other digital (or numerical) range. LCDs are typically used as display devices in mobile telephones, but any device(s) capable of rendering pixel data in visually perceivable form may be employed.
The camera modules 15a, 15b (or “cameras 15”) each acquire pixel data and provide the pixel data to the graphics controller 10 in addition to any pixel data provided by the host. The cameras are programmatically controlled through a serial “control” interface 13. The control interface 13 provides for transmitting control data (“S_Data”) to and from the respective cameras 15 and a clock signal (“S_Clock”) for clocking the control data. The bus serving the interface 13 is preferably that known in the art as an inter-integrated circuit (“I2C”) bus. Each I2C data transfer starts with an ID being transmitted and only the device with the matching ID receives or transmits data for that transfer. The data from the cameras 15 are typically processed by the graphics controller 10, such as by being cropped, scaled, and resized, or JPEG encoded, and that the data received from camera modules 15a and 15b are stored in respective portions of an internal memory 24.
In contrast to the prior art and in accordance with the invention, the graphics controller 10 includes a single, parallel “data” interface 17 for receiving pixel data streamed from the cameras 15 to the graphics controller. The data interface 17 is coupled to a bus 19 having DATA and other lines. The data interface 17 provides the data received on the bus to the graphics controller 10, along with vertical and horizontal synchronizing signals (“VSYNC” and “HSYNC”). The data interface 17 provides a clock signal CAMCLK that is transmitted from the graphics controller to the cameras 15 over a dedicated line of the parallel bus 19. The graphics controller 10 includes a clock generator 22 that produces the (common) clock signal CAMCLK. Other clock sources, either located within or external to, the graphics controller 10 may be substituted, in whole or in part, for the clock generator 10. The exemplary graphics controller 10 also includes an enable control for setting registers in the camera as described below, and a sampling circuit 32 for sampling the data streams received from the data interface 17.
Also in contrast to the prior art and in accordance with the invention, the camera output is modified to cooperate with the camera interface 17. The cameras modules 15a, 15b include, in one embodiment, respective switching circuits 24a, 24b, buffers 20a, 20b, and control registers R1, R2. The signal CAMCLK is provided to the switching circuits 24a, 24b. Each switching circuit is coupled to an enable/disable input of its respective buffer and to its respective control register. Input to the buffers 20a, 20b are provided at respective inputs A and B. Each buffer 20 may be enabled or disabled, at the respective point labeled “Enable,” to either place valid data on its outputs or to place its outputs in a high impedance state. While the buffers 20 may be provided integrally with the cameras (as shown in
In a preferred embodiment, the graphics controller initiates the clock signal CAMCLK and upon first receipt of the clock signal, each camera determines the clock pulse on which to initiate the transmission of pixel data to the graphics controller. A camera determines which clock pulse to initiate data transmission by consulting a temporal-shift register (not shown) in the camera which the graphics controller 10 programs through the control interface 13. The value stored in the temporal-shift register specifies the number of clock pulses the camera must wait before transmitting the first line of a data stream of pixel data. By this means, data streams output from the cameras may be temporally shifted relative to one another by amounts that are integer multiples of the period of the signal CAMCLK. Notwithstanding any relative temporal shifting, the data streams output from the cameras remain synchronized to the common clocking signal CAMCLK.
For clarity of presentation, the data stream DSB is shown temporally shifted with respect to the data stream DSA by an amount that is equal to half the period of a pixel datum (ΔTLOW), to achieve an anti-parallel alignment in which the two data streams are 180 degrees out of phase. Such a shift could be obtained in practice, in a similar manner to that used for obtaining temporal shifts as described above, by utilizing a derivative clock signal that is divided down from the clock signal CAMCLK.
In a preferred embodiment, the data streams DSA and DSB are interleaved for transmission to the graphics controller 10 over the DATA lines of the interface 17. In other embodiments, two or more data streams are interleaved. By interleaving the data streams from multiple cameras, data from the multiple cameras may be transmitted over the interface 17 at essentially the same time.
To permit the interleaving of the two original data streams DSA and DSB, high impedance (“High-Z”) states are inserted into the original data streams to produce corresponding modified data streams DSA′ and DSB′. For example, with reference to
The modified data streams DSA′ and DSB′ are preferably interleaved in a particular manner.
As an example, referring to
Turning to
Lines 3B and 3D show, respectively, the modified data stream DSA′ of line 2B of
While a methodology has been described above with examples having two and three data streams, it is contemplated that the methodology may be advantageously employed with more than three data streams, corresponding to more than three data sources, which may or may not be cameras. For example, the third data source in the example shown in
To permit the interleaving of the two or more original data streams, High-Z states are inserted into the original data streams to produce corresponding modified data streams. For example, to produce the modified data stream DSA′ shown in
To achieve the sampling and High-Z state triggering, the switching circuits 24a and 24b, which are depicted in the exemplary system shown in
The data interface 17 receives pixel data streamed from the cameras 15. The data interface 17 is coupled to a sampling circuit 32. The sampling circuit 32 samples the data streams as the data streams are received by the data interface 17. Preferably, the sampling circuit 32 includes one or more registers (not shown) for defining the superimposed data streams. As one example, a first sampling circuit register specifies that there are two camera data streams, and a second sampling circuit register specifies which of the cameras is set to provided data in-phase with the clock signal.
Referring again to
Referring again to
The invention provides the outstanding advantage of providing an exceptionally low cost alternative to multiplexing the output of multiple cameras on a single parallel data interface, for realizing savings in hardware cost and power consumption that are important in low-cost, battery powered consumer appliances such as cellular telephones. It is especially advantageous that the invention provides for the elimination of at least one parallel bus.
The camera modules 15a and 15b are preferably substantially the same, i.e., they are of the same manufacture and of the same model or type, so that their timing will be optimally matched for synchronization (“matched”); however, this is not essential to the invention.
In the examples presented herein, the multiple devices providing streaming data have been cameras outputting image data. However, any other device outputting image data may be substituted in alternative embodiments. All that is required of the streaming data source is that its output data stream be capable of being synchronized and modified as described herein. As one example, the device may be a memory, such as a flash memory or a hard disk drive. In one embodiment, the memory device is used for storing image data, which may have been previously captured by a camera module of the system 8, or which may have been transmitted to the system 8. In another embodiment, the memory device is used for storing audio files, such as mp3 or wav files, and the system 8 includes an audio output for playing the music files.
It should be understood that, while preferably implemented in hardware, the features and functionality described above could be implemented in a combination of hardware and software, or be implemented in software, provided the graphics controller is suitably adapted. For example, a program of instructions stored in a machine readable medium may be provided for execution in a processing device included in the graphics controller.
It is further to be understood that, while a specific method and apparatus for streaming data from multiple devices over a single data bus has been shown and described as preferred, other configurations and methods could be utilized, in addition to those already mentioned, without departing from the principles of the invention.
The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.