Patent Application
20050073586
The present invention is generally directed to an interface and, more specifically, to a digital camera interface.
Today, digital cameras have increasingly been integrated within motor vehicles to provide real-time images for further processing. In a typical situation, a digital camera may be separated from an electronic control unit (ECU), which includes an imbedded processor and/or a digital signal processor (DSP), that implements vision processing and decision making algorithms.
Captured images may be transferred from a camera to a host processor in various formats. For example, captured images may be transferred from a camera to a host processor as analog video, transmitted through an inexpensive coaxial cable. Interfaces implementing a universal serial bus (USB) 2.0/on-the-go (OTG) protocol may also be used to serially transmit digital data from a digital camera to a host processor. Further, an IEEE 1394 (Firewire) interface may be utilized to transmit digital data from a digital camera to a host processor for further processing. MOST and camera link are two other interfaces that may also be utilized to transmit data from a camera to a host processor for further processing. It should be appreciated that transferring the data via an analog video signal is generally not desirable when the data is provided from a camera in a digital format. Further, technologies, such as USB and IEEE 1394, are generally either not suitable or are too expensive for automotive applications. Additionally, technologies such as MOST are generally too slow and camera link tends to be relatively expensive.
What is needed for automotive applications is a low-cost, high-performance camera to host processor interface.
According to the present invention, a host processor to camera interface includes a camera side interface, a host processor side interface and a cable for coupling a camera side transmitter to a host processor side receiver. A camera side link layer is coupled to a camera, which provides video data that the camera side link layer converts to a desired video data format. A serializer, that is coupled to the camera side link layer, serializes the video data in the desired video data format and a camera side transmitter, which is coupled to the serializer, transmits the serialized video data in the desired video data format. The host processor side receiver receives the serialized video data and a deserializer that is coupled to the host processor side receiver deserializes the video data. A host processor side link layer that is coupled between the deserializer and a host processor is adapted to convert the deserialized video data into a format compatible with the host processor, when required. The cable includes wires for carrying power and couples the camera side transmitter to the host processor side receiver.
In another embodiment, the camera side link layer is configured to convert a plurality of camera video data formats into the desired video data format. In yet another embodiment, the camera video data formats include a single 8-bit data format, a dual 8-bit data format, a single 16-bit data format, a single 24-bit data format and a single 12-bit data format.
According to another embodiment, at least one of the camera video data formats provides a frame valid (FVAL) signal and a line valid (LVAL) signal that are combined by the camera side link layer into a single validation (XVAL) signal that is provided to the host processor side link layer. In still another embodiment, the XVAL signal corresponds to the LVAL signal with an added end-of-frame (EOF) signal.
In yet another embodiment, a low-voltage differential signaling (LVDS) receiver, which provides for serial-to-camera (SERTC) channel communications, is located within the camera side interface. The inputs of the LVDS receiver are coupled to outputs of the camera side transmitter and an output of the LVDS receiver is coupled to the camera side link layer. In this embodiment, an LVDS transmitter, which provides for SERTC channel communications, is located within the host processor side interface. The SERTC channel is provided for reconfiguring the camera and an input of the LVDS transmitter is coupled to the host processor side link layer and outputs of the LVDS transmitter are coupled to inputs of the host processor side receiver.
In yet another embodiment, the cable includes a first pair of signal wires for carrying the video data, which is in the form of a low-voltage differential signaling (LVDS) data stream. In another embodiment, the camera side link layer and the host processor side link layer are configured to share the first pair of signal wires to communicate the video data in the desired video signal format and configuration signals for the SERTC channel.
These and other features, advantages and objects of the present invention will be further understood and appreciated by those skilled in the art by reference to the following specification, claims and appended drawings.
The present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
According to the present invention, a low-cost, high-performance host processor to camera interface is disclosed herein. In its basic embodiment, the host processor to camera interface includes a camera side interface that is coupled by a cable to a host processor side interface. The camera side interface includes a camera side link layer coupled to a camera, with the camera providing video data and the camera side link layer converting the video data to a desired video data format. The camera side interface also includes a serializer coupled to the camera side link layer for serializing the video data in the desired video data format. A camera side transmitter that is coupled to the serializer transmits the serialized video data to the host processor side interface, via the cable.
The host processor side interface includes a host processor side receiver, a deserializer and a host processor side link layer. The host processor side receiver receives serialized video data and is coupled to the deserializer, which deserializes the video data. The host processor side link layer is coupled to the deserializer and the host processor and is adapted to convert the deserialized video data into a format compatible with the host processor, when required. The cable includes wires for carrying power and couples the camera side transmitter to the host processor side receiver.
In one embodiment, the camera side link layer is configured to convert a plurality of camera video data formats into a desired video data format. The video data formats may include a variety of formats, e.g., a single 8-bit data format, a dual 8-bit data format, a single 16-bit data format, a single 24-bit data format and/or a single 12-bit format, among other data formats. When the video data is in electronic form, i.e., not optical, the cable includes at least one pair of wires for transmitting the data stream and two wires for power and ground. It should be appreciated that existing USB2.0/OTG cables or an IEEE 1394 cable can generally provide adequate data and power carrying capability. When the data stream is in an optical format, it should be appreciated that a fiber optic cable carries the video data signal. It should also be appreciated that when a signal is in an optical format, the host processor side interface includes circuitry for converting the optical signal to an electronic signal.
The camera side link layer and the host processor side link layer may be implemented with a variety of devices, e.g., an inexpensive field programmable gate array (FPGA), an electrically programmable logic device (EPLD) or a low cost microcontroller. The serializer for the camera side interface and the deserializer for the host processor side interface, as well as the SERTC transceiver may be off-the-shelf components. For example, whenever the implemented system utilizes low-voltage differential signaling (LVDS): the deserializer may be a DS92LV1212A manufactured and made commercially available by National Semiconductor; the serializer may be a DS92LV1021A manufactured and made commercially available by National Semiconductor; and the transceiver may be a DS90LV019 driver/receiver manufactured and made commercially available by National Semiconductor. Alternatively, the serializer, deserializer and SERTC transceiver may be implemented using USB or IEEE 1394 technologies.
As implemented, a single or dual camera system may provide a digital video data stream to the camera side link layer at a pixel clock rate of CM-MCLK. Depending upon the camera(s) implemented, the video data or camera pixel data may have different data formats corresponding to a specific imager manufacturer. The camera side link layer provides a set of digital logics that convert different types of camera pixel data into a desired video data format. It should be appreciated that the desired video data format may vary based upon a format required by an implemented serializer, which performs real-time parallel-to-serial conversion. For example, a serializer that converts 10-bit parallel data into a serial output would generally use 8-bits of pixel data, D0-D7, a validation (XVAL) signal, which is required to synchronize the frames and lines of the images, and may include an additional bit for flexible purposes, such as to read out camera parameters in real-time.
The host processor side interface 134 includes a deserializer 126 (that includes a receiver 128), a driver/receiver 122 and a host processor side link layer 124. The deserializer 126 converts the data received from the serializer 106 into parallel data, including 8-bits of pixel data D[0:7], a validation (XVAL) signal and an additional bit SerTFG for flexible purposes, such as to read out camera parameters in real-time. The host processor side link layer 124 converts the data, if required, to be compatible with host processor 10. The conversion may include converting the 8-bits of data D[0:7] to 16-bits of data Data[0:15], which represents the two 8-bit serial streams provided by the cameras 102A and 102B. The host processor 10 may provide a reset (RST) signal to the host processor side link layer 124, which in turn provides the RST signal to the cameras 102A and 102B, via the driver/receiver 122, driver/receiver 112 and the camera side link layer 104. In the event that the data provided by the deserializer 126 is in a format that is compatible with the host processor 10, the functions performed by the host processor side link layer 124 may essentially be performed by the host processor 10.
The table below depicts the conversion of several popular digital imager data formats into the above-described 10-bit parallel data format.
With reference to
As a general rule, most digital cameras output horizontal and vertical clocks, as well as camera pixel data. These horizontal and vertical clocks are used to indicate which pixel stands for the first pixel of a frame and which pixel stands for a specific line. For example, in the camera link standard these horizontal and vertical clocks are known as a frame valid (FVAL) signal and a line valid (LVAL) signal, respectively. According to an embodiment of the present invention, a validation (XVAL) signal is utilized to represent both the FVAL and LVAL signals using only one data bit. It should be appreciated that there are a number of ways to combine the FVAL and LVAL signals to provide the XVAL signal.
With reference to
According to the present invention, a remotely mounted camera or cameras can provide video data in real-time with minimized delay and essentially zero data loss, similar to a camera that is directly coupled to a host processor.
In another embodiment of the present invention, in systems that need a camera that requires reconfiguration, a serial-to-camera (SERTC) channel may be provided. It should be appreciated that the SERTC channel can share the same wire pair utilized to transfer camera pixel data or, alternatively, another wire pair may be provided, if desired. In the event that bus sharing is desired for lower cost cabling, the link layers of both the transmitter and receiver side may follow a simple protocol of sharing the serial bus. In any event, the protocol defines when and how to enable or disable the serializer or the SERTC channel transmitter in an appropriate manner.
In the event that the serial bus needs to be released to the SERTC channel, a host processor side link layer may send a trigger pulse (SERTC-Request) to the transmitter through the power line (VBUS). In this case, the camera side link layer responds to the SERTC-Request signal by disabling the serializer and enabling the SERTC channel receiver. At that point, the bus is free and ready to be released to the SERTC channel. The processor side link layer 124 may then detect the bus status by monitoring the deserializer 126 outputs. When a free bus is detected, the host processor side link layer 124 enables the SERTC channel transmitter 122 and the bus is released to the SERTC channel to establish a serial communication interface (SCI) between the cameras 102A and 102B and the host processor 10.
The serial bus may be released to provide a video data stream from the camera to the host processor in a number of ways. For example, an SERTC-Request trigger pulse may be generated. Alternatively, the SCI to camera connection may be utilized. When the SCI to camera interface is utilized, the host processor may send a reset byte to the host processor side link layer 124 or, alternatively, reset the host processor side link layer 124 using a reset (RST) pin. In any event, the host processor side link layer 124 is programmed such that on detecting a reset message, it disables the SERTC channel transmitter 122 and enables the deserializer 126. In this embodiment, the camera side link layer 104 detects the bus release and consequently disables the SERTC channel receiver 112 and enables the serializer 106.
It should be appreciated that a camera can be reset through a number of techniques, e.g., by switching off the power to the camera or by sending an SERTC-Request pulse train, i.e., more than one pulse, to the camera side link layer 104. An outside trigger signal may also be used to initiate capturing of new frames by the camera. This may be achieved by imbedding a trigger camera pulse on the power line VBUS. Alternatively, once the serial bus is released to the SCI, the host processor 10 may send a trigger camera message to the camera side link layer 104. In this embodiment, the camera side link layer 104 decodes the message and sends a trigger pulse to the cameras 102A and 102B. Utilizing an interface configured according to the present invention allows for data transfer at a rate of up to about 400 megabits per second, which provides for about 65 frames per second for dual 8-bit mono VGA camera applications.
It should be appreciated that the host processor to camera interface described herein may be utilized in a number of applications, such as interior looking cameras that utilize single or dual 8-bit mono VGA and outside looking cameras, such as single 16-bit cameras.
Accordingly, a low-cost high-speed interface for linking a camera to a remote electronic control unit (ECU) processor for automotive applications has been described herein. Embodiments of the present invention provide a unique way of sending real-time digital camera data without the need for expensive serial data bus link layer electronics. Such a host processor to camera interface may be particularly useful in applications, such as eye gaze monitoring, lane tracking, pre-crash sensing, among other applications.
The above description is considered that of the preferred embodiments only. Modifications of the invention will occur to those skilled in the art and to those who make or use the invention. Therefore, it is understood that the embodiments shown in the drawings and described above are merely for illustrative purposes and not intended to limit the scope of the invention, which is defined by the following claims as interpreted according to the principles of patent law, including the doctrine of equivalents.
