Patent Application
20050068336
1. Field of the Invention
The present invention relates generally to display of digital image data.
2. Description of the Related Art
Modern display devices are commonly equipped with an ability to display a picture-in-picture window. The picture-in-picture window is used for displaying an overlay image on a background image. For example, the background image may represent a desktop view of a computing system, and the overlay image (i.e., picture-in-picture window) may represent an image captured by a digital camera. Traditionally, to display the overlay image on the background image, separate portions of display memory are required. One portion of display memory is required for a main window, and another portion of display memory is required for a sub-window. The portion of display memory for the main window contains display data for the entire background image, and the portion of display memory for the sub-window contains display data for the overlay image. The background image data is retrieved from the main window portion of display memory and displayed. Then, the overlay image data is retrieved from the sub-window portion of display memory and displayed over a portion of the background image.
With the traditional technique for displaying the overlay image on the background image, the display memory is required to be large enough to accommodate data for both the entire main window (i.e., entire background image) and the sub-window (i.e., overlay image). A set of pixels used to define the display image will have two sets of data stored in the display memory: a first set of data for display in a context of the main window and a second set of data for display in a context of the sub-window. Therefore, additional display data is stored in the display memory for each pixel defining the sub-window display area.
A larger display memory requires more silicon space for implementation and consumes more power during operation. Also, the larger display memory increases an overall device cost. Furthermore, since storage of additional display data is required for a each pixel within the sub-window display area, it is necessary to implement separate sub-window functionality (i.e., logic) for determining boundaries that are indicative of which portion of memory (i.e., that containing main window display data or that containing sub-window display data) to retrieve data from for a given pixel.
In view of the foregoing, there is a need for an apparatus and method that will provide for overlaying of image data without requiring storage and retrieval of additional display data.
Broadly speaking, the present invention fills these needs by providing an image overlay apparatus and corresponding method of operation. More specifically, the present invention provides for combining multiple sets of display data in an efficient manner while utilizing a display memory having a capacity to store one complete set of display data for an entire viewable area of a display component. In the present invention, key data is written to the display memory to define a portion of the viewable area of the display component that will be used for rendering an incoming set of display data. The portion of the viewable area of the display component used for rendering the incoming display data can be defined as a picture-in-picture window. The incoming display data can represent a stand-alone image (e.g., picture) or a sequence of images (e.g., video). The apparatus of the present invention can be operated in either of two modes: snapshot mode and view mode. When operated in snapshot mode, the present invention overwrites the key data within the display memory with the incoming stand-alone image display data, prior to transmission of the display data to the display component. When operated in view mode, the present invention does not require that the display data for the incoming sequence of images be written to the display memory. Rather, in view mode, the present invention uses the display data of the incoming sequence of images to modify the key data as the display data is transmitted to the display component. Thus, in view mode, the key data remains stored in the display memory for use with each incoming frame representing the sequence of images.
In one embodiment, an image overlay apparatus is disclosed. The image overlay apparatus includes a memory for storing a first set of display data. The first set of display data includes a set of key data associated with a key data value. The image overlay apparatus also includes an input for receiving a second set of display data and an output for transmitting a set of output display data. A comparison component of the image overlay apparatus is configured to compare a portion of the first set of display data to the key data value to determine whether the portion of the first set of display data is to be modified. If the portion of the first set of display data is to be modified, the modification is performed with a corresponding portion of the second set of display data.
In another embodiment, a display controller is disclosed. The display controller includes a memory region configured to store display data within which key data is integrated. The display controller also includes comparison circuitry that is configured to receive both image overlay data and the display data from the memory region. The image overlay data is received from a source external to the memory region. Both the image overlay data and the display data are received by the comparison circuitry in accordance with a synchronization signal. The comparison circuitry is further configured to modify the key data using the image overlay data during transmission of the display data to a display panel.
In another embodiment, a method for operating an image overlay apparatus is disclosed. The method includes establishing a portion of a first set of display data as key data, wherein the key data is defined by a key data value. The method also includes receiving a second set of display data defining an image of a given shape and size. The method further includes comparing a portion of the first set of display data to the key data value. The portion of the first set of display data that is compared to the key data value represents an equivalent shape and size of the image defined by the second set of display data. The method continues with modifying the key data within the portion of the first set of display data using corresponding portions of the second set of display data. The method also includes transmitting the first set of display data to a display component.
Other aspects of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the present invention.
The invention, together with further advantages thereof, may best be understood by reference to the following description taken in conjunction with the accompanying drawings in which:
Broadly speaking, an invention is disclosed for an image overlay apparatus and corresponding method of operation. More specifically, the present invention provides an apparatus and method for combining multiple sets of display data in an efficient manner that optimizes memory usage. It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a device, or a method. Several exemplary embodiments of the invention will now be described in detail with reference to the accompanying drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.
The display controller 103 is configured to operate in two modes: 1) snapshot mode; and 2) view mode. In snapshot mode, the set of display data received by the display controller 103 represents a single frame of image data. The single frame of image data contains data for each pixel within a portion of a viewable area of the display component 105. For purposes of discussion in the remainder of this document, the set of display data received by the display controller 103 during operation in snapshot mode is referred to as snapshot data. When operated in snapshot mode, the display controller 103 transmits the snapshot data to the display component 105 to be displayed.
The display memory 203 is capable of storing image data for each pixel within a coordinate system used to define the viewable area of the display component 105. In snapshot mode, the snapshot data is written to the display memory 203. However, the snapshot data is only written to portions of the display memory 203 containing a designated key data value. For example, in the case where the snapshot data represents a rectangular image, there will be a corresponding portion of the display memory 203 for storing display data capable of defining the complete rectangular image associated with the snapshot data. For purposes of discussion, this corresponding portion of the display memory 203 is referred to as an image overlay area. An example image overlay area 204 is shown within the display memory 203 of
With respect to the example embodiment of
In one embodiment, the complete set of display data can be transmitted to an encode/decode device 207 as indicated by an arrow 213. The encode/decode device 207 can be an encoding component capable of receiving the complete set of display data and converting the complete set of display data to a digital format suitable for storage. In one embodiment, the encode/decode device 207 is a Joint Photographic Experts Group (JPEG) coder-decoder (CODEC). The JPEG CODEC includes hardware or software for converting display data to JPEG code and vice versa. Thus, the JPEG CODEC is capable of converting the complete set of display data to a JPEG format suitable for storage on a computer readable media. After being converted to the digital format suitable for storage, the complete set of display data can be transmitted to the MPU host 107, as indicated by arrows 215 and 111, for storage or further processing. Also, as indicated by arrows 215 and 111, the encode/decode device 207 is capable of receiving and decoding JPEG formatted data from the MPU host 107. The decoded JPEG data can be transmitted from the encode/decode device 207 to the snapshot comparison component 210 for processing in the manner previously discussed. Thus, in one embodiment, the JPEG data can serve as the display data source.
In one embodiment, the designated key data value is represented as a data value mapped to a color (i.e., RGB value). In this embodiment, the designated key data value is referred to as a key color. In one embodiment, the key color is user specified. Key data specifications (i.e., color value, shape, size, and position in the viewable area of the display component 105) can be specified by the user through the MPU host 107 and stored in internal registers within the display controller 103. The shape, size, and position of the key data area (i.e., key color area) is variable. Thus, the key data area can be defined to allow the snapshot data to be displayed in any suitable shape (e.g., square, heart, cloverleaf, etc . . . ), size, and position in the viewable area of the display component 105.
When the display controller 103 is operated in view mode, the set of display data received by the display controller 103 represents multiple frames of image data transmitted in a particular sequence at a particular rate. For example, in one embodiment, the multiple frames of image data correspond to a digital video stream. For purposes of discussion in the remainder of this document, the set of display data received by the display controller 103 during operation in view mode is referred to as view data. When operated in view mode, the display controller 103 transmits the view data in an appropriate sequence and at an appropriate rate to the display component 105 to be displayed.
As previously described, the display memory 203 is capable of storing image data for each pixel within the coordinate system used to define the viewable area of the display component 105. Unlike snapshot mode, in view mode the view data is not written to the display memory 203. Rather, the view data is used to modify designated key data during transmission of the complete set of display data from the display memory 203 to the display component 105. In one embodiment, modification of the key data is performed by replacing the key data with corresponding view data. In another embodiment, modification of the key data includes performing logical operations with the display data and the key data. Examples of the logical operations that can be performed include AND, OR, XOR, AVG, BITSHIFT LEFT, BITSHIFT RIGHT, among others. It should be appreciated by those skilled in the art that modification of the key data can be achieved by performing other logical operations not explicitly exemplified. For example, in the case where the view data represents a sequence of rectangular images (i.e., frames), there will be a corresponding portion of the display memory 203 for storing display data capable of defining the complete rectangular image associated with each frame of the view data. For purposes of discussion, this corresponding portion of the display memory 203 is referred to as a frame overlay area. An example frame overlay area 304 is shown within the display memory 203 of
The view comparison component 301 performs a pixel-by-pixel comparison of the data stored in the frame overlay area 304 of the display memory 203 with the designated key data value. The pixel-by-pixel comparison is performed by the view comparison component 301 during transmission of data from the display memory 203 to the display component 105. With respect to
With respect to the example of
In view mode, view data can be transmitted from the display data source 101 to the display component 105 in a synchronous manner to avoid visual anomalies. With respect to
In a manner similar to the snapshot mode, one embodiment of the view mode defines the designated key data value as a data value mapped to a color (i.e., RGB value). In this embodiment, the designated key data value is referred to as a key color. In one embodiment, the key color is user specified. Key data specifications (i.e., color value, shape, size, and position in the viewable area of the display component 105) can be specified by the user through the MPU host 107 and stored in internal registers within the display controller 103, as discussed with respect to the snapshot mode. The shape, size, and position of the key data area (i.e., key color area) is variable. Thus, the key data area can be defined to allow the view data to be displayed in any suitable shape (e.g., square, heart, cloverleaf, etc . . . ), size, and position in the viewable area of the display component 105, during either view mode or snapshot mode.
In one embodiment, the display data source 101, MPU host 107, display component 105, and display controller 103 can all be on-board a device. For example, in one embodiment the device is a wireless phone having a digital video camera as the display data source 101, an MPU host 107, an LCD panel as the display component 105, and the display controller 103 of the present invention. Additionally, the display component 105 of the device may have integrated memory capability.
The method also includes an operation 503 for receiving a second set of display data. The second set of display data is received from a display data source. As previously discussed with respect to various embodiments, the display data source can be a digital video camera, a digital camera, an electromagnetic transmission, a digital data storage device, or an external MPU host, among others. Therefore, depending on the display data source, the second set of display data can represent either a stand-alone single frame of image data or a frame of image data from a video transmission containing multiple frames of sequenced image data. As such, the second set of display data defines an image having a shape and a size.
The method further includes an operation 505 for comparing a portion of the first set of display data to the key data value. The portion of the first set of display data used in the comparison of operation 505 represents an image overlay area contained within the viewable area of the display component. The image overlay area is defined to correspond to the shape and size of the image represented by the second set of display data. In one embodiment, the display data for each pixel within the portion of the first set of display data is compared to the key data value.
The method continues with an operation 507 for modifying the key data within the portion of the first set of display data using corresponding portions of the second set of display data. In one embodiment, modifying the key data is performed by replacing the key data with the corresponding portions of the second set of display data. In another embodiment, modifying the key data includes performing logical operations using the key data and the corresponding portions of the second set of display data. Examples of the logical operations that can be performed include AND, OR, XOR, AVG, BITSHIFT LEFT, BITSHIFT RIGHT, among others. It should be appreciated by those skilled in the art that modifying the key data can be performed using other logical operations not explicitly exemplified.
In one embodiment, the key data is modified using the corresponding portion of the second set of display data while the first set of display data is contained within a memory. This embodiment corresponds to the snapshot mode of operation as previously discussed. In another embodiment, the key data is modified using the corresponding portion of the second set of display data during a transmission of the first set of display data to the display component. In this embodiment, the corresponding portion of the second set of display is not written to memory. This embodiment corresponds to the view mode of operation as previously discussed. Also in this embodiment, the second set of display data includes a synchronization signal. The synchronization signal can be used to initiate the modification of the key data with the corresponding portion of the second set of display data.
The method also includes an operation 509 for transmitting the first set of display data to a display component. The first set of display data transmitted in the operation 509 represents the first set of display data existing after the modification of the key data using the corresponding portion of the second set of display data as performed in the operation 507. In one embodiment, the display component is a liquid crystal display panel. However, the method of
The display memory of the present invention contains one set of display data for the entire viewable area of the display component. The key data is written to the display memory to define the portion of the viewable area of the display component that will be used for rendering of the incoming display data. In one embodiment, the portion of the viewable area of the display component used for rendering the incoming display data is a picture-in-picture window. The incoming display data can represent a stand-alone image (e.g., picture) or a sequence of images (e.g., video) to be displayed in the portion of the viewable area of the display component. When operated in snapshot mode, the present invention overwrites the key data within the display memory with the incoming stand-alone image display data, prior to transmission of the display data to the display component. When operated in view mode, the present invention does not require that the display data for the incoming sequence of images be written to the display memory. Rather, in view mode, the present invention uses the display data of the incoming sequence of images to modify the key data as the display data is transmitted to the display component. Thus, in view mode, the key data remains stored in the display memory for use with each incoming frame representing the sequence of images.
In accordance with the foregoing, the present invention does not require the storage of additional display data for common pixels. Thus, the present invention reduces the amount of required memory. For example, consider a device incorporating a 100×100 pixel picture-in-picture window displaying an image on a display panel at 16 bits-per-pixel. Without the present invention, the memory will need to store display data for the entire display panel, including the display area to be overlayed by the picture-in-picture window. In addition, without the present invention, the memory will need to store display data for the picture-in-picture window. The present invention does not require storage of additional display data for the pixels defining the display area to be occupied by the picture-in-picture window. Therefore, with the present invention, the memory requirements are reduced by approximately 19.5 KBytes, which may be significant for a handheld device having limited computational capabilities.
Also, since storage of additional display data is not required for a given pixel, the present invention alleviates the need to have separate functionality for determining which portion of memory to retrieve data from for the given pixel. Due to the reduced memory capacity and access requirements afforded by the present invention, savings can be realized in terms of device size, device power consumption, and device cost. Also, by eliminating the need to read from display memory more than once for a given pixel, the present invention provides improved device performance in terms of operating speed.
While this invention has been described in terms of several embodiments, it will be appreciated that those skilled in the art upon reading the preceding specifications and studying the drawings will realize various alterations, additions, permutations and equivalents thereof. It is therefore intended that the present invention includes all such alterations, additions, permutations, and equivalents as fall within the true spirit and scope of the invention.
