This application relates generally to display technology and more specifically to circuitry for controlling displays.
Power consumption is a concern with modern electronic devices, particularly portable handheld devices. Battery-powered cell phones and wireless electronic reading devices incorporating conventional display technologies require frequent re-charging of the batteries, in some cases, several times in a single day. The need to constantly re-charge such devices interferes with their fundamental purpose, that is, to allow a user to continue using them (i.e., not be interrupted to have to re-charge them) as the user moves from place-to-place throughout the day.
A significant amount of power, often the majority of power, is consumed by the displays in many modern portable electronic devices for certain applications. Currently, the majority of displays used on mobile devices are Liquid Crystal Displays (LCDs), which require continuous updates of video data to maintain the video output on the display. Electronic reading devices with bi-stable displays do not require continuous updates but still consume an unacceptable amount of power. The power across a display interface tends to be high, particularly for larger displays. Indeed, the power required by active display interfaces in modern devices is growing rapidly, particularly as display resolutions increase for these devices. The power consumed by the display interface is generally proportional to the square of the switching voltage, the frequency of the display data, and the capacitance of the interconnect lines of the interface.
Thus, an overall concern with modern electronic devices is conservation of power used to drive the displays.
Disclosed are methods, apparatus, and systems, implementing and using techniques for processing frames of video data sent across a display interface using a block-based encoding scheme and tag ID's.
Some aspects of the present application incorporate techniques which cooperate with a host element, often in the form of a graphics processor or controller, and a display element, often in the form of a display controller which drives a display. A display interface connects the graphics processor with the display controller. The disclosed apparatus and methods provide for the compression of video data at the host element, before it is sent across the display interface, and then the de-compression of this data at the display element.
The display interface is traditionally viewed as a physical layer or connection between the host element and the display element. Some aspects of the present application are based on a logical view of the display interface. Logical operations can be performed to organize and transmit the data across the display interface. These operations are applicable to various physical interfaces and connections. Regardless of the physical nature of the display interface layer, applying techniques disclosed herein, video data can be encoded on the graphics processor side of the interface and selectively decoded at the display controller side after it is sent across the interface. The decoded data is, accordingly, selectively output from the display controller to the display.
Some aspects of the present application provide for optimization of the display interface, situated between the graphics processor and the display controller of an electronic device. The optimization techniques described herein minimize the amount of signaling over the interface and reduce the power consumed at the interface. Accordingly, the battery life of some electronic devices can be extended.
According to one aspect of the present application, an apparatus comprises a graphics processor configured to receive frames of video data. Each frame includes one or more blocks of the video data. The graphics processor is configured to encode each block of video data and generate a tag ID associated with each encoded block of video data. The graphics processor is configured to output each encoded block of video data and associated tag ID. A display interface is in communication with the graphics processor. A display controller is in communication with the display interface. The display controller is configured to receive the encoded blocks of video data and associated tag ID's from the graphics processor via the display interface. The display controller is configured to interpret the tag ID associated with a respective encoded block of video data and determine whether to decode at least part of the respective encoded block of video data according to the tag ID. A display, such as a memory-based display, is in communication with the display controller. The display is configured to receive decoded blocks of video data from the display controller and to display the decoded blocks of video data.
According to one implementation, the tag ID can include one or more indications such as: a start of a new frame of video data, a redundant frame of video data, a start of a new block of video data, and a redundant block of video data. For instance, the display controller can be configured to disregard the encoded block of video data if the tag ID indicates a start of a redundant block of video data.
Depending on the desired implementation, the graphics processor can be configured to encode the blocks of video data using processing techniques such as Run Length Encoding (RLE), Arithmetic Coding (AC), or Huffman Coding (HC).
According to one implementation, the display is a bi-stable display such as: an interferometric modulation display (IMOD), a cholesteric liquid crystal display (ChLCD), or an electrophoretic display.
Depending on the desired implementation, the display interface can be configured to pass the encoded blocks of video data using a standard such as: the Mobile Industry Processor Interface (MIPI) standard, the Mobile Display Digital Interface (MDDI) standard, the Low-Voltage Differential Signaling (LVDS) standard, or the High-Definition Multimedia Interface (HDMI) standard.
Another aspect of the present application relates to a method in which each block of video data is encoded. Tag ID's associated with each encoded block of video data are generated. For instance, the tag ID can be generated by performing a compare operation between successive blocks of video data. Encoded blocks of video data and associated tag ID's are provided from the graphics processor to a display interface in communication with the graphics processor. A display controller in communication with the display interface receives the encoded blocks of video data and associated tag ID's. The tag ID associated with a respective encoded block of video data is interpreted. It is determined whether to decode at least part of the respective encoded block of video data according to the tag ID. Decoded blocks of video data are provided from the display controller to a display in communication with the display controller. The display is configured to display the decoded blocks of video data.
These and other methods and apparatus of aspects of the present application may be implemented using various types of hardware, software, firmware, etc., and combinations thereof. For example, some features of the application may be implemented, at least in part, by computer programs embodied in machine-readable media. The computer programs may include instructions for operating, at least in part, the devices described herein. These and other features and benefits of aspects of the application will be described in more detail below with reference to the associated drawings.
The included drawings are for illustrative purposes and serve only to provide examples of possible structures and process steps for the disclosed methods, apparatus, and systems for processing frames of video data sent across a display interface using a block-based encoding scheme and a tag ID.
While the present application will be described with reference to a few specific embodiments, the description and specific embodiments are merely illustrative and are not to be construed as limiting. Various modifications can be made to the described embodiments without departing from the true spirit and scope as defined by the appended claims. For example, the steps of methods shown and described herein are not necessarily performed in the order indicated. It should also be understood that the methods may include more or fewer steps than are indicated. In some implementations, steps described herein as separate steps may be combined. Conversely, what may be described herein as a single step may be implemented in multiple steps.
Similarly, device functionality may be apportioned by grouping or dividing tasks in any convenient fashion. For example, when steps are described herein as being performed by a single device (e.g., by a single logic device), the steps may alternatively be performed by multiple devices and vice versa. Moreover, the specific components, parameters, and numerical values described herein are provided merely by way of example and are in no way limiting. The drawings referenced herein are not necessarily drawn to scale.
Embodiments of the present application overcome some of the drawbacks of conventional electronic devices, by reducing the amount of power consumed at the display stage. By incorporating embodiments of the present application, electronic devices are able to reduce this power drain, which is a significant component of the overall power consumption of the device. Thus, some of the features described herein provide for a longer lasting memory display, such as a bi-stable display, for instance, in a battery-powered mobile reading device.
The apparatus and methods described herein leverage the characteristics of both the content of the video data being transmitted as well as the features of memory displays. As used herein, “memory display” refers to any display having a memory function, that is, where the display is capable of retaining displayed video data. Examples of suitable memory displays include bi-stable displays as well as other types of displays incorporating memory devices such as frame buffers. With respect to the content, one technique involves the use of tag ID's associated with blocks of video data sent across the display interface. Embodiments of the present application can use a block-based approach to sending data across the display interface, in which individual blocks of pixels within a frame of video data are processed. A tag ID generator is provided on the graphics processor side of the display interface, as further explained below, and a counterpart tag ID reader is located on the display controller side. The tag ID generator generates a tag ID for unique blocks of video data being sent. The tag ID reader interprets the tag ID to determine whether to write a particular block to the display.
A second technique described herein uses a block-based encoder, for instance, a Run Length Encoder on the graphics processor side, and a counterpart block-based decoder on the display controller side of the display interface. In some implementations, Run Length Encoding (RLE) is desirable because it is lossless, meaning no loss is introduced by the encoding scheme in signals sent from the graphics processor to the display controller. In addition, RLE is desirable because it can be simple to implement, thus reducing code delay and processing power. In some embodiments, RLE is performed according to color of the pixels. The data in images, particularly in sub-portions or blocks of the image is often correlated by color. Thus, higher encoding and decoding efficiency can be achieved by grouping red, green, and blue pixels together, for example. Also, depending on the desired implementation, raster scanning or serpentine scanning can be used to read and encode the pixel value colors row-by-row or in some other sequence within a block.
Different encoding and decoding schemes can be incorporated into embodiments of the present application, as an alternative to RLE. Examples of such schemes include Arithmetic Coding (AC) and Huffman Coding (HC). AC and HC are useful in some implementations in which more compression is desired.
In one embodiment, the encoder is configured to encode m×n blocks within in each frame of video data. The m×n block could be variable or fixed size, depending on the implementation. Also, when implementing block-based encoding and decoding in this manner, tradeoffs can be made between memory size code delay, implementation delay, and compression efficiency, by varying the m×n size. Encoding successive blocks of pixel data in this manner can take advantage of spatial correlation and colors in most images, thus significantly reducing the size of the data to send across the display interface. For instance, for each m×n block, a Run Length Encoded packet can be generated and sent to the display controller. The block-based decoder is configured to decode and output the data in the packet when the associated tag ID indicates it is appropriate to do so.
Other apparatus and methods in addition to the use of tag ID's and block-based encoding/decoding are disclosed herein. The embodiments incorporating the various features are applicable to a variety of displays, but are particularly beneficial for memory-based display technology. For instance, because bi-stable displays have a memory state, bi-stable displays do not have a requirement on the display controller to provide continuous updates of video data to the display. Bi-stable displays can afford some latency. Thus, the display controller need not decode and output every block or frame of data it receives when the data is redundant, i.e., a copy of previously received data for the region of the display corresponding to the received block. Also, using RLE in combination with memory-based displays facilitates the handling of “bursty” data signals, i.e., including data which is uneven in nature.
Embodiments of the present application can be incorporated in a variety of modern electronic devices, particularly those in which it is desirable to incorporate energy-efficient bi-stable displays, such as Interferometric Modulator Displays (IMODs), Cholesteric LCDs (ChLCDs), electrophoretics (e-ink), and other displays that have bi-stable properties. The techniques described herein optimize the architecture of graphics processors and display controllers for such displays. The amount of signaling required between the graphics processor and the display controller, i.e., over the display interface, is reduced to lower the overall energy consumption of the device.
Embodiments of the present application can be incorporated into electronic devices having other types of memory displays, i.e., displays having a frame buffer or other memory unit local to the display so that incoming video data can be buffered. For instance, as described in greater detail below, a frame buffer can be provided on the display controller side of the display interface and used to buffer data provided from the display controller to the display.
In
In
The MIPI standard, which is a serial interface providing differential signaling, is a common interface standard for electronic devices with smaller displays, for instance, cell phones. In such implementations, the bandwidth of the display interface 116 can be relatively smaller, for instance, 6 bits wide. MDDI is another standard used for electronic devices 100 with smaller displays. The encoding and selective decoding techniques using tag ID's, as described herein, are equally applicable to electronic devices having larger displays, such as those having an HDMI standard at display interface 116.
As shown in the embodiment of
In another embodiment, for instance, when display interface 116 is implemented in accordance with the MIPI or the Low-Voltage Differential Signaling (LVDS) standard, interface 116 can have a different physical configuration. When LVDS is used, a serializing transmitter and a de-serializing receiver can be situated on opposite sides of display interface 116. The transmitter would encode the video data and clock signal to be sent over interface 116 into a differential serial signal. The receiver would be operatively coupled on the display controller side to receive differential data sent over interface 116, perform serial to parallel conversion of the data, and provide the converted data to the display controller. In other implementations, display interface 116 can be configured as a memory-mapped interface, for instance, with a multiplexed address and data bus.
In
Returning to
In one embodiment, block-based encoder 212 and block-based decoder 220 cooperate to encode and decode blocks of video data using the RLE scheme. RLE is a form of encoding in which runs of data, that is, sequences in which the same pixel value occurs in consecutive data elements, are stored as a single data value and count, rather than as the original run.
As described in further detail below, the RLE scheme can be applied to portions of a frame of video data to be transmitted across display interface 116. Using the RLE scheme in this manner saves energy by reducing the amount of data sent over display interface 116. Block-based encoder 212 can apply the RLE technique or other encoding schemes to take advantage of spatial correlations in the video data to compress the data before sending it. For example, a frame of video data retrieved by graphics processor 108 can be separated into 8×8 blocks. Thus, for instance, an all black image in a particular 8×8 block of pixels could be encoded by block-based encoder 212, applying the RLE scheme, as an L64c0x0 or length 64, color 0 (black) sequence. Thus, for a black block of 8×8 pixels, the RLE scheme saves 192 bytes of data, assuming the data is 24 bits. The handling of video data in frames and division into blocks is described in greater detail below.
In
The tag ID generator 216 is configured to generate a tag ID with each encoded block of video data. The tag ID, in the embodiment of
In
The operations carried out to generate tag ID's at graphics processor 108 and read tag ID's at display controller 120 are described in further detail below, following a general discussion of embodiments of methods for encoding and selectively decoding blocks of video data using the apparatus of
In
In
In
Thus, in
Returning to
In
Implementations of the methods and apparatus described herein provide for reducing the amount of data sent across display interface 116. The amount of active time that the CLK signal 204 of
Embodiments of the methods and apparatus described herein bring the power-saving benefits of compression and decompression to the display interface 116. The techniques described herein do so without much cost in the way of additional circuitry, as illustrated by the incorporation of the block-based encoder and tag ID generator in graphics processor 108 and incorporation of block-based decoder 220 and tag ID reader 224 into display controller 120, as shown in
Using the block-based approaches described herein provide opportunities for exploiting areas of a display screen that have redundant content. This is to be contrasted with raster scan technology used in display interfaces, thus maximizing the benefit for bi-stable and other memory-based displays. For instance, with video signals having primarily textual content, the display interface write time could be reduced by 30-50%. Reducing the write time at the display interface corresponds to a reduction in time that the interface is required to be active. The power consumption of the various components active on both sides of display interface 116 is also reduced.
The embodiments described herein may be implemented in any electronic device that is configured to display an image, whether in motion (e.g., video) or stationary (e.g., still image), and whether textual or pictorial. More particularly, it is contemplated that the embodiments may be implemented in or associated with a variety of electronic devices such as, but not limited to, mobile telephones, wireless devices, personal data assistants (PDAs), hand-held or portable computers, GPS receivers/navigators, cameras, MP3 players, camcorders, game consoles, wrist watches, clocks, calculators, television monitors, flat panel displays, computer monitors, auto displays (e.g., odometer display, etc.), cockpit controls and/or displays, display of camera views (e.g., display of a rear view camera in a vehicle), electronic photographs, electronic billboards or signs, projectors, architectural structures, packaging, and aesthetic structures (e.g., display of images on a piece of jewelry).
The controller 21 is configured to communicate with a display controller 120, as shown in
The display device 40 includes a housing 41, a display 30, an antenna 43, a speaker 45, an input device 48, and a microphone 46. The housing 41 is generally formed from any of a variety of manufacturing processes, including injection molding, and vacuum forming. In addition, the housing 41 may be made from any of a variety of materials, including but not limited to plastic, metal, glass, rubber, and ceramic, or a combination thereof. In one embodiment the housing 41 includes removable portions (not shown) that may be interchanged with other removable portions of different color, or containing different logos, pictures, or symbols.
The display 30 of exemplary display device 40 may be any of a variety of displays, including a bi-stable or other memory display, as described herein. In other embodiments, the display 30 includes a flat-panel display, such as plasma, EL, OLED, STN LCD, or TFT LCD as described above, or a non-flat-panel display, such as a CRT or other tube device. However, for purposes of describing the present embodiment, the display 30 includes an interferometric modulator display, as described herein.
The components of one embodiment of exemplary display device 40 are schematically illustrated in
The network interface 27 includes the antenna 43 and the transceiver 47 so that the exemplary display device 40 can communicate with one or more devices over a network. In one embodiment the network interface 27 may also have some processing capabilities to relieve requirements of the controller 21. The antenna 43 is any antenna for transmitting and receiving signals. In one embodiment, the antenna transmits and receives RF signals according to the IEEE 802.11 standard, including IEEE 802.11(a), (b), or (g). In another embodiment, the antenna transmits and receives RF signals according to the BLUETOOTH standard. In the case of a cellular telephone, the antenna is designed to receive CDMA, GSM, AMPS, W-CDMA, or other known signals that are used to communicate within a wireless cell phone network. The transceiver 47 pre-processes the signals received from the antenna 43 so that they may be received by and further manipulated by the controller 21. The transceiver 47 also processes signals received from the controller 21 so that they may be transmitted from the exemplary display device 40 via the antenna 43.
In an alternative embodiment, the transceiver 47 can be replaced by a receiver. In yet another alternative embodiment, network interface 27 can be replaced by an image source, which can store or generate image data to be sent to the controller 21. For example, the image source can be a digital video disc (DVD) or a hard-disc drive that contains image data, or a software module that generates image data.
Controller 21 generally controls the overall operation of the exemplary display device 40. The controller 21 receives data, such as compressed image data from the network interface 27 or an image source, and processes the data into raw image data or into a format that is readily processed into raw image data. The controller 21 then sends the processed data to the driver controller 29 or to frame buffer 28 for storage. Raw data refers to the information that identifies the image characteristics at each location within an image. For example, such image characteristics can include color, saturation, and gray-scale level.
In one embodiment, the controller 21 includes a microcontroller, CPU, or other logic device to control operation of the exemplary display device 40. Conditioning hardware 52 generally includes amplifiers and filters for transmitting signals to the speaker 45, and for receiving signals from the microphone 46. Conditioning hardware 52 may be discrete components within the exemplary display device 40, or may be incorporated within the controller 21 or other components.
The driver controller 29 takes the raw image data generated by the controller 21 either directly from the controller 21 or from the frame buffer 28 and reformats the raw image data appropriately for high speed transmission to the display controller 120. Specifically, the driver controller 29 reformats the raw image data into a data flow having a raster-like format, such that it has a time order suitable for scanning across the display array 30. Then the driver controller 29 sends the formatted information to the display controller 120. Although a driver controller 29, such as a LCD controller, is often associated with the system controller 21 as a stand-alone Integrated Circuit (IC), such controllers may be implemented in many ways. For example, they may be embedded in the controller 21 as hardware, embedded in the controller 21 as software, or fully integrated in hardware with the display controller 120.
The display controller 120 receives the formatted information from the driver controller 29 and reformats the video data into a parallel set of waveforms that are applied many times per second to the hundreds and sometimes thousands of leads coming from the display's x-y matrix of pixels.
In one embodiment, the driver controller 29, display controller 120, and display array 30 are appropriate for any of the types of displays described herein. For example, in one embodiment, driver controller 29 is a conventional display controller or a bi-stable display controller (e.g., an interferometric modulator controller). In another embodiment, display controller 120 is a conventional driver or a bi-stable display driver (e.g., an interferometric modulator display). In one embodiment, a driver controller 29 is integrated with the display controller 120. Such an embodiment is common in highly integrated systems such as cellular phones, watches, and other small area displays. In yet another embodiment, display array 30 is a bi-stable display array (e.g., a display including an array of interferometric modulators).
The input device 48 allows a user to control the operation of the exemplary display device 40. In one embodiment, input device 48 includes a keypad, such as a QWERTY keyboard or a telephone keypad, a button, a switch, a touch-sensitive screen, a pressure- or heat-sensitive membrane. In one embodiment, the microphone 46 is an input device for the exemplary display device 40. When the microphone 46 is used to input data to the device, voice commands may be provided by a user for controlling operations of the exemplary display device 40.
Power supply 50 can include a variety of energy storage devices as are well known in the art. For example, in one embodiment, power supply 50 is a rechargeable battery, such as a nickel-cadmium battery or a lithium ion battery. In another embodiment, power supply 50 is a renewable energy source, a capacitor, or a solar cell, including a plastic solar cell, and solar-cell paint. In another embodiment, power supply 50 is configured to receive power from a wall outlet.
In some implementations control programmability resides, as described above, in a driver controller which can be located in several places in the electronic display system. In some cases control programmability resides in the display controller 120. The above-described optimization may be implemented in any number of hardware and/or software components and in various configurations.
Returning to
Although illustrative embodiments and applications are shown and described herein, many variations and modifications are possible which remain within the concept, scope, and spirit, and these variations should become clear after perusal of this application. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the application is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.