Patent Application
20100208830
This disclosure relates to video decoders in general and more specifically to a video decoder that is compatible with personal computers and even more specifically to a video decoder that can handle highly coded media over bandwidth-limited physical channels.
It is well-known that in order to deliver large data files, such as a video file of a movie, over a given communication channel the bandwidth of the channel determines the maximum rate at which the file can be received from the source (such as a cable provider head end) to a receiver (such as a cable TV user). A transmission medium, such as a coaxial cable or a wireless protocol, has finite bandwidth that must be shared among all the video files that are simultaneously using that medium. Thus, the more bandwidth one data file requires for transmission from point A to point B over the medium, the fewer the number of different data files that can be transmitted over a given period of time.
One system for reducing the size of a video data file, and thereby the bandwidth required for its transmission and reception, is to compress the size of the file before it is transmitted and then to decompress the file at the receiver. While video compression techniques have advanced greatly over the years, some data (and the information content of that data) are usually permanently lost during the compression/decompression process. When video data files are transmitted in a compressed format this “lost” data results in artifacts in the displayed video that may be visible to the viewer. As the need for more channels increases, the tendency is to increase the extent to which each video file is compressed so as to preserve bandwidth since it is not practical to continually increase transmission medium bandwidth. However, a point is reached where the artifacts become visibly objectionable.
The problem of visibly objectionable artifacts is compounded because the video file may have undergone more than one compression/decompression cycle as well as previous color mixes or re-sizes (each of which can add artifacts to the video) prior to the transmission of the file to the ultimate viewer. Thus, it is possible that a provider (a cable company) could compress a video file to an extent that is acceptable for the case of original undistorted videos but unacceptable for videos that have pre-existing distortion artifacts.
To put the problem in perspective, it must be remembered that each pixel on a screen is made up of three color components. A typical viewing quality might be 8 data bits per color or 24 bits per pixel. With a high definition viewing screen there easily could be over a million pixels per frame. Each pixel must be written to the display once per frame at a frame rate of over 24 frames per second. The H264 (MPEG 4) video compression protocol typically compresses the 24 bits per pixel down to less than 1/20th of a bit per pixel, corresponding to a compression factor of approximately 500 (20×24=480). However, at this level of compression the artifacts in the uncompressed H264 video are typically both visible and objectionable.
The most serious objectionable artifacts of compression at this level are in the form of small rectangular blocks that typically vary with time, size and orientation in ways that depend on the local spatial-temporal characteristics of the video scene. In particular, the nature of the artifact blocks depends upon the local motions of objects in the video scene and on the amount of spatial detail that those objects contain. As the compression ratio is increased for a particular video, MPEG-based video encoders allocate progressively fewer bits to the so-called quantized basis functions that represent the intensities of the pixels within each block. This is so because the intensity (for example, color depth or color shading) is not as important to the human vision system (HVS) as is the detail pertaining to, for example, the details of moving objects. In attempting to retain and track the motion of spatially-detailed regions of a scene, as demanded by the HVS, the encoder eventually allocates a constant (or almost constant) intensity to each block and it is this block-artifact that is usually the most visually objectionable. It has been estimated that adjacent artifact blocks that differ in relative uniform intensity by greater than 3% are visible and objectionable. Large regions of many frames contain such artifacts.
In terms of the intensity difference between the compressed and uncompressed versions of a video, the blocky regions may not be the largest contributors to a mathematical metric of overall distortion. There is typically greater mathematical distortion in the detailed regions of a video but the HVS does not perceive that distortion as readily as it perceives the block artifacts. In particular, the HVS is highly-sensitive to the horizontal and vertical discontinuities of intensity at the edges of adjacent block artifacts in the large open regions of a video scene. Thus, since the HVS has evolved to be sensitive to edges it therefore unfortunately recognizes the edges created by block artifacts in much the same way as it recognizes the legitimate edges of objects.
Discussed herein are very effective methods for removing the undesired artifacts and for compressing the resulting video into data formats that can be easily transported to conserve bandwidth. However, assume that such data, for example a movie that is stored in compressed format on a personal device, such as a PC, is to be played (viewed) by a user on a TV. Assume also that the viewer wants the picture to be in high definition (HD) format, such as, 480p, 576p, 720p, 1080i, 1080p, 2K, 2160p, 4K, 2540p, 4320p. This then would require transporting in excess of 746 Mbits per second over a connector from the PC to the TV. Problems exist, however, in that the typical connection to/from a PC is currently the USB port which has a specified bandwidth of 400 Mbits per second for USB 2. In addition, there is a practical limitation of 300 Mbits/sec or higher, but in some situations the bandwidth can be 480 Mbits/sec.
The present invention is directed to a system and method which allows highly compressed data files to be expanded in a standalone decoder having an output at least 80 times larger than the compressed input. In one embodiment, a universally available interface is used to transport encoded media from a host PC to a decoder external to the PC for expansion and presentation as a HD display image. In one embodiment, a USB port of the user's PC (or other storage device) is used to transport data at a rate of between 0.3 Mbit/sec and 5 Mbits/sec for expansion into a robust HD signal of up to 3Gbits/sec or higher. In other embodiments, provision is made for reducing the decompressed expanded data file to below 480 Mbits/sec so as to accommodate retransport of the decompressed file back to the PC or storage device for temporary storage or display.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.
For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
Within the decoder the hardware, software and firmware could be constructed as a Field Programmable Gate Array (FPGA), a Gate Array (GA), an Application Specific Integrated Circuit (ASIC) or a combination thereof, or by using any other combination of elements. The fundamentals of decoder 100 are physical USB electrical connector 102 with its controller 103, high capacity decoder 105, video buffers 106A and 106B, digital video controller 107 and HDMI encoder 108. Also, included in the decoder, is compressor 109 which reduces the data amount in the return signal to a point that complies with the requirements of the USB connector (under 300 Mbits/sec). Compressor 109 has the ability to recompress the results of high capacity decoder 105 in whatever amount is necessary in order to pass it back across the USB bus.
In addition to what is shown, the decoder has theft protection to prevent people for gaining unauthorized access to media. This is shown in DTCP encryption block 104 which allows for encryption and decryption of the video files as they move in and out of the USB port. In this embodiment, we show a DTCP block however any type of encryption block can be used.
In operation, highly compressed video streams into the USB connector from the host device (
Decoder 105 decompresses the highly compressed video, for example, as taught in the above-identified patent applications and the uncompressed video data is streamed to buffers 106A and 106B. Any number of buffers (including zero) can be used as desired to maintain a smooth data transmission flow. At this point, the decoder output data transmission rate for this embodiment is in the range of 800 Mbits/sec which will yield a HDTV image. However, the output could be in the range of 3Gbits/sec and above, if desired. If the output is recompressed then the output would be typically under 1 Gbit/sec. The dual buffers shown are a typical double buffering implementation that allows the decoder to output one frame buffer while working on a second frame buffer and pulling it into another stage of the pipeline. However, one could reduce or eliminate the requirements for double buffering depending on the algorithms and implementations of decoder 105 and compressor 109. In this embodiment, the data passes out to digital video processor 107 and in conjunction with HDMI control 108 conditions the video for HD output to a TV or other display device. Note that any output format can be used by simply replacing, or adding, another processor type to the output of buffers 106A and B.
In situations where it is desired to feed a decoded signal back over cable A the data rate must be reduced to comply with the limitations of the bidirectional data transfer capability of the USB interface. In order to accomplish such a reduction, the decompressed data from buffers 106A and 106B is passed through compressor 109 which operates to reduce the data rate to an acceptable range. In the embodiment being discussed, compressed output from compressor 109 is sent to encryption control 104 and then via USB 103, 102 and cable A to the host device. Note that in some situations encryption in one or both directions may not be necessary and in those situations encryption control 104 can be eliminated from the path or even from the device. Also note that if and when USB interfaces can handle higher throughput, the recompression, if any, can be less.
In the embodiment shown in
When the decompressed data returns to the host device via the USB connection, either for storage, or to be viewed by a user at the host device, the data is decrypted by device 205A (or if desired, by device 210) and expanded (if necessary) by expander control to remove the compression introduced by compressor 109 of the decoder. The output from the expander is presented to a video conditioner circuit, such as circuit 206, to condition the data for presentation to a user via display 207 and/or for storage in the expanded format in memory 208. Circuit 206 could be a frame buffer, such as a graphics card, typically used in the video electronics within the system on the host device.
Note that storage of the returned expanded data file can be accomplished either before or after expansion by circuit 205, if desired.
In the embodiment shown in
In some situation there is no need to return the decoded video to the host and in those situations there is no need for compressor circuitry 109 (
In other situations, the only viewing of the video file will be via the host and in those situations there is not a need for the TV output from the decoder. However, both circuits are shown for situations where it is desired to have a more universal adaptor to accommodate both situations. Note also, that the decompressed signal can be combined with other signals, such as menus or other data particularly at the host, so that the user can have a full range of options.
In situations where USB is not available as an interface, the connection could be made for any other type of interface since the output from the host is in a relatively low data transmission range. One example would be to use the PCI interface. Other interfaces may be used in embodiments of the invention and include PCI Express, ethernet and SATA. In some situations it might be desired for a user to be able to store the decoded data file on an external memory. In this situation, a USB port could be provided to allow a user to insert a flash memory, such as memory 110, to the decoder and then be able to store the decoded data file. In some situations, the flash memory could act as the host device to provide the input to the decoder. In such a situation, the data from the flash drive could be stored on the decoder in a memory, not shown, and then the decoder connected to a PC. The connection to the PC could be eliminated, but the PC connection is a good place to obtain synchronization for the USB interface as well as to supply power (which could be supplied externally, if desired). Also, in some situations it might be desirable to use some of the computing capability of the PC across the USB interface to the decoder. This then reduces the computing power required on the decoder.
It should be noted that while the compressed video file is shown as being contained in a memory within the host, the data file can be streamed from the Internet, or other data network, perhaps at a data rate at or below 10 Mbits/sec, and the streaming data could then be sent directly to the decoder, via the USB interface, for decoding.
In one alternate embodiment, the “host” could be a TV set with a USB (or any other universally accepted) interface. In such a situation, the highly compressed streaming data from the Internet (or other data network) could be sent directly to the TV set and the decoder (dongle) plugged into the interface. In such a situation, decoding of the highly compressed data would occur outside of the TV in the dongle and the HD signal then could be externally fed back to the TV via the HDMI output of the dongle or it could be sent back via the USB interface if a lower fidelity signal is acceptable.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
This Application claims the benefit of U.S. Provisional Application 61/203,104 filed on Dec. 18, 2008, the disclosure of which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61203104 | Dec 2008 | US |
