Patent Application
20050229231
The implementations of the claimed invention relate to media encoding and, more particularly, to encoding received video information.
Several reasons may exist for one to encode video information. In some cases, one may wish to transcode or re-encode the video information from one format to another format for storage and/or display. Such transcoding or re-encoding may involve direct conversion of video information from a first format to a second format that differs in some way from the first format. Alternately, it may involve decoding the first format to a third, intermediate format and then encoding the third format to the second format. In other cases, one may wish to store video information for later use at a certain quality level. Other reasons, not specifically mentioned here, may exist for desiring to encode video information.
Encoding video information at high quality levels, however, may necessitate a relatively large amount of processing power. Further, technical impediments may hinder encoding of certain video information.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more implementations consistent with the principles of the invention and, together with the description, explain such implementations. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the invention. In the drawings,
The following detailed description refers to the accompanying drawings. The same reference numbers may be used in different drawings to identify the same or similar elements. In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular structures, architectures, interfaces, techniques, etc. in order to provide a thorough understanding of the various aspects of the invention. However, it will be apparent to those skilled in the art having the benefit of the present disclosure that the various aspects of the invention may be practiced in other examples that depart from these specific details. In certain instances, descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
System 100 may include a media source 110, a converter 120, an encoder 140, an optional storage/output 150, and a storage device 160. Converter 120 may send intermediate media information 132 to encoder 140 and to optional storage/output 150, if present. Converter 120 also may send other information 134 to encoder 140, and in some implementations other information 134 may include meta-information relating to intermediate media information 132.
Media source 110 may include a device that provides media information to the remainder of system 100. The media information provided by media source 110 may include video information encoded in a format such as MPEG-1, MPEG-2, MPEG-4, H.264, Windows Media Video version 9 (WMV9) and Advanced Video System (AVS) formats. The claimed invention is not limited to the formats specifically mentioned herein; rather any now-known or later-developed media format may be used in accordance with the schemes taught herein. In some implementations, the media information provided by media source 110 may include motion vectors.
In some implementations, media source 110 may include a tuner to separate a stream or channel of video information (e.g., high definition (HD) MPEG-2 information) from other streams or channels of media information. In some implementations, media source 110 may include a reader to read the media information from a storage medium. For example, such a reader may include an optical, magnetic, and/or electrical reader to extract the video information from a DVD, hard disk, semiconductor storage device, or other storage medium.
Converter 120 may convert the video information from media source 110 into intermediate media information 132 and other information 134. In some implementations, converter 120 may extract other information 134 from the video information from media source 110 before the video information is converted into intermediate media information 132. In some implementations, converter 120 may decode the video information (e.g., substantially reverse its encoding) into intermediate media information 132. In some implementations, converter 120 may transform the video information (e.g., modify without reversing its encoding) into intermediate media information 132. One possible implementation of converter 120 is illustrated in
Decoder 210 may also extract other information 134 from its input video information. Such other information 134 may include motion vectors that describe displacements of macroblocks within the video information. Such motion vectors may be generated by the original encoder of the video information, and may be extracted from the information by decoder 210. Decoder 210 may also extract quantization data (e.g., used in variable uniform quantization of discrete cosine transform (DCT) coefficients to reduce the number of bits required to represent them) and picture data (e.g., whether a certain frame includes an intracoded (I) picture, a predicted (P) picture, or a bi-directionally predicted (B) picture). In some implementations, decoder 210 may not include certain information (e.g., a broadcast flag or other rights management information) in other information 134, instead focusing on information that may be re-used to later encode the video information.
Decoder 210 may use frame store 220 to buffer information as needed during its decoding and other processing. D/A converter 230 may receive the decompressed video steam of pixels from decoder 210, and may convert the stream to an analog signal (e.g., intermediate media information 132). Because of the nature of D/A conversion, D/A converter 230 may introduce some error into the analog intermediate media information 132 relative to the decompressed video steam of pixels received from decoder 210.
Although other information 134 in
Returning to
In
A/D converter 240 may convert the analog intermediate media information 132 into a corresponding stream of digital pixels. A/D converter 240, however, may introduce some quantization error in the conversion. Hence, the stream of digital pixels leaving A/D converter 240 may be different from the digital pixels entering D/A converter 230 due to the double conversion. The digital pixels from A/D converter 240 may be close enough, however, to allow reuse of the motion vectors, quantization data and picture data (e.g., other information 234) from decoder 210.
Frame control block 250 may control the capture of the digital pixel stream from A/D converter 240 and may send the data to the compression block 260. Frame control block 250 may use frame store 280 to buffer information as needed during its processing. The data fed to compression block 260 may be blocks of pixels pointed to by the motion vectors after they have been decoded, as well as any error data associated with the block of pixels. In some implementations, the associated error data may be recreated by frame control block 250. In some implementations, however, the associated error data may be included in other information 134 extracted by decoder 210.
Rate control block 270 may receive the quantization data and the picture data in other information 134 from decoder 210. If, for example, the picture data indicates that a particular frame is an I frame or that a particular block is coded as an intra-frame block, then the block of pixels may be fed directly from frame control block 250 to compression block 260. Compression block 260 may be arranged to compress/encode received picture information via certain schemes, such as DCT, quantization, and variable length coding (VLC) operations in the case of MPEG-2 encoded data.
Because the digital pixels in frame store 280 of encoder 140 may be different from the corresponding pixels in the frame store 220 of the converter 120, the error data also may be different. This difference in error data may result in different data rates for the compressed stream output by compression block 260. Rate control block 270 may be designed to increase the quantization parameters when needed to bring the bit rate down to a target bit rate. Similarly, rate control block 270 may modify the quantization parameters, if needed, to increase the output bit rate of compression block 260. Otherwise, the rate control block 270 may follow the clues given to it by decoder 210 in the quantization data in other information 134.
Frame control block 250's reuse of the motion vectors extracted from the original video stream allows compression block 260 to eliminate motion estimation processing, saving numerous computations. In some implementations, compression block 260 may perform a relatively simple, localized motion search to compensate for errors introduced by A/D converter 240's re-digitization of the pixels. For example, in typical video compression the block matching may involve searching over a fairly large area. Typically, 100×70 pixel search windows may be used, with matching done at sub-pixel levels, necessitating many billions of operations per second. Reusing the motion vectors in other information 134 from decoder 210 may permit frame control block 250 to use a much smaller search window, perhaps as small as +2 pixels, to compensate for the errors introduced in the D/A and A/D conversions.
The operation of rate control block 270 also may be significantly reduced by the reuse of the quantization data and picture data in other information 134 that is extracted from the original stream. Even though the stream of output video information generated by compression block 260 may not match the original stream input to decoder 210, a user may experience substantially the same video quality when the output video information is played back. The schemes described herein enable a design of encoder 140 that is less complex and needs less computational power than would be possible otherwise.
Although encoder 140 in
Returning to
In some implementations, storage/output 150 may include a storage device for storing intermediate media information 132. For example, storage/output 150 may include a hard disk or flash memory to store information 132. In some implementations, storage/output 150 may include a writeable optical drive (e.g., DVD-RW, etc.) to transfer intermediate media information 132 to a portable storage medium. In some implementations, optional storage/output 150 may not be present, and intermediate media information 132 may be used only by encoder 140.
Storage 160 may include a storage device for storing the encoded video information output by encoder 140. A processor (not shown) may access the encoded video information in storage 160 for playback or some other purpose at a later time. In some implementations, storage 160 may include a hard disk or flash memory to store the encoded video information. In some implementations, storage 160 may include a writeable optical drive (e.g., DVD-RW, etc.) to transfer the encoded video information to an optical storage medium. Although several exemplary implementations have been discussed for storage 160, the claimed invention should not be limited to those explicitly mentioned, but instead should encompass any device capable of storing the encoded video information from encoder 140.
Processing may continue with converter 120 converting the encoded video information into an intermediate format for intermediate media information 132 [act 320]. In some implementations, the intermediate format may include analog pixel data, as in
Encoder 140 may encode intermediate media information 132 using other information 134 that was extracted by converter 120 [act 330]. In some implementations (e.g., in
In some implementations, encoder 140 may use motion vectors from other information 134 to aid in encoding intermediate media information 132. Encoder 140 may also use one or more of quantization data, picture data, and/or error data, if available, in other information 134 to assist in encoding intermediate media information 132. In some implementations, encoder 140 may use any other information that is computationally expensive to regenerate and that may be extracted by converter 120, in addition to or instead of the motion vectors to aid in encoding intermediate media information 132.
Processing may continue with storage 160 storing the encoded video information output by encoder 140 [act 340]. Such storing may occur on magnetic, semiconductor, optical, or any other suitable media or device capable of storing such encoded video information. In some implementations, act 340 may include, additionally or alternately, outputting the encoded video information from encoder 140 for use (e.g., playback, transmission, by another portion of system 100 (not shown) or another system. Although not explicitly shown in
The foregoing description of one or more implementations consistent with the principles of the invention provides illustration and description, but is not intended to be exhaustive or to limit the scope of the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of various implementations of the invention.
For example, any audio information in the media data input to converter 120 may be converted to analog and back to digital for re-compression by encoder 140, because the compute energy required is relatively minimal. Further, although shown as directly connected in
Moreover, the acts in
No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Variations and modifications may be made to the above-described implementation(s) of the claimed invention without departing substantially from the spirit and principles of the invention. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
