Not applicable
Not applicable
The present application relates generally to systems and methods of video transcoding, and more specifically to systems and methods of video transcoding that employ perceptual processing techniques for enhancing the perceptual quality of transcoded video information, communications, entertainment, and other video content.
Systems and methods of video encoding are known that employ quantization techniques for reducing the bits from video encoders. Such video encoders frequently employ quantization with varying step-sizes that can have substantial effects on the output bit rates of the video encoders, and on the perceptual quality (also referred to herein as a/the “quality of experience” or “QoE”) of encoded video information, communications, entertainment, and other video content (also referred to herein as an/the “encoded video”). For example, such video encoders can employ larger quantization step-sizes to perform coarser quantization, which can decrease the output bit rates of the video encoders and diminish the QoE for the encoded video. Further, such video encoders can employ smaller quantization step-sizes to perform finer quantization, which can increase the output bit rates of the video encoders and enhance the QoE for the encoded video. Accordingly, in such video encoders, it is generally considered desirable to employ quantization step-sizes that are large enough to constrain the output bit rates of the video encoders within a given bit budget, while at the same time providing the best possible QoE for the encoded video delivered to an end user.
In such video encoders, it is also generally considered desirable to maintain a consistent QoE throughout a sequence of video frames (also referred to herein as a/the “video sequence”), without having the QoE for the video sequence vary widely from video frame to video frame. Because the end user is typically a human in most video applications, prior attempts to maintain a consistent QoE throughout a video sequence have exploited the perceptual insensitivity characteristics of the human visual system (also referred to herein as a/the “HVS”). For example, a conventional approach employed in such video encoders for maintaining a consistent QoE in a video sequence, known as “perceptual quantization,” typically includes performing coarser quantization in areas of video frames where distortion is generally less perceptible to the HVS, and performing finer quantization in areas of the video frames where distortion may be more perceptible to the HVS. Using a conventional perceptual quantization approach, such video encoders typically classify each macroblock (also referred to herein as a/the “MB”) of each video frame in a video sequence as one of a plurality of distortion imperceptibility levels, based on a spatial variation for the MB, (typically represented as the sum of the mean removed absolute difference of pixel values in the MB), a temporal variation for the MB (typically represented as the magnitude of motion vector(s) for the MB), and a brightness of the MB (typically represented as the mean of pixel values in the MB). Having classified each MB of a video frame as one of the plurality of distortion imperceptibility levels, such video encoders typically employ a lower quantization parameter (also referred to herein as a/the “QP”) or a higher QP for the MB, based on the distortion imperceptibility level of the respective MB. For example, such video encoders typically employ a lower QP for MBs having lower spatial and temporal variations and lower brightness, where the HVS can generally perceive distortion more easily. Further, such video encoders typically employ a higher QP for MBs having higher spatial and temporal variations and higher brightness, where the HVS is less likely to perceive such distortion.
Video transcoders have been increasingly employed due to the proliferation and diversity of multimedia applications, video coding standards, networks, displays, etc. In such video transcoders, video streams can be translated or “transcoded” into signal streams compressed according to coding formats supported by the networks that carry the video streams, and/or endpoint devices that receive the video streams, such as traditional personal computers (PCs), mobile PCs, personal digital assistants (PDAs), video-enabled mobile phones, mobile televisions (TVs), third and fourth generation (3G and 4G) phone sets, or any other suitable multimedia systems or devices. Such coding formats include the H.264 coding format (also referred to herein as the “MPEG-4 Advanced Video Coding (AVC) standard”), which has generally improved the efficiency of video encoding, resulting in an increased need for video transcoders that can support the H.264 coding format in real-time video communications applications, video streaming applications, etc.
As in video encoders, it is generally considered desirable in video transcoders to provide the best possible QoE for transcoded video information, communications, entertainment, and other video content (also referred to herein as a/the “transcoded video”) delivered to the end user. However, perceptual quantization approaches typically employed by video encoders for maintaining a consistent QoE in video sequences have drawbacks when employed in conjunction with video transcoders. For example, a video encoder typically receives a relatively high quality video input, whereas a video transcoder receives a video input that has already been encoded by an external video encoder, and may therefore be of lower quality. Further, the characteristics of the external video encoder that encoded the video input may be unknown to the video transcoder. Because the video input of a video transcoder has typically already been encoded by an external video encoder with possibly unknown characteristics, the video transcoder employing a conventional perceptual quantization approach may classify the MBs of video frames from the video input based on inaccurate spatial variation, temporal variation, and/or brightness information for the respective MBs, resulting in the use of QPs that may not provide the best possible QoE for the transcoded video delivered to the end user. For example, such problems with the classification of MBs of video frames within the video transcoder may be more likely to occur if the external video encoder has employed higher QPs for MBs that have higher spatial variation, higher temporal variation, and/or higher brightness. Such use of higher QPs can significantly reduce the spatial variation, and can cause a video transcoder employing a conventional perceptual quantization approach to use a smaller QP when encoding such MBs, resulting in a waste of bits where the HVS is less likely to perceive such distortion.
Moreover, the video input received at a video transcoder, and/or the output of a video decoder within the video transcoder, can carry information that may be useful in the classification of the MBs of video frames from the video input. For example, if an external video encoder from which the video transcoder received the video input employed perceptual quantization in the encoding of the video, then such information may be useful in classifying the MBs of the video frames within the video transcoder. However, a video transcoder employing a conventional perceptual quantization approach may not take into account such information when classifying the MBs. Information about the QPs employed for MBs encoded by the external video encoder using a skip coding mode may also be useful in the classification of the MBs within the video transcoder. However, video frames compressed according to the H.264 coding format do not generally carry such QP information for the MBs of video frames. Further, a video transcoder employing a conventional perceptual quantization approach may not have the capability of estimating the QPs for such MBs of video frames compressed according to the H.264 coding format when the skip coding mode is employed.
It would therefore be desirable to have systems and methods of video transcoding that employ perceptual processing techniques which avoid at least some of the drawbacks of the conventional perceptual quantization approaches described above.
In accordance with the present application, systems and methods of video transcoding are disclosed that employ perceptual processing techniques for enhancing the perceptual quality (also referred to herein as a/the “quality of experience” or “QoE”) of transcoded video information, communications, entertainment, and other video content (also referred to herein as a/the “transcoded video”). Such systems and methods of video transcoding are operative to perform perceptual processing of an input video bitstream (also referred to herein as an/the “input bitstream”) using predetermined information carried by the input bitstream. Having performed such perceptual processing of the input bitstream, the QoE for transcoded video delivered to an end user is significantly improved.
In accordance with one aspect, a system for video transcoding that employs a perceptual processing technique includes a video decoder and a video encoder. In accordance with an exemplary aspect, the video encoder includes a perceptual classification component, a rate control component, a motion estimation and compensation component, a transformation component, a perceptual quantization parameter modulation component, a quantization component, and a coding component. In accordance with another exemplary aspect, the coding component is operative to perform entropy coding. The video decoder is operative to receive an input bitstream, including a plurality of video frames, from an external video encoder. In accordance with still another exemplary aspect, the plurality of video frames from the input bitstream comprise a plurality of YUV video frames. The video decoder is further operative to decode the plurality of video frames, and to provide decoded versions of the video frames to each of the perceptual classification component, the rate control component, and the motion estimation and compensation component, within the video encoder. The video decoder is also operative to extract predetermined information from the input bitstream, and to provide the predetermined information to the perceptual classification component within the video encoder. In accordance with a further exemplary aspect, the predetermined information extracted from the input bitstream can include, for each video frame from the input bitstream, video frame information such as the video frame type and the total number of bits in the video frame, macroblock (also referred to herein as a/the “MB”) information such as the MB type for each MB in the video frame and the total number of bits in each MB in the video frame, and/or quantization parameter (also referred to herein as a/the “QP”) information such as the QP used to encode each MB in the video frame and the quantization step-size associated with the QP used to encode each MB in the video frame.
In accordance with an exemplary aspect, in a mode of operation the perceptual classification component is operative to calculate a distortion imperceptibility for each MB in each video frame from the input bitstream. In accordance with an exemplary aspect, the perceptual classification component is operative to calculate the distortion imperceptibility for each MB based at least on a calculated spatial variation for the MB, and a calculated brightness of the MB. In accordance with another exemplary aspect, the perceptual classification component is operative to adjust the calculated spatial variation for each MB, using at least some of the QP information contained in the predetermined information extracted from the input bitstream. The perceptual classification component is further operative to adjust the distortion imperceptibility for the MB based at least on a likelihood that the external video encoder employed perceptual processing in the encoding of the input bitstream, and/or on one or more estimated QPs for MBs that were encoded by the external video encoder using a skip coding mode. Using at least some of the QP information and at least some of the MB information contained in the predetermined information extracted from the input bitstream, the perceptual classification component is also operative to determine the likelihood that the external video encoder employed perceptual processing based, at least in part, on the average QP used by the external video encoder to encode the input bitstream, and the average number of bits per MB, excluding any MBs that were skipped in the respective video frame.
In further accordance with this exemplary mode of operation in this exemplary aspect, the perceptual classification component is operative to determine a distortion imperceptibility level for each MB in the video frame based at least on the calculated distortion imperceptibility for the MB. In accordance with an exemplary aspect, each distortion imperceptibility level has an associated threshold value, and the respective threshold values monotonically increase for each higher level of distortion imperceptibility. In accordance with another exemplary aspect, each threshold value is dependent upon a picture resolution, as determined from the video frame information contained in the predetermined information extracted from the input bitstream. The perceptual classification component is further operative to generate a distortion imperceptibility map using the distortion imperceptibility levels for the MBs in the video frame. In accordance with a further exemplary aspect, because large differences in QPs used to encode adjacent MBs in a video frame can adversely affect the perceptual quality of transcoded video at the boundaries of such MBs, the perceptual classification component is operative to adjust the distortion imperceptibility map using the distortion imperceptibility levels for adjacent MBs in the video frame. The perceptual classification component is also operative to provide the information contained in the distortion imperceptibility map to the perceptual quantization parameter modulation component.
In still further accordance with this exemplary mode of operation, the rate control component is operative to generate a QP and a target number of bits to be allocated for the video frame, using a predetermined rate control function, such as a video frame layer rate control function, an MB layer rate control function, a slice layer rate control function, or any other suitable type of rate control function. The rate control component is further operative to provide an indication of the QP and the target number of bits to be allocated for the video frame to the perceptual QP modulation component. Using the information from the distortion imperceptibility map, and the target number of bits from the rate control function, the perceptual QP modulation component is operative to modulate the QP for each MB in the video frame based at least on its associated distortion imperceptibility level. In accordance with an exemplary aspect, the perceptual QP modulation component is operative, for all MBs in the video frame, to find a set of QPs that provides an estimated number of bits to be allocated for the video frame that is closest to the target number of bits generated by the rate control component. The perceptual QP modulation component is further operative to provide the QPs for the respective MBs in the video frame to the quantization component. In addition, the motion estimation and compensation component is operative to perform motion estimation and motion compensated prediction on the decoded video frames, to calculate motion compensated prediction errors (also referred to herein as the “MCP errors”) for the MBs in each video frame, and to provide the MCP errors for the respective video frames to the transformation component, which, in turn, provides, to the quantization component, transform coefficients for the MCP errors that describe prediction correction. Using the QPs for the respective MBs in each video frame from the perceptual QP modulation component, and the transform coefficients for the MCP errors from the transformation component, the quantization component is operative to quantize the respective video frames, and to provide quantized versions of the respective video frames to the coding component, which, in turn, is operative to encode the video frames, and to provide the transcoded video in an output video bitstream (also referred to herein as an/the “output bitstream”) for subsequent display to the end user.
By employing predetermined information carried by an input bitstream to classify each MB in each video frame in the input bitstream, creating distortion imperceptibility maps for the respective video frames using the classifications for the respective MBs, and performing perceptual processing of the respective video frames using the information contained in the distortion imperceptibility maps, enhanced perceptual quality of transcoded video information, communications, entertainment, and other video content delivered to an end user can be achieved.
Other features, functions, and aspects of the invention will be evident from the Drawings and/or the Detailed Description of the Invention that follow.
The invention will be more fully understood with reference to the following Detailed Description of the Invention in conjunction with the drawings of which:
a is a table illustrating an exemplary distortion imperceptibility map for an exemplary video frame from an exemplary video bitstream;
b is a table illustrating another exemplary distortion imperceptibility map for the exemplary video frame after decoding, in which distortion imperceptibility levels for macroblocks in the video frame have been determined using an exemplary first perceptual classification technique that can be implemented by the video transcoding system of
c is a table illustrating still another exemplary distortion imperceptibility map for the exemplary video frame after decoding, in which the distortion imperceptibility levels for the macroblocks in the video frame have been determined using an exemplary second perceptual classification technique that can be implemented by the video transcoding system of
a is a display window illustrating the exemplary video frame, which has not been subjected to perceptual processing;
b is another display window illustrating the exemplary video frame of
Systems and methods of video transcoding are disclosed that employ perceptual processing techniques for enhancing the perceptual quality (also referred to herein as a/the “quality of experience” or “QoE”) of transcoded video information, communications, entertainment, and other video content (also referred to herein as a/the “transcoded video”). Such systems and methods of video transcoding are operative to perform perceptual processing of an input video bitstream (also referred to herein as an/the “input bitstream”) using predetermined information carried by the input bitstream. Having performed such perceptual processing of the input bitstream, the QoE for transcoded video delivered to an end user is significantly improved.
In further accordance with the illustrative embodiment of
Dp=f(MAD,brightness), (1)
in which “MAD” (also referred to herein as a/the “mean removed absolute difference” of pixel values in the MB) represents the calculated spatial variation for the MB, and “brightness” represents the calculated brightness of the MB (also referred to herein as a/the “mean” of pixel values in the MB). The perceptual classification component 106 is further operative to calculate the MAD of the respective MB by summing the MADs corresponding to a plurality of M×N blocks (e.g., 4 M×N blocks, or any other suitable number of blocks) within the MB. For example, the MAD of one of the M×N blocks within the MB, calculated by the perceptual classification component 106, can be expressed as
in which “block (i, j)” represents a pixel at the (i, j) position within the M×N block of the respective MB, which is derived from at least one component of the video frame, such as the luminance component, Y, the chrominance component, U, and the chrominance component, V. The perceptual classification component 106 is also operative to calculate the corresponding brightness of the M×N block within the respective MB. For example, the brightness of one of the M×N blocks within the MB, calculated by the perceptual classification component 106, can be expressed as
Accordingly, the distortion imperceptibility, Dp, for each MB in a video frame can be expressed as an increasing function of MAD and brightness, as follows,
Dp=α·MADw1+β·brightnessw2, (4)
in which “α” can be set to be equal to 0.8 or any other suitable value, “β” can be set to be equal to 0.2 or any other suitable value, and “w1” and “w2” can each be set to be equal to ⅔ or any other suitable value.
It is noted that if the external video encoder 118 employs a relatively large QP for encoding a relatively complicated MB from the input sequence, then some of the details contained in that MB may effectively get lost in the encoding process within the external video encoder 118. As a result, the perceptual classification component 106 may calculate an inaccurate distortion imperceptibility, Dp, for that MB. For example, the perceptual classification component 106 may produce a distortion imperceptibility, Dp, that is lower than it should be for such a relatively complicated MB, based on a spatial variation (MAD) that is too low. Consequently, the quantization component 114 may quantize such a relatively complicated MB using a QP that is smaller than warranted by its distortion imperceptibility, Dp, potentially wasting a significant number of bits in the encoding process within the video encoder 104.
To avoid such a potential waste of bits within the video encoder 104, and to reduce perceptual classification errors that may result therefrom, the perceptual classification component 106 is operative to adjust the calculated spatial variation (MAD) for each MB, using at least some of the QP information contained in the predetermined information extracted from the input bitstream. For example, the adjusted spatial variation (also referred to herein as a/the “
in which “f(QP)” is a function of the quantization step-size associated with the QP used to encode the respective MB. Moreover, if the input bitstream is to be transcoded into a signal stream compressed according to the H.264 coding format (also referred to herein as the “MPEG-4 Advanced Video Coding (AVC) standard”), then the function, f(QP), in equation (5) above, can be expressed as
f(QP)=max(1,(step_size(QP)/step_size(16))w3), (6)
in which “step_size(QP)” represents the step-size corresponding to the QP used to encode the respective MB, as defined in the H.264 coding standard, “step_size (16)” represents the step-size corresponding to a QP equal to 16, as defined in the H.264 coding standard, and “w3” can be set to be equal to 0.5 or any other suitable value.
It is further noted that an inefficient use of bits within the video encoder 104 can also occur if the external video encoder 118 employs perceptual quantization when encoding the input bitstream. For example, using perceptual quantization, the external video encoder 118 may employ, at least at some times, a smaller QP for one or more relatively simple MBs, and employ, at least at some times, a larger QP for one or more relatively complicated MBs. To avoid such inefficient use of bits within the video encoder 104, and to reduce the perceptual classification errors that may result therefrom, the perceptual classification component 106 is further operative to adjust the distortion imperceptibility for each MB based at least on a likelihood that the external video encoder 118 employed perceptual processing in the encoding of the input bitstream.
Using at least some of the QP information and at least some of the MB information contained in the predetermined information extracted from the input bitstream, the perceptual classification component 106 is operative to calculate the average QP (also referred to herein as a/the “QPavg”) used by the external video encoder 118 to encode the MBs in the video frames of the input bitstream, and to calculate the average number of bits (also referred to herein as a/the “bitsavg”) per MB in the respective video frames, excluding any MBs that may have been skipped in the video frame. The perceptual classification component 106 is further operative to determine the likelihood that the external video encoder 118 employed perceptual processing based at least on the calculated average QP, and the calculated average number of bits per MB. For example, for the coded MBs in the video frames of the input bitstream, the average QP (QPavg), calculated by the perceptual classification component 106, can be expressed as
QPavg=average(QPMB), (7)
in which “QPMB” represents the actual QPs used by the external video encoder 118 to encode the respective MBs in a respective video frame. Further, for the coded MBs in the video frames of the input bitstream, the average number of bits (bitsavg) per MB, calculated by the perceptual classification component 106, can be expressed as
bitsavg=average(bitsMB), (8)
in which “bitsMB” represents the actual number of bits for the respective MBs in the respective video frame.
Using the average QP (QPavg) and the average number of bits (bitsavg) per MB expressed above in equations (7) and (8), respectively, the likelihood that the external video encoder 118 employed perceptual processing in the encoding of the input bitstream, as determined by the perceptual classification component 106, can be expressed as
likelihoodPQ=average((QPMB−QPavg)*(bitsMB−bitsavg)/bitsavg). (9)
Based on whether or not the likelihoodPQ, as expressed in equation (9) above, exceeds a predetermined threshold value, thrlikelihood, the perceptual classification component 106 is operative to set or reset a flag, PPflag, to indicate whether or not the external video encoder 118 employed perceptual processing, as follows,
in which “thrlikelihood” can be set to be equal to 0.8 or any other suitable value.
Accordingly, using the adjusted spatial variation (
Dp=α·
in which “g(QPMB−QPavg)” is a function of the difference between QPMB and QPavg. For example, the function, g (QPMB−QPavg), may be expressed as
g(QPMB−QPavg)=4.0*(QPMB−QPavg)w4, (11.1)
in which “w4” may be set equal to 1.2, or any other suitable value.
To avoid further inefficient use of bits within the video encoder 104 and to further reduce the perceptual classification errors that may result therefrom, the perceptual classification component 106 is operative to estimate the QPs for MBs that were encoded by the external video encoder 118 using a skip coding mode, and to adjust the distortion imperceptibility, Dp, for each MB based at least on the estimated QPs for the skipped MBs. It is noted that video decoders are generally incapable of determining, from input bitstreams, the QPs that external video encoders may have employed for any skipped MBs. For example, such MBs may have been skipped because prediction errors for the MBs were relatively small, or because the external video encoders employed relatively large QPs for encoding the MBs. Such QP information for skipped MBs can be used in the function, f(QP), in equation (5) above, and/or in the function, g(QPMB−QPavg), in equation (11) above.
In accordance with the illustrative embodiment of
in which “QPpreviously
The perceptual classification component 106 is further operative to determine a distortion imperceptibility level for each MB in each video frame included in the input bitstream, based at least on the calculated distortion imperceptibility, Dp, for the MB. In further accordance with the illustrative embodiment of
It is further noted that the distortion imperceptibility, Dp, for each MB in each video frame is generally dependent on the size or resolution of the video frame that includes the MB. Accordingly, the threshold values, Thri, associated with the levels i to i+1, can be expressed as
in which “QCIF” corresponds to the size or resolution of a video frame in the quarter common intermediate format (176×144 pixel resolution), “VGA” corresponds to the size or resolution of a video frame in the video graphics array format (640×480 pixel resolution), and “picture_size” corresponds to the size or resolution of the video frame that includes the respective MB. The perceptual classification component 106 is operative to determine the picture_size from the video frame information contained in the predetermined information extracted from the input bitstream. It is noted that “αi” in equation (13) above can be set to be equal to 450, 750, 1050, 1500, 2100, 2850, and 3750 for i equal to 0, 1, 2, 3, 4, 5, and 6, respectively, or any other suitable values, and that “βi” in equation (13) above can be set to be equal to 300, 500, 700, 1000, 1400, 1900, and 2500 for i equal to 0, 1, 2, 3, 4, 5, and 6, respectively, or any other suitable values.
Using the distortion imperceptibility levels for the MBs in each video frame included in the input bitstream, the perceptual classification component 106 (see
To illustrate the reduction in perceptual classification errors that can be achieved by using the perceptual classification component 106, a mean squared error function is employed, as follows,
in which “Plevel(i, j)” corresponds to the distortion imperceptibility level located at position (i,j) in the distortion imperceptibility map 202b or the distortion imperceptibility map 202c, and “Porg
TABLE I below provides a comparison of the perceptual classification errors measured using equation (14) above, resulting from the use of the distortion imperceptibility levels from the distortion imperceptibility map 202b calculated using equation (4) above, and the use of the distortion imperceptibility levels from the distortion imperceptibility map 202c calculated by the perceptual classification component 106 using equation (11) above.
As shown in TABLE I above, the total perceptual classification error resulting from the use of the distortion imperceptibility levels calculated by the perceptual classification component 106 using equation (11) above is 0.104, and the total perceptual classification error resulting from the use of the distortion imperceptibility levels calculated using equation (4) above is 0.204, providing a classification improvement of about 49.1%. Accordingly, as illustrated by TABLE I above, a significant reduction in perceptual classification errors can be achieved using the perceptual classification component 106.
It is noted that, in real-time video communications and video streaming applications, the video encoder 104 within the video transcoding system 100 may be required to send QP information to a video decoder that is external to the video transcoding system 100 (also referred to herein as an/the “external video decoder”). For example, when an input bitstream is encoded into a signal stream compressed according to the H.264 coding format, the video encoder 104 employs a differential coding approach to reduce the number of bits for such QP information. Specifically, such QP information that is sent by the video encoder 104 to the external video decoder includes information pertaining to the difference between the QP for a currently encoded MB, and the QP for an adjacent, previously encoded MB. Further, the video encoder 104 employs fewer bits to indicate smaller QP differences, and more bits to indicate larger QP differences, in such QP information. Frequent changes in QPs for MBs in each video frame can therefore cause an increased number of bits to be consumed for such QP information. It is further noted that larger QP differences for adjacent MBs in each video frame can adversely affect the QoE for transcoded video at the boundaries of the adjacent MBs.
To avoid such potential problems involving QP information that may be sent from the video encoder 104 to an external video decoder, and adjacent MBs in each video frame included in the input bitstream, the perceptual classification component 106 is operative to adjust one or more distortion imperceptibility levels, Dplevel, in each distortion imperceptibility map, using the distortion imperceptibility levels for the adjacent MBs in the video frame. For example, the distortion imperceptibility level for a currently encoded MB, Dplevel [current], in a distortion imperceptibility map, can be adjusted by the perceptual classification component 106, as follows,
in which “i” represents one of the 4 positions for the MBs in the video frame, namely, top, bottom, left, and right, and “Dplevel [i]” represents the distortion imperceptibility level for the MB in the ith position in the video frame. The perceptual classification component 106 is further operative to provide the information contained in the distortion imperceptibility map to the perceptual quantization parameter modulation component 112.
In further accordance with the illustrative embodiment of
Using the information from the distortion imperceptibility map generated by the perceptual classification component 106, and the target number of bits generated by the rate control component 108, the perceptual QP modulation component 112 is operative to modulate the QP for each MB in each video frame included in the input bitstream, based at least on the distortion imperceptibility level for the MB. The perceptual QP modulation component 112 is further operative, for all MBs in the video frame, to find a set of QPs that provides an estimated number of bits to be allocated for the video frame that is closest to the target number of bits generated by the rate control component 108. For example, the perceptual QP modulation component 112 may employ the QP generated by the rate control component 108 as a starting point in the search for a “QP_index” to be used in performing such perceptual QP modulation.
In still further accordance with the illustrative embodiment of
Using the table of QPs (TABLE II) above, the perceptual QP modulation component 112 is operative to search for the QP_index that provides an estimated number of bits to be allocated for the P-frame that is closest to the target number of bits generated by the rate control component 108. For example, the perceptual QP modulation component 112 may employ a bisectional search algorithm that starts the search using the QP generated by the rate control component 108, and iterates the search until the QP_index that estimates the number of bits to be allocated for the P-frame closest to the target number of bits is found. In a similar fashion, a table of QPs may be constructed for each B-frame, and used to search for a QP_index that provides an estimated number of bits to be allocated for the B-frame that is closest to a target number of bits generated by the rate control component 108. The perceptual QP modulation component 112 is further operative to provide the modulated QPs for the MBs in the video frame to the quantization component 114.
In addition, the motion estimation and compensation component 110 is operative to perform motion estimation for each MB in the video frame to obtain one or more motion vectors, to perform motion compensated prediction using the motion vector(s), to calculate motion compensated prediction errors (also referred to herein as the “MCP errors”) for each MB in the video frame, and to provide the MCP errors for the respective video frames to the transformation component 111, which, in turn, transforms the MCP errors into transform coefficients describing prediction correction, and provides the transform coefficients to the quantization component 114. Using the modulated QPs for the MBs in each video frame from the perceptual QP modulation component 112, and the transform coefficients from the transformation component 111, the quantization component 114 is operative to quantize the transform coefficients for the respective video frame, and to provide quantized versions of the transform coefficients for the respective video frame to the coding component 116. Moreover, the coding component 116 is operative to encode the quantized transform coefficients for the respective video frame, and to provide the transcoded video in an output video bitstream (also referred to herein as an/the “output bitstream”) for subsequent display to the end user.
a depicts an illustrative display window 300a containing the exemplary video frame (designated by reference numeral 302a) discussed above with reference to the distortion imperceptibility maps 202a (see
b depicts another illustrative display window 300b containing the exemplary video frame (designated by reference numeral 302b). In accordance with the illustrative display window 300b of
An illustrative method 400 of operating the video transcoding system 100 is depicted in
It is noted that the operations depicted and/or described herein are purely exemplary, and imply no particular order. Further, the operations can be used in any sequence, when appropriate, and/or can be partially used. With the above illustrative embodiments in mind, it should be understood that such illustrative embodiments can employ various computer-implemented operations involving data transferred or stored in computer systems. Such operations are those requiring physical manipulation of physical quantities. Typically, though not necessarily, such quantities take the form of electrical, magnetic, and/or optical signals capable of being stored, transferred, combined, compared, and/or otherwise manipulated.
Further, any of the operations depicted and/or described herein that form part of the illustrative embodiments are useful machine operations. The illustrative embodiments also relate to a device or an apparatus for performing such operations. The apparatus can be specially constructed for the required purpose, or can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines employing one or more processors coupled to one or more computer readable media can be used with computer programs written in accordance with the teachings disclosed herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
The presently disclosed systems and methods can also be embodied as computer readable code on a computer readable medium. The computer readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of such computer readable media include hard drives, read-only memory (ROM), random-access memory (RAM), CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and/or any other suitable optical or non-optical data storage devices. The computer readable media can also be distributed over a network-coupled computer system, so that the computer readable code can be stored and/or executed in a distributed fashion.
The foregoing description has been directed to particular illustrative embodiments of this disclosure. It will be apparent, however, that other variations and modifications may be made to the described embodiments, with the attainment of some or all of their associated advantages. Moreover, the procedures, processes, and/or modules described herein may be implemented in hardware, software, embodied as a computer-readable medium having program instructions, firmware, or a combination thereof. For example, the functions described herein may be performed by a processor executing program instructions out of a memory or other storage device.
It will be appreciated by those skilled in the art that modifications to and variations of the above-described systems and methods may be made without departing from the inventive concepts disclosed herein. Accordingly, the disclosure should not be viewed as limited except as by the scope and spirit of the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
6363113 | Faryar et al. | Mar 2002 | B1 |
6366705 | Chiu et al. | Apr 2002 | B1 |
6441754 | Wang et al. | Aug 2002 | B1 |
6654417 | Hui | Nov 2003 | B1 |
20020181587 | Vetro | Dec 2002 | A1 |
20050175109 | Vetro et al. | Aug 2005 | A1 |
20050232497 | Yogeshwar et al. | Oct 2005 | A1 |
20090086816 | Leontaris et al. | Apr 2009 | A1 |
20100183069 | Chen et al. | Jul 2010 | A1 |
20110002381 | Yang et al. | Jan 2011 | A1 |
20120020415 | Yang et al. | Jan 2012 | A1 |
Entry |
---|
Liu et al., “Perceptually Motivated Adaptive Quantization Algorithm for Region-of-Interest Coding in H.264,” LNCS 5353, 2008, pp. 129-137. |
Minoo et al., “Perceptual Video Coding with H.264,” RCE Dept., UCSD, Mar. 2009, pp. 741-745. |
Puri et al., “Motion-Compensated Video Coding with Adaptive Perceptual Quantization,” IEEE Transactions on circuits and Systems for Video Technology, Dec. 1991, pp. 351-361, 1(4). |
Vetro et al., “Video Transcoding Architectures and Techniques: An Overview,” IEEE Signal Processing Magazine, Mar. 2003, pp. 18-29. |
Wiegand et al., “Overview of the H.264/AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, Jul. 2003, pp. 560-576,13(7). |
Yang et al., “Perceptual Quantization for Predictive Coding of Images,” Dec. 1999, pp. 381-385. |
Number | Date | Country | |
---|---|---|---|
20120257672 A1 | Oct 2012 | US |