Patent Grant
8660182
The invention generally relates to computer systems, and more particularly relates to MPEG motion estimation.
Moving Pictures Experts Groups (MPEG) is an International Standards Organization (ISO) standard for compressing video data. Video compression is important in making video data files, such as full-length movies, more manageable for storage (e.g., in optical storage media), processing, and transmission. In general, MPEG compression is achieved by eliminating redundant and irrelevant information. Because video images typically consist of smooth regions of color across the screen, video information generally varies little in space and time. As such, a significant part of the video information in an image is predictable and therefore redundant. Hence, a first objective in MPEG compression is to remove the redundant information and leaving only the true or unpredictable information. On the other hand, irrelevant video image information is information that cannot be seen by the human eye under certain reasonable viewing conditions. For example, the human eye is less perceptive to noise at high spatial frequencies than noise at low spatial frequencies and less perceptive to loss of details immediately before and after a scene change. Accordingly, the second objective in MPEG compression is to remove irrelevant information. The combination of redundant information removal and irrelevant information removal allows for highly compressed video data files.
MPEG compression incorporates various well-known techniques to achieve the above objectives including: motion-compensated prediction/estimation, Discrete Cosine Transform (DCT), quantization, and Variable-Length Coding (VLC). In general, prediction/estimation is a process in which past information is used to predict/estimate current information. There is typically a difference/error between the past information used and the actual/current information. As part of the compression scheme, this difference (instead of the actual/current video information) is transmitted for use in reconstructing/decoding a compressed video frame by essentially adding it to existing past information that may be referred to as a reference frame. How well the decompression process performs depends largely on the estimate of this difference. When successive video frames involve moving objects, the estimate must also include motion compensation. This is done through the use of motion vectors which are the displacement measurements of objects between successive video frames. These motion vectors are then additionally transmitted as part of the compression scheme to be used in reconstructing/decoding the compressed video frame.
One of the motion-compensated estimation techniques that is most suitable for hardware implementation due to its consistency and simplicity is block matching. In block matching, motion is estimated on the basis of blocks and a motion vector is generated for each block under the assumption that all the pixels within a block have the same motion activity. In short, a block from a search area in the reference video frame (i.e., a frame that has been received and/or processed previously) is identified through a search based on a match selection criteria relative to a block from a present frame. Such selection criteria is typically designed to ensure a minimized estimation difference. The most effective search but also the most processing and computing intensive is a full exhaustive search in which every block within the search area is examined and corresponding computation made. If a search area is limited to ±16 pixels displacement in the X and Y directions, then the total number of matches need to be made is approximately (2*(16)+1)2=1089. The match selection criterion used for the full search may be the Sum of Absolute Difference (SAD) (other match selection criteria include mean absolute difference, mean square difference, etc.). The SAD for a block A of size N×N inside the current frame compared to a block B of a distance (Δx, Δy) from A in the previous (or reference) frame is defined as:
As shown in the SAD equation above, an addition and a subtraction operation are required for each pixel match. Hence, an approximate total of 2178 operations are required for each pixel match in a full search. Consequently, each macroblock (16×16 pixels) requires approximately 2178×256 or 557K operations which is processor intensive and therefore undesirable. The corresponding blocks from the current frame and the reference frame with the smallest SAD value are then selected as the best matched (i.e., having the least difference/error) for transmission as compression information. The associated motion (displacement) vector is computed from the selected pair of blocks for use as motion compensation information.
To reduce the processing needed while minimizing estimation difference/error, other search techniques have been developed. One such search techniques is the Diamond Search (DS). In a DS, which is based on the assumption that motion vectors are in general center biased, a search area (in a block in the reference frame) includes nine checking points as shown for example in
While DS typically requires only a fraction of the processing required in a full search, a DS is susceptible to getting caught up with local minimums which are not desirable because they may not represent the best matched macroblock. In other words, while the DS search relies on the inherent center-biased nature of motion vectors and allows for iteration searches to examine additional checkpoints, it has no mechanism to ensure that the minimum SAD can be quickly determined.
Moreover, the paper “A New Predictive Diamond Search Algorithm for Block Based Motion Estimation” cited above takes advantage of the high correlation of neighboring macroblocks (and therefore their associated motion vectors) and use as the starting point of the DS the median value of the motion vectors of three neighboring blocks: left macroblock LMB relative to the current macroblock that is designated the center of the search area, up macroblock UMB relative to the current macroblock that is designated the center of the search area, and up-right macroblock URMB relative to the current macroblock that is designated the center of the search area. In other words, instead of using the center pixel of the diamond as a starting point, the median of these three neighboring blocks is used. By taking into consideration the correlation between adjacent macroblocks (and their associated motion vectors), an improved prediction can be made thereby shortening the search.
Furthermore, in MPEG-4, in addition to a stage involving the aforementioned DS which is an integer pixel motion estimation, a half-pixel motion estimation stage may be implemented. As its name suggests, a half-pixel motion estimation involves a search of checking points that are at a half-distance between two checking points of the integer pixel motion estimation search. The half-distance can easily be interpolated from the checking points of the integer pixel motion estimation search. The half-pixel motion estimation stage is designed to improve the accuracy of motion vectors. See “A New Predictive Diamond Search Algorithm for Block Based Motion Estimation” by W. Zheng, I. Ahmad, and M. Liou, International Conf. on Information Systems, Analysis and Synthesis, SCI 2001/ISAS 2001 Vol. 13, 2001. It is desirable to reduce even further the processing required for motion-compensated estimation which translates to less power and smaller die size required.
Thus, a need exists for a more efficient, less complex, and effective motion-compensated estimation technique that can be easily implemented in hardware.
Accordingly, the present invention provides a more efficient, less complex, and effective motion-compensated estimation technique that can be easily implemented in hardware.
The present invention meets the above need with a motion estimation technique that involves selecting a plurality of predicted start motion vectors, performing coarse searches using the plurality of predicted start motion vectors (either sequentially or in parallel), determining a Sum of Absolute Difference (SAD) value and the associated vector from the coarse searches, and performing a fine search using the vector determined from the coarse searches as a starting position. In the preferred embodiment, the plurality of predicted start motion vectors are two (2) start motion vectors related to the macroblock that is immediately above the current macroblock (a.k.a. the UP macroblock) and the macroblock that is immediately on the left of the current macroblock (a.k.a. the LEFT macroblock). In accordance with the present invention, the correlation between adjacent macroblocks (and their associated motion vectors) is utilized by using two predicted starting points based on adjacent macroblocks. This results in an improved prediction (especially for fast motion videos) which in turn shortening the search.
In the preferred embodiment, the coarse searches are 16×16 Diamond Searches (DS) and the fine search involves an 8×8 search and a half-pixel search. More particularly, the DS's are based on a 5-Steps Diamond Search (5SDS). 5SDS involves five checking points. As a result of the use of a plurality of predicted starting points as well as other improvements, the processing required for carrying out the whole search is significantly reduced thereby rendering an efficient motion estimation technique that requires minimal system resources (e.g., hardware resources and CPU processing).
All the features and advantages of the present invention will become apparent from the following detailed description of its preferred embodiment whose description should be taken in conjunction with the accompanying drawings.
In the following detailed description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be obvious to one skilled in the art that the present invention may be practiced without these specific details. In other instances well known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the present invention. While the following detailed description of the present invention is related to MPEG compressed video image data, it is to be appreciated that the present invention is also applicable to other video compression schemes.
The motion estimation technique in accordance with the present invention utilizes a plurality of predicted start points (e.g., two predicted start points) based on blocks adjacent to the current block together with other improvements to provide a more efficient motion estimation process that requires minimal system resources (e.g., hardware resources and CPU processing) in its hardware implementation. More particularly, the motion estimation technique in accordance with the present invention performs a plurality of coarse searches (either sequentially or in parallel) using a plurality of start motion vectors followed by a fine search.
Referring now to
In scenario 1, if the encoded mode of current macroblock 412 is set as Intra (a.k.a, I-encoded), typically a whole frame is Intra, no estimation search is performed. In scenario 2, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is Inter then starting point V0 is set to the motion vector of UP 8×8 block 418. In scenario 3, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is Intra then starting point V0 is set to (0,0). In scenario 4, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is out of video range, the current macroblock is considered an I-encoded macroblock and the starting point is set to (0,0).
Alternatively, for scenario 2, the start motion vector V0 can also be set to the motion vector of any of the remaining 8×8 blocks of UP macroblock 414 or to the motion vector of UP macroblock 414 itself. Or, the starting point V0 can be set to the motion vector of an 8×8 block in LEFT macroblock 416 or the motion vector of LEFT macroblock 416 itself whose position relative to current macroblock 412 in current frame 400 is illustrated in
Using the first predicted start motion vector selected in step 212, a large diamond search is performed (step 214). A description of this large diamond search is provided below. The large diamond search involves searching the checking points (V0.x,V0.y), (V0.x,V0.y+4), (V0.x,V0.y−4), (V0.x+4,V0.y), and (V0.x−4,V0.y) located in the current video frame which correspond to motion vectors V0, V0+(0,4), V0−(0,4), V0+(4,0), and V0−(4,0). The first predicted start motion vector is the center of the five checking points. As indicated, the step offset is 4 pixels. In this large diamond search, which is illustrated in
where IV is the intensity level of the pixels inside the macroblock inside the previous frame, ICP is the intensity level of the pixel at a checking point inside the current frame, and N is 16 (the number of rows and the number of columns in macroblock).
Essentially, the SAD value indicates how well a particular block in the current frame matches a block at a position in the previous frame. The smaller the SAD value the better match it is. SAD calculation is center-biased. If the vector associated with a pixel position for which a SAD value is computed is a center point (0,0), the SAD(v) value is subtracted by a predetermined factor FAVOR_0 preferably set at 129 to favor the selection of this vector as the one having the minimum SAD. In so doing, SAD calculation is ensured to be center biased. Additionally, if the range of the X or Y component of a motion vector exceeds [−15, 15], then the SAD associated with this vector is maintained at the maximum allowable SAD value of 255×256. In other words, when the X or Y component of a motion vector exceeds the allowable range, the motion vector is excluded by assigning a large SAD value to it.
From the SADs calculated for each checking point relative to the pixels within the macroblock inside the reference frame, the pixel position (hereinafter referred to as vector) with the minimum SAD and the corresponding SAD value are selected and stored. Preferably, the search order for the checking points is V0−(4,0), V0, V0+(4,0), V0−(0,4), and V0+(0,4).
In terms of computation operations, for a large diamond search, the SAD value calculation for each vector inside the macroblock associated with the first predicted start motion vector requires 5 (checking points)×2 (one addition operation and one subtraction operation)=10 operations.
Next, using the vector selected and stored in step 214 as the starting position, a small diamond search based on a 5SDS algorithm is performed (step 216). The small diamond search involves searching the checking points (v.x,v.y), (v.x,v.y+1), (v.x,v.y−1), (v.x+1,v.y), and (v.x−1,v.y) located in the current video frame which correspond to motion vectors v (the motion vector stored in step 214), v+(0,1), v−(0,1), v+(1,0), and v−(1,0). As indicated, the step offset is 1 one pixel. The starting point is the center of the five checking points. The small diamond search of step 216 is similar to the large diamond search of step 214. Equation 1 from above is also used to calculate the SADs for the checking points relative to the pixels inside the macroblock inside the previous/reference frame. However, unlike the large diamond search in step 214 which finishes when the SADs for all checking points are computed and the relative minimum SAD among the five checking points is determined, the small diamond search in step 216 will finish if the pixel with the minimum SAD is the center of the diamond made up by the checking points or if the search iteration reaches four (4).
In terms of computation operations, for a small diamond search, the SAD value calculation for each vector inside the macroblock associated with the first predicted start motion vector requires a maximum of (4+3+3+3)×2=26 operations. This is due to the maximum 4 iterations involved and the geometry of the diamond when the step offset is one which dictates that for each iteration there is one or two checking points that involve a recalculation of a vector position that have been calculated previously.
In step 218, the minimum SAD of the small diamond search is determined. The vector with the minimum SAD and the SAD value associated is stored in memory. Next, it is determined whether a second coarse search is required (step 220). To do so, the distance between the second predicted starting position motion vector and the motion vector having the minimum SAD value calculated and stored in step 218 (the result of the coarse search using the second predicted starting position) is computed and the result compared to a predetermined value. Preferably, this predetermined value is 4 which is selected because it represents a threshold above which the result of the first coarse search may be a local minimum. Hence, a second search is required if:
|V0.x−v.x|+|V0.y−v.y|≧4 (2)
where V0 is the second predicted starting position (motion vector) and v is the motion vector associated with the minimum SAD stored in step 218.
If a second search is not needed, skip to step 232. On the other hand, if a second search is needed, step 222 is performed to select a second predicted start motion vector associated with a macroblock as the starting position of the second coarse search. Depending on the encoded mode associated with current video macroblock 412 of
In scenario 1, if the encoded mode of current macroblock 412 is set as Intra (a.k.a, I-encoded), typically a whole frame is Intra, no estimation search is performed. In scenario 2, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is Inter then starting point V0 is set to the motion vector of UP 8×8 block 418. In scenario 3, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is Intra then starting point V0 is set to (0,0). In scenario 4, if the encoded mode of current macroblock 412 is set as Inter (a.k.a. P-encoded) and UP macroblock 414 is out of video range, the current macroblock is considered an I-encoded macroblock and the starting point is set to (0,0).
Alternatively, for scenario 2, the start motion vector V0 can also be set to the motion vector of any of the three remaining 8×8 blocks of LEFT macroblock 416 or to the motion vector of LEFT macroblock 416 itself. Or, the starting point V0 can be set to the motion vector of an 8×8 block in UP macroblock 414 or the motion of UP macroblock 414 itself whose position relative to current macroblock 412 in the current frame 400 is illustrated in
Using the second predicted start motion vector selected in step 222, a large diamond search is performed (step 224). Because the large diamond search including the checking points is substantially similar to that described in step 214, it is not repeated here. As a result of the large diamond search, the vector and the associated minimum SAD value are stored separately from the vector and associated minimum SAD value that are the result of the first coarse search. Next, using the vector selected and stored in step 224 as the starting position, a small diamond search is performed (step 226). The small diamond search is substantially similar to that performed in step 216 and is not repeated here.
In step 228, the minimum SAD of the small diamond search is determined. The vector with the minimum SAD and the SAD value associated are stored in memory separately from the vector and associated minimum SAD value that are the result of the first coarse search. Next, the minimum SAD value stored in step 218 and the SAD value stored in step 228 are compared to determine the vector with the smallest (minimum) SAD value (step 230). Using the vector with the smallest SAD value determined from step 230 as input, a fine search is performed in step 232. Preferably, the fine search includes an 8×8 diamond search followed by a half-pixel search. In essence, the fine search carries out the search in a narrow area to improve the resolution and accuracy of the search.
The 8×8 search may be disabled through a programmed register. In an 8×8 search, the search area, where the checking points are located, is reduced from 16×16 pixels (a macroblock) to 8×8 pixels, the number of checking points are nine in a square (instead of diamond) pattern, the center of nine checking points is the vector from step 230, and there are only two steps/iterations.
In terms of computation operations, for an 8×8 search, the SAD value calculation for each vector inside the 8×8 pixels block of the reference frame requires a maximum of [(9)×2]+(5*2)=28 operations.
The sum of the four minimum SADs of the four 8×8 blocks (subdivided from the macroblock) is then computed and stored. In shorthand, this sum can be referred to as ΣSAD(8×8). Next a determination is made to determine whether ΣSAD(8×8) is greater than the smallest SAD value stored in step 230 minus FAVOR—16×16. FAVOR—16×16 is a weight factor preset to 129 to favor the selection of the minimum SAD associated with a macroblock which is stored in step 230. If it is, the new minimum SAD value is set to ΣSAD(8×8) and set flag mv4flag to 1 to indicate that four motion vectors (MV4) from four 8×8 blocks are used. The four minimum SAD values along with the four motion vectors associated with the four vectors having the minimum SADs are then stored in memory. Otherwise, the vector and the smallest SAD value determined from step 230 are maintained and set flag mv4flag to 0 indicating that only one motion vector is required for each macroblock.
In the preferred embodiment, the half-pixel search is performed as part of the fine search following the 8×8 search. The half-pixel search may be disabled through a programmed register. The same search steps apply whether one macroblock motion vector is involved or four 8×8 pixels block motion vectors are involved. First, flag mv4flag is checked. If mv4flag=0 indicating one macroblock motion vector is involved, the vector with the smallest SAD value stored in step 230 is used as a starting point. In addition, four half-pixel positions (hereinafter half-pixel vectors) relative to the starting point are determined. Hence, assuming the starting vector is (v.x,v.y), then the four half-pixel vectors are (v.x,v.y+½), (v.x,v.y−½), (v.x+½,v.y), and (v.x−½,v.y). The corresponding motion vectors are v, v+(0,½), v−(0,½), v+(½,0), and v−(½,0). These five vectors serve as the five checking vector wherein the step offset is % pixel. The intensity value of a half-pixel vector is determined by averaging the intensity values of the top and bottom pixels (whose offset distance is one) or the left and right pixels (whose offset distance is one) between which the half-pixel vector is located. If the half-pixel vector is located in the center of four pixels whose offset distances are one, then the intensity value of that half-pixel vector is determined by averaging the intensity values of these four pixels.
Conversely, if mv4flag=1 indicating that four motion vectors are required for each macroblock, the same two steps search described above is applied to each 8×8 pixels block. For each 8×8 pixels block, the vector with the smallest SAD value stored in system memory at the conclusion of the 8×8 search is used as a starting point. In addition, four half-pixel vectors relative to the starting point are determined. Hence, assuming the starting vector is (v.x,v.y), then the four half-pixel vectors are (v.x,v.y+½), (v.x,v.y−½), (v.x+½,v.y), and (v.x−½,v.y). The corresponding motion vectors are v, v+(0,½), v−(0,½), v+(½,0), and v−(½,0). These five vectors serve as the five checking vector wherein the step offset is % pixel.
After the half-pixel searches for all four motion vectors associated with four 8×8 pixels blocks are carried out, the four computed “minimum” SADs are compared to each other to determine the smallest SAD value and the vector with this smallest SAD value. These information are stored in memory and provided to rate control module 711 ending the serial motion estimation process in accordance with the present invention.
In terms of computation operations, for the half-pixel search, the worst case scenario is when 4 motion vectors are involved. In this case, the SAD value calculation for each vector in the half-pixel search requires 4*(1+1+1) operations for the first step and 2*(3+1+1) operations for the second step. The total number of operations per pixel for the half-pixel search are 22. The calculations include the half pixel interpolation as well as operations required for SAD calculation. The total number of operations for the entire motion compensated estimation process (assuming two coarse searches) in accordance with the present invention are 122.
Referring now to
Two large diamond searches which are substantially similar to steps 214 and 224 of
Next, the minimum SAD values determined from the two small diamond searches are compared against each other (step 320). Using the vector with the smaller SAD value determined from step 320 as input, a fine search is performed in step 322. The fine search includes an 8×8 search and a half-pixel search which are substantially similar to those in step 232 of the
Reference is now made to
As shown in
While peripheral controller 502 is connected to integrated processor circuit 501 on one end, ROM 503 and RAM 504 are connected to integrated processor circuit 501 on the other end. Integrated processor circuit 501 comprises a processing unit 505, memory interface 506, graphics/display controller 507, direct memory access (DMA) controller 508, and core logic functions including encoder/decoder (CODEC) interface 509, parallel interface 510, serial interface 511, and input device interface 512. Processing unit 505 integrates a central processing unit (CPU), a memory management unit (MMU), together with instruction/data caches.
CODEC interface 509 provides the interface for an audio source and/or modem to connect to integrated processor circuit 501. Parallel interface 510 allows parallel input/output (I/O) devices such as hard disks, printers, etc. to connect to integrated processor circuit 501. Serial interface 511 provides the interface for serial I/O devices such as Universal Asynchronous Receiver Transmitter (UART), Universal Serial Bus (USB), and Firewire (IEEE 1394) to connect to integrated processor circuit 501. Input device interface 512 provides the interface for input devices such as keyboard, mouse, and touch pad to connect to integrated processor circuit 501.
DMA controller 508 accesses data stored in RAM 504 via memory interface 506 and provides the data to peripheral devices connected to CODEC interface 509, parallel interface 510, serial interface 511, or input device interface 512. DMA controller 508 also sends data from CODEC interface 509, parallel interface 510, serial interface 511, and input device interface 512 to RAM 504 via memory interface 506. Graphics/display controller 507 requests and accesses the video/graphics data from RAM 504 via memory interface 506. Graphics/display controller 507 then processes the data, formats the processed data, and sends the formatted data to a display device such as a liquid crystal display (LCD), a cathode ray tube (CRT), or a television (TV) monitor. In computer system 500, a single memory bus is used to connect integrated processor circuit 501 to ROM 503 and RAM 504.
In the preferred embodiment, the present invention is implemented as part of graphics/display controller 507. Reference is now made to
The frame buffer in SRAM 602 is used to store the pixmap (i.e., a pixel pattern mapped into the frame buffer) of the image to be displayed on the monitor as well to act as a temporary buffer for various purposes. Additionally, SRAM 602 may have memory allocated for video buffers and transactional registers. The transactional registers can be used to store quantization factors for each block of video image pixel data. However, the transactional registers used in storing quantization factors are preferably implemented inside MPEG-4 decoder 612. GE 606 processes graphics/video image data which is then stored in the buffer in SRAM 602 based on commands issued by the host CPU. GE 606 performs graphics operations (e.g., BitBLTs and ROPs, area fills, line drawing) and provides hardware support for clipping, transparency, rotation, and color expansion. In the present embodiment, GE 606 through a built-in Stretch Block Transfer (STRBLT) function further performs video image expansion, progressive scanning conversion, YcbCr (YUV) to RGB color-space conversion, etc. In short, GE 606 frees processing unit 505 from the video/image display rendering function to allow processing unit 505 to perform time-critical or real-time operations.
MIU 607 controls all read and write transactions from/to the frame buffer, video buffers, and transactional registers in SRAM 602. Such read and write requests may come from the host CPU via CIF 601, GE 606, pixel processing logic 608, FPI 609, etc. In addition, MIU 607 performs tasks associated with memory addressing, memory timing control, and others. Post-processing module 611 removes blocking and ringing artifacts from decompressed MPEG video image data to improve the quality of the decompressed video data. The decompressed MPEG video image data can be received from, for example, an optical media player via serial interface 511 or MPEG-4 decoder 612. The filtered video image data is then sent to SRAM 602.
Pixel processing logic 608 retrieves video/image data from the buffers in SRAM 602 via MIU 607, serializes the image data into pixels, and formats the pixels into predetermined formats before outputting them to FPI 609 or CRT DAC 610. Accordingly, pixel processing logic 608 generates the required horizontal and vertical display timing signals, memory addresses, read requests, and control signals to access image data stored in SRAM 602. If the display device involved is a LCD, pixel data from pixel processing logic 608 is sent to FPI 609 before being passed on to the LCD. FPI 609 further processes the data by further adding different color hues or gray shades for display. Additionally, depending on whether a thin film transistor (TFT) LCD (a.k.a., active matrix LCD) or a super twisted nematic (STN) LCD (a.k.a., passive matrix LCD) is used, FPI 609 formats the data to suit the type of display.
Furthermore, FPI 609 allows color data to be converted into monochrome data in the event a monochrome LCD is used. Conversely, if the display device is a cathode ray tube (CRT), pixel data is provided to CRT digital-to-analog converter (DAC) 610 prior to being sent to the CRT. CRT DAC 610 converts digital pixel data from pixel processing logic 608 to analog Red Green and Blue (RGB) signals to be displayed on the CRT monitor.
Reference is now made to
Video input from a source such as a video camera connected to system 500 is provided to memory 704. The video input from the source is the current video frame data. Preferably, memory 704 is separated into a current frame area to store data from the current video input, a reference frame/reconstructed frames area to store data from the reference video frame and data from a video frame reconstructed from compression, and an encoded bitstream buffer to store data from a freshly encoded video frame by encoder 613. Cache 709 fetches current video frame data from memory 704 one macroblock at a time and reference video frame data several macroblocks at a time (these several macroblocks are adjacent neighbors). Cache 709 receives as input the motion vectors associated with the “good” match video block determined from motion estimator 710. Cache 709 provides data from the current video frame and data from the reference video frame to MC− 701 and motion estimator 710 based on the motion vectors of the “good” match video block received. Cache 709 also provides data from the reference video frame to MC+ 708 based on the motion vectors of the “good” match video block received.
MC− 701 is essentially a subtractor in which prediction data from a reference video frame is subtracted from data from a current video frame, which has been presented in the correct order for encoding according to the desired Group Of Pictures (GOP) structure. The subtractor is disabled (e.g., the prediction is set to zero) for I-frames or I-macroblocks. The output of MC− 701, which is the prediction error (or the video input in the case of I-frames), is passed to DCT module 702 which performs the Discreet Cosine Transformation and outputs DCT coefficients to quantizer 703. Quantizer 703 carries out the quantization process which may be generalized as a step to weight the DCT coefficients based on the amount of noise that the human eye can tolerate at each spatial frequency so that a reduced set of coefficients can be generated. The quantized DCT coefficients are provided to buffer 705 for temporary storage before they are passed on to AC/DC module 712 and inverse quantizer 706. AC is typically defined as a DCT coefficient for which the frequency in one or both dimensions is non-zero (higher frequency). DC is typically defined as a DCT coefficient for which the frequency is zero (low frequency) in both dimensions. AC/DC module 712 predicts the AC and DC for the current block based on AC and DC values of adjacent blocks such as an adjacent left top block, a top block, and an adjacent left block. For example, the prediction can be made as follows:
where a is the AC or DC value of the left top block relative to the current block, b is the AC or DC value of the top block relative to the current block, c is the AC or DC value of the left block relative to the current block, and x is the AC or DC value of the current block.
Such predictions are described in “MPEG-4 Information Technology-Coding of Audio-Visual Objects-Part 2: Visual” ISO/IEC/14496-2:1999 which is herein incorporated by reference.
AC/DC module 712 provides the AC and DC predicted coefficients to ZZ module 713 which forms a 64-elements long vector such that the low frequency (e.g., DC) components are placed at the beginning of the vector. ZZ module 713 provides the 64-elements vector to RLE module 714 which basically determines the number of consecutive zeros in the vector and forms RLE acceptable words based on the determination. After quantization, there are likely a significant number of zeros (likely to be the high frequency components) in the block and there is no need to transmit or store such information. Accordingly, a RLE word represents the number of zeros between consecutive non-zero elements in the vector. The RLE word also includes the value of the last non-zero element after the zeros and information indicating whether this value is the very last component in the vector.
The RLE words are provided to VLC module 704 which maps RLE words into VLC patterns. For example, certain RLE words are given specific bit pattern. The most common RLE words are given the shortest VLC bit pattern. VLC patterns are specified in MPEG-4 standard. (See “MPEG-4 Information Technology-Coding of Audio-Visual Objects-Part 2: Visual” ISO/IEC/14496-2:1999). Run-length and variable-length coding (the combination coding is commonly referred to as Huffman coding). In general, due to the non-uniform distribution of the DCT coefficients wherein most of the non-zero DCT coefficients of an image tend to be located in a general area, VLC and run-length encoding are used to exploit this distribution characteristic to identify non-zero DCT coefficients from zero DCT coefficients. In so doing, redundant/predictable information can be removed. The encoded (i.e., compressed) block of video frame data is then sent to memory 704 for storing in the encoded frame cache.
The process of motion compensated prediction requires a signal on which to base the prediction. This signal represents the reference/previous video frame data which is stored in the reference cache of memory 704. To ensure that the prediction process in MPEG-4 video encoder 613 bases its prediction on a signal that is substantially similar to that available in MPEG-4 video decoder 612 (i.e., a remote video decoder), a local decoder is included in video encoder 613 to generate a locally decoded signal in the encoder. The local decoder, which consists of inverse quantizer 706, IDCT 707, and MC+ 708, basically undoes the encoding stages of quantizer 703 and DCT 702 to produce a decoded prediction error and adds it back into a suitably delayed version of the prediction (reference frame) data to produce a locally decoded (reconstructed) signal with motion compensation. The delayed prediction data is provided by cache 709.
The reconstructed signal is sent to the reconstruction cache of memory 704 for storage. For each macroblock in the current video frame, motion estimator 710, which implements the present invention, searches for a “good” matched macroblock in the reference video frame based on a minimum SAD value. Motion estimator 710 receives as input blocks of current frame and reference frame data. Motion estimator 710 also receives a signal indicating the frame type from rate control module 711. Motion estimator 710 also determines the motion vector. Motion estimator 710 further determines whether a macroblock in the current video frame is intra (encoded independently) or inter (encoded after motion compensation). These determinations are communicated to rate control module 711. The motion vector determinations are communicated to cache 709 and MC+ 708.
The rate of the bitstream output by VLC module 704 fluctuates over time depending on the content of the video data (i.e., changing scenes and objects). This variable bit rate is undesirable because the primary object of MPEG coding is to generate a constant bit rate to fit the available channel or in the case of statistical multiplexing to share a constant bit rate between several video signals. It is then important to ensure that the average bit-rate of the buffer input is the same as that of the channel and neither buffer overflows or underflows. Rate control module 711 is used to control the average bit rate at the bitstream buffer in memory 704 to stay inside an acceptable limit range to prevent overflow and underflow. To achieve the average bit rate control, rate control module 711 varies the quantization factors in quantizer 703 and AC/DC module 712. While coarser scale generates a lower average bit rate, at the expense of picture quality, a finer scale produces better pictures but at a higher average bit rate. As the buffer fills, quantizer 703 and AC/DC module 712 get coarser, which tends to reduce the average bit rate, helping the buffer to empty. Additionally, rate control module 711 takes into consideration the expected differences (e.g., through modeling projection) in bit rates generated by I and P frames.
Referring to
Macroblock address calculator 803 receives a signal from scheduler 804 and search pixel addresses from the search cache. Calculator 803 computes the address of the search (current) block (macroblocks or 8×8 blocks) which contains the search pixel addresses based on the signal from scheduler 804 and provides the macroblock addresses to motion vector calculator 801 for determining the motion vectors of the corresponding blocks. As an example, the signal from scheduler 804 may involve information related to checking points that allows macroblock address calculator 803 to determine which macroblock address to compute.
In the preferred embodiment, SRAM 802 stores the motion vector for the 8×8 pixels block related to the LEFT macroblock and/or the motion vector for the 8×8 pixels block related to the UP macroblocks relative to the current macroblock in the current video frame. SRAM 802 is updated after each current macroblock is finished. The motion vectors for the UP and LEFT macroblocks are used as first and second predicted start vectors, respectively, in accordance with the preferred embodiment of the present invention. Whether one or two predicted start vectors are provided by SRAM 802 depends on a signal provided by programmed registers 812 which is controlled by the CPU. SRAM 802 provides either the UP or LEFT motion vector as the predicted start vector to scheduler 804 and diamond search controller 808. Using the predicted start vector from SRAM 802, current minimum SAD from compare & motion vector generation module 811, the associated motion vectors from motion vector calculator 801, together with reference and current (search) pixel information (e.g., intensity level for a pixel), which are stored in diamond SAD array 807 in a format recognizable by diamond search controller 808, diamond search controller 808 carries out the steps of a large diamond search and a small diamond search that were discussed earlier. Whether a large diamond search or small diamond search is performed depends on the instruction given by scheduler 804. Scheduler 804 also provides diamond search controller with the offset step (e.g., four, one, or one-half pixel), the checking points, and constant FAVOR_0. The minimum SAD value Minsad computed from the large diamond search or small diamond search by diamond search controller 808 is passed to compare & motion vector generation module 811 which monitors and stores the minimum SAD during the entire motion estimation process as well as monitors and stores the motion vector associated with the block (either macroblock or 8×8 block) in which the minimum SAD locates. Compare & motion vector generation module 811 outputs the updated minimum SAD value and the associated motion vector. For the small diamond search, diamond search controller 808 further receives the updated minimum SAD value and the associated motion vector resulted from the large diamond search from compare & motion vector generation module 811. Diamond search controller 808 also receives a signal indicating whether a second search (see step 220 of
Using reference and current (search) pixel information (e.g., intensity level for a pixel), which are stored in 8×8 SAD array 809 in a format recognizable by 8×8 search controller 810, and the updated minimum SAD value together with the associated motion vector resulted from the small diamond search from compare and motion vector generation module 811, 8×8 search controller 810 carries out the steps of a 8×8 search that was discussed earlier. Scheduler 804 initiates the 8×8 search process carried out by 8×8 search controller 810. Scheduler 804 also provides 8×8 search controller 810 with the offset step, the value of FAVOR13 16×16, and the checking points. The minimum SAD value Minsad computed from each 8×8 search by controller 810 is used to compute ΣSAD(8×8) which is then compared against [SAD(16×16)−FAVOR—16×16]. Depending on the result of the comparison, either the SAD and the motion vector of one macroblock or four SADs and motion vectors of four 8×8 blocks are provided to compare & motion vector generation module 811. 8×8 search controller 810 also provides signal mv4flag to scheduler 804 indicating whether four 8×8 block motion vectors or one macroblock motion vector are used.
Diamond search controller 808 also performs the half-pixel search discussed earlier in step 232 using inputs such as the four minimum SAD values and the associated motion vectors for the four 8×8 pixels blocks (or the minimum SAD value and the associated motion vectors for the macroblock) from compare & motion vector generation module 811, the offset step and the checking points from scheduler 804, and reference and current (search) pixel information (e.g., intensity level for a pixel), which are stored in diamond SAD array 807. Scheduler 804 initiates the half-pixel search carried out by diamond search controller 808 and indicates whether such half-pixel search involves one motion vector for a macroblock or four motion vectors for four 8×8 blocks. The minimum SAD value Minsad computed from the diamond search controller 808 is passed to compare & motion vector generation module 811 which monitors and stores the minimum SAD during the entire motion estimation process as well as monitors and stores motion vectors (e.g., associated with the block, which may be either macroblock or 8×8 pixels, in which the minimum SAD locates, checking points etc.). Compare & motion vector generation module 811 outputs the minimum SAD value and the associated motion vector.
From the discussion above, scheduler 804 monitors and controls the motion estimation process in accordance with the present invention. Hence, scheduler 804 is a state machine that can be configured to carry out the steps of either
Standard deviation calculator 806 receives pixel information (e.g., pixel coordinates, intensity, etc.) from the search cache storing the current video frame from scheduler 804. When the motion estimation process is complete, standard deviation calculator 806 computes the Mean And Standard Deviation (MAD) according to:
where IV(j,k) and IV(p,q) are the intensity level of the pixels inside a macroblock and N is 16 (the number of rows and the number of columns in macroblock).
This MAD value is provided as an input to Intra/inter decision module 805 which additionally receives the final minimum SAD value (resulted from the motion compensated estimation process) from compare & motion vector generation block 811 via scheduler 804, intra/inter decision block 805 determines whether the macroblock is I-encoded or P-encoded (i.e., I coding mode or P coding mode) and outputs intra/inter signal to so indicate.
Preferably, intra/inter decision module 805 indicates that a macroblock is I-encoded if the MAD value is less than or equal to the minimum SAD value. If the macroblock is I-encoded, the motion vector is set to zero and the minimum SAD value of the motion compensated estimation process is set to the MAD value. Otherwise, intra/inter decision module 805 indicates that the macroblock is P-encoded if the MAD value is greater than the minimum SAD value. If the macroblock is P-encoded, the minimum SAD value and its associated motion vector currently stored in compare and motion vector generation module 811 are output.
An embodiment of the present invention, a system, apparatus, and method to perform motion estimation using two predicted start points is presented. While the present invention has been described in particular embodiments, the present invention should not be construed as limited by such embodiments, but rather construed according to the below claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3679821 | Schroeder | Jul 1972 | A |
| 4177514 | Rupp | Dec 1979 | A |
| 4583164 | Tolle | Apr 1986 | A |
| 4591979 | Iwashita | May 1986 | A |
| 4644461 | Jennings | Feb 1987 | A |
| 4755810 | Knierim | Jul 1988 | A |
| 4814978 | Dennis | Mar 1989 | A |
| 4992857 | Williams | Feb 1991 | A |
| 5045940 | Peters et al. | Sep 1991 | A |
| 5130797 | Murakami et al. | Jul 1992 | A |
| 5146324 | Miller et al. | Sep 1992 | A |
| 5212742 | Normile et al. | May 1993 | A |
| 5225875 | Shapiro et al. | Jul 1993 | A |
| 5233689 | Rhoden et al. | Aug 1993 | A |
| 5267334 | Normille et al. | Nov 1993 | A |
| 5267344 | Nelson, III | Nov 1993 | A |
| 5369744 | Fukushima et al. | Nov 1994 | A |
| 5371896 | Gove et al. | Dec 1994 | A |
| 5596369 | Chau | Jan 1997 | A |
| 5598514 | Purcell et al. | Jan 1997 | A |
| 5608652 | Astle | Mar 1997 | A |
| 5613146 | Gove et al. | Mar 1997 | A |
| 5623311 | Phillips et al. | Apr 1997 | A |
| 5630033 | Purcell et al. | May 1997 | A |
| 5646692 | Bruls | Jul 1997 | A |
| 5657465 | Davidson et al. | Aug 1997 | A |
| 5768429 | Jabbi et al. | Jun 1998 | A |
| 5790881 | Nguyen | Aug 1998 | A |
| 5809538 | Pollmann et al. | Sep 1998 | A |
| 5821886 | Son | Oct 1998 | A |
| 5845083 | Hamadani et al. | Dec 1998 | A |
| 5870310 | Malladi | Feb 1999 | A |
| 5883823 | Ding | Mar 1999 | A |
| 5889949 | Charles | Mar 1999 | A |
| 5898881 | Miura et al. | Apr 1999 | A |
| 5909224 | Fung | Jun 1999 | A |
| 5923375 | Pau | Jul 1999 | A |
| 5954786 | Volkonsky | Sep 1999 | A |
| 5969728 | Dye et al. | Oct 1999 | A |
| 5999220 | Washino | Dec 1999 | A |
| 6035349 | Ha et al. | Mar 2000 | A |
| 6073185 | Meeker | Jun 2000 | A |
| 6088355 | Mills et al. | Jul 2000 | A |
| 6098174 | Baron et al. | Aug 2000 | A |
| 6104470 | Streefkerk et al. | Aug 2000 | A |
| 6144362 | Kawai | Nov 2000 | A |
| 6148109 | Boon et al. | Nov 2000 | A |
| 6157751 | Olson et al. | Dec 2000 | A |
| 6175594 | Strasser et al. | Jan 2001 | B1 |
| 6188799 | Tan et al. | Feb 2001 | B1 |
| 6195389 | Rodriguez et al. | Feb 2001 | B1 |
| 6222883 | Murdock et al. | Apr 2001 | B1 |
| 6269174 | Koba et al. | Jul 2001 | B1 |
| 6272281 | De Vos et al. | Aug 2001 | B1 |
| 6305021 | Kim | Oct 2001 | B1 |
| 6311204 | Mills | Oct 2001 | B1 |
| 6317124 | Reynolds | Nov 2001 | B2 |
| 6356945 | Shaw et al. | Mar 2002 | B1 |
| 6360234 | Jain et al. | Mar 2002 | B2 |
| 6418166 | Wu et al. | Jul 2002 | B1 |
| 6459738 | Wu et al. | Oct 2002 | B1 |
| 6539060 | Lee et al. | Mar 2003 | B1 |
| 6539120 | Sita et al. | Mar 2003 | B1 |
| 6560629 | Harris | May 2003 | B1 |
| 6647062 | Mackinnon | Nov 2003 | B2 |
| 6665346 | Lee et al. | Dec 2003 | B1 |
| 6687788 | Vorbach et al. | Feb 2004 | B2 |
| 6690835 | Brockmeyer et al. | Feb 2004 | B1 |
| 6690836 | Natarajan et al. | Feb 2004 | B2 |
| 6721830 | Vorbach et al. | Apr 2004 | B2 |
| 6751721 | Webb, Jr. et al. | Jun 2004 | B1 |
| 6760478 | Adiletta et al. | Jul 2004 | B1 |
| 6782052 | Sun et al. | Aug 2004 | B2 |
| 6799192 | Handley | Sep 2004 | B1 |
| 6807317 | Mathew et al. | Oct 2004 | B2 |
| 6823443 | Horiyama et al. | Nov 2004 | B2 |
| 6950473 | Kim et al. | Sep 2005 | B2 |
| 6996645 | Wiedenman et al. | Feb 2006 | B1 |
| 7038687 | Booth, Jr. et al. | May 2006 | B2 |
| 7173631 | Anderson | Feb 2007 | B2 |
| 7260148 | Sohm | Aug 2007 | B2 |
| 7277101 | Zeng | Oct 2007 | B2 |
| 7289672 | Sun et al. | Oct 2007 | B2 |
| 7379501 | Lainema | May 2008 | B2 |
| 7403564 | Laksono | Jul 2008 | B2 |
| 7450640 | Kim et al. | Nov 2008 | B2 |
| 7499491 | Lee et al. | Mar 2009 | B2 |
| 7548586 | Mimar | Jun 2009 | B1 |
| 7548596 | Yen et al. | Jun 2009 | B2 |
| 7551671 | Tyldesley et al. | Jun 2009 | B2 |
| 7565077 | Rai et al. | Jul 2009 | B2 |
| 7581182 | Herz | Aug 2009 | B1 |
| 7630097 | Kodama et al. | Dec 2009 | B2 |
| 7689000 | Kazama | Mar 2010 | B2 |
| 7693219 | Yan | Apr 2010 | B2 |
| 7720311 | Sriram | May 2010 | B1 |
| 7721069 | Ramchandran et al. | May 2010 | B2 |
| 7792194 | Zhong et al. | Sep 2010 | B2 |
| 7924923 | Lee et al. | Apr 2011 | B2 |
| 8009923 | Li et al. | Aug 2011 | B2 |
| 8369402 | Kobayashi et al. | Feb 2013 | B2 |
| 20010020941 | Reynolds | Sep 2001 | A1 |
| 20010024448 | Takase et al. | Sep 2001 | A1 |
| 20010028353 | Cheng | Oct 2001 | A1 |
| 20010028354 | Cheng et al. | Oct 2001 | A1 |
| 20020015445 | Hashimoto | Feb 2002 | A1 |
| 20020015513 | Ando et al. | Feb 2002 | A1 |
| 20020025001 | Ismaeil et al. | Feb 2002 | A1 |
| 20020041626 | Yoshioka et al. | Apr 2002 | A1 |
| 20020109790 | Mackinnon | Aug 2002 | A1 |
| 20020114394 | Ma | Aug 2002 | A1 |
| 20020118743 | Jiang | Aug 2002 | A1 |
| 20030020835 | Petrescu | Jan 2003 | A1 |
| 20030048361 | Safai | Mar 2003 | A1 |
| 20030078952 | Kim et al. | Apr 2003 | A1 |
| 20030141434 | Ishikawa et al. | Jul 2003 | A1 |
| 20030161400 | Dinerstein et al. | Aug 2003 | A1 |
| 20040057523 | Koto et al. | Mar 2004 | A1 |
| 20040095998 | Luo et al. | May 2004 | A1 |
| 20040100466 | Deering | May 2004 | A1 |
| 20040150841 | Lieberman et al. | Aug 2004 | A1 |
| 20040156435 | Itoh et al. | Aug 2004 | A1 |
| 20040174998 | Youatt et al. | Sep 2004 | A1 |
| 20040181564 | MacInnis et al. | Sep 2004 | A1 |
| 20040181800 | Rakib et al. | Sep 2004 | A1 |
| 20040190613 | Zhu et al. | Sep 2004 | A1 |
| 20040190617 | Shen et al. | Sep 2004 | A1 |
| 20040202245 | Murakami et al. | Oct 2004 | A1 |
| 20040213348 | Kim et al. | Oct 2004 | A1 |
| 20040218626 | Tyldesley et al. | Nov 2004 | A1 |
| 20040218675 | Kim et al. | Nov 2004 | A1 |
| 20040228415 | Wang | Nov 2004 | A1 |
| 20040257434 | Davis et al. | Dec 2004 | A1 |
| 20050008254 | Ouchi et al. | Jan 2005 | A1 |
| 20050033788 | Handley | Feb 2005 | A1 |
| 20050047502 | McGowan | Mar 2005 | A1 |
| 20050066205 | Holmer | Mar 2005 | A1 |
| 20050079914 | Kaido et al. | Apr 2005 | A1 |
| 20050105618 | Booth et al. | May 2005 | A1 |
| 20050123040 | Bjontegard | Jun 2005 | A1 |
| 20050190976 | Todoroki et al. | Sep 2005 | A1 |
| 20050238102 | Lee et al. | Oct 2005 | A1 |
| 20050238103 | Subramaniyan et al. | Oct 2005 | A1 |
| 20050265447 | Park | Dec 2005 | A1 |
| 20050265454 | Muthukrishnan et al. | Dec 2005 | A1 |
| 20050276493 | Xin et al. | Dec 2005 | A1 |
| 20050281337 | Kobayashi et al. | Dec 2005 | A1 |
| 20050286630 | Tong et al. | Dec 2005 | A1 |
| 20060002466 | Park | Jan 2006 | A1 |
| 20060017802 | Yoo et al. | Jan 2006 | A1 |
| 20060056513 | Shen et al. | Mar 2006 | A1 |
| 20060056708 | Shen et al. | Mar 2006 | A1 |
| 20060109910 | Nagarajan | May 2006 | A1 |
| 20060115001 | Wang et al. | Jun 2006 | A1 |
| 20060133501 | Lee et al. | Jun 2006 | A1 |
| 20060133506 | Dang | Jun 2006 | A1 |
| 20060176299 | Subbalakshmi et al. | Aug 2006 | A1 |
| 20060176962 | Arimura et al. | Aug 2006 | A1 |
| 20060203916 | Chandramouly et al. | Sep 2006 | A1 |
| 20070002945 | Kim | Jan 2007 | A1 |
| 20070002950 | Yang | Jan 2007 | A1 |
| 20070036215 | Pan et al. | Feb 2007 | A1 |
| 20070070080 | Graham et al. | Mar 2007 | A1 |
| 20070133689 | Park et al. | Jun 2007 | A1 |
| 20070171981 | Qi | Jul 2007 | A1 |
| 20070217506 | Yang et al. | Sep 2007 | A1 |
| 20070230564 | Chen et al. | Oct 2007 | A1 |
| 20070274389 | Kim et al. | Nov 2007 | A1 |
| 20070286284 | Ito et al. | Dec 2007 | A1 |
| 20070286508 | Le Leannec et al. | Dec 2007 | A1 |
| 20080069203 | Karczewicz et al. | Mar 2008 | A1 |
| 20080117214 | Perani et al. | May 2008 | A1 |
| 20080137726 | Chatterjee et al. | Jun 2008 | A1 |
| 20080151997 | Oguz | Jun 2008 | A1 |
| 20080285444 | Diab et al. | Nov 2008 | A1 |
| 20090060277 | Zhang et al. | Mar 2009 | A1 |
| 20090086827 | Wu et al. | Apr 2009 | A1 |
| 20090116549 | Shen et al. | May 2009 | A1 |
| 20090122864 | Palfner et al. | May 2009 | A1 |
| 20090161763 | Rossignol et al. | Jun 2009 | A1 |
| 20090196350 | Xiong | Aug 2009 | A1 |
| 20090268974 | Takagi | Oct 2009 | A1 |
| 20100034268 | Kusakabe et al. | Feb 2010 | A1 |
| Number | Date | Country |
|---|---|---|
| 1489391 | Apr 2004 | CN |
| 1283640 | Feb 2003 | EP |
| 1283640 | Dec 2003 | EP |
| 2348559 | Apr 2000 | GB |
| 2348559 | Nov 2000 | GB |
| 04162893 | Jun 1992 | JP |
| 11096138 | Apr 1999 | JP |
| 2001184323 | Jul 2001 | JP |
| 2005354686 | Dec 2005 | JP |
| 2006287315 | Oct 2006 | JP |
| 9827742 | Jun 1998 | WO |
| 0233650 | Apr 2002 | WO |
| 2005001625 | Jan 2005 | WO |
| 2005001625 | Jan 2005 | WO |
| 2005096168 | Oct 2005 | WO |
| 2006085137 | Aug 2006 | WO |
| Entry |
|---|
| Tourapis et al., Proc. Of SPIE Conf. Vis. Comm. and Img. Procssng, vol. 3, , pp. 1365-1373, Jun. '00. |
| Zheng et al., Inter. Conf. Info. Systms, Analysis and Synthesis, SCI 2001/ISAS 2001, vol. 13, 2001. |
| Jong, et al., “Accuracy Improvement and Cost Reduction of 3-Step Search Block Matching Algorithm for Video Coding”, Feb. 1, 1994, IEEE Transaction on Circuits and Systems for Video Technology, vol. 4 No. 1, pp. 88-90, XP000439487. |
| Tourapis et al., Proc. of SPIE Conf. Vis. Comm. And Img. Processing, vol. 3, pp. 1365-1373, Jun. '00. |
| Zheng, et al., Inter. Conf. Info. Systems, Analysis and Synthesis, SCI 2001-ISAS 2001, vol. 13, 2001. |
| Realization of Low-Bit—Ratio Video Encoder Using Mpact Media Processor; IWASAKI, Junichi et al.; 1997. |
| Rohini Krishnan, et al., “Design of a 2D DCT/IDCT Application Specific VLIW Processor Supporting Scaled and Sub-sampled Blocks,” 16th International Conference on VLSI Design, 2003, six pages. |
| Shite-Hao Wang et al.; “A platform-based MPEG-4 advanced video coding (AVC) decoder with block level pipelining,” Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia Proceedings of the 2003 Joint Conference of the Fourth International Conference on, vol. 1, No., p. 51-55 vol. 1, Dec. 2003. |
| Tu, C., Liang, J., and Tran, T. “Adaptive Runlength Coding”, in—IEEE Signal Processing Letters—vol. 10, No. 3, pp. 61-6. Mar. 2003. |
| Tung-Chien Chen; Yu-Wen Huang; Liang-Gee Chen, “Analysis and design of macroblock pipelining for H.264/AVC VLSI architecture,” Circuits and Systems, 2004. ISCAS '04. Proceedings of the 2004 International Symposium on, vol. 2, No., pp. 11-273-6 vol. 2, 23-26 May 2004. |
| “Ralf Schafer et al.,”, “H.264/AVC”, Dated: Jan. 2003, pp. 1-12. |
| —The Merriam-Webster Dictionary—. 2005 ed. Springfield, MA: Merriam-Webster Inc., 2005. |
| 3D Graphics Processor for Mobile Set Based on Configurable Processor; TAKEMOTO, Takashi et al. |
| A Single-Chip Video/Audio Codec for Low Bit Rate Application Seongmo Park, Seongmin Kim, Igkyun Kim, Kyungjin Byun, Jin Jong Cha, and Hanjin Cho, ETRI Journal, vol. 22, No. 1, Mar. 2000, pp. 20-29. |
| Advanced Video Coding for generic audiovisual services, Telecommunication Standardization Sector of ITU, Dated Mar. 2005, pp. 1-343. |
| Andrew Gibson, “H.264 Video Encoding Standard”, year Aug. 2002, pp. 1-98, Queens University Kingston Ontario Canada. |
| Chen, Hao et al., “Adaptive FMO Selection Strategy for Error Resilient H.264 Coding” International Conference on Audio, Language and Image Processing, 2008. ICALIP 2008, Jul. 7-9, 2008, pp. 868-872. |
| Hannuksela, Miska et al., “Random Access Using Isolated Regions”, IEEE 2003, pp. III-841 to III to 844. |
| Iwasaki, I.; Naganuma, J.; Nitta, K.; Yoshitome, T.; Ogura, M.; Nakajima, Y.; Tashiro, Y.; Onishi, T.; Ikeda, M.; Endo, M., “Single-chip MPEG-2 422P@HL CODEC LSI with multi-chip configuration for large scale processing beyond HDTV level,” Design, Automation and Test in Europe Conference and Exhibition, Mar. 2003. |
| Jamil-Ur-Rehman and Dr. Zhang Ye, “Efficient Techniques for Signaling Intra Prediction modes of H.264/Mpeg-4 Part 10”, Proceedings of the First International Conference on Innovative Computing, Information and Control, ICICIC, Year 2006, pp. 1-4. |
| Mizuno, M. et al.; “A 1.5-W single-chip MPEG-2 MP@ML video encoder with low power motion estimation and clocking,” Solid-State Circuits, IEEE Journal of, vol. 32, No. 11, pp. 18-7-1816, Nov 1997. |
| Kadono Shinya, et. Al. Revised edition H. 264/AVC Textbook, Impress R&D, Jan. 1, 2006, pp. 131-133. |
| Number | Date | Country | |
|---|---|---|---|
| 20040247029 A1 | Dec 2004 | US |
