Patent Grant
7450184
The following co-pending and co-assigned applications contain related information and are hereby incorporated by reference:
The present invention relates in general to video processing techniques, and in particular, to circuitry and methods for detecting 2:2 encoded video and systems utilizing the same.
Two primary video format standards are utilized worldwide to record, transmit, and display composite video data, namely, the National Television Systems Committee (NTSC) and the Phase Alternating Line (PAL) standards. Both the NTSC and PAL standards define interlaced video systems in which one frame of display pixels is partitioned into alternating interlaced fields, with each interlaced field updated at twice the update rate of the frame. Additionally, many digital versatile disk (DVD) players, DVD player-recorders, and similar video recording and playback systems, output data in an interlaced format, depending on the format used to broadcast or record the given playback media.
On the other hand, many state of the art display systems, such as high definition television (HDTV) sets, generate displays using a progressive scan format. In the progressive scan format, video data are transmitted and displayed in frames, which are not partitioned into fields. In other words, each display frame is generated by sequentially scanning through all the lines of each frame at the original field update rate. Hence, in order to interface an interlaced video source, such as television cable interface or a DVD player playing-back an interlaced format DVD, with a progressive-scan display system, such as a HDTV set, deinterlacing must be performed.
Furthermore, films are typically produced with a frame rate of twenty four (24) frames per second, while in the PAL video system, frames are generated at a rate of twenty five (25) frames per second from interlaced fields at a rate of fifty (50) fields per second. During conversion of film to interlaced PAL video, the 2:2 pull-down technique is typically utilized to convert the film frames into interlaced fields, which are then used to reconstruct frames on the PAL display, in either an interlaced or progressive scan format. In particular, the frame rate of twenty four (24) frames per second is increased by four percent (4%) to generate frames at a rate of twenty five (25) frames per second. The 2:2 pull-down technique then converts the twenty five (25) frames per second data into fifty (50) fields per second data. Additional applications of the 2:2 pull-down technique include the conversion of computer generated images into interlaced video, and the conversion of thirty (30) frames per second film into sixty (60) fields per second interlaced video. For discussion purposes, a PAL system will be used as an example.
The 2:2 pull-down technique replicates the content of each film frame twice to generate identical odd and even interlaced video fields. In an interlaced playback system, these interlaced fields are each displayed at the PAL field rate of fifty (50) fields per second to generate full video frames at the PAL frame rate of twenty five (25) frames per second. The PAL frame rate is therefore slightly higher than the original film rate of twenty four (24) frames per second. For progressive scan systems, corresponding pairs of odd and even fields are weaved together to generate frames at the PAL interlaced frame rate of twenty-five (25) frames per second, and then each frame is repeated twice to generate a stream of frames at the progressive scan frame rate of fifty (50) frames per second.
In order to correctly generate either interlaced or progressive scan frames from interlaced PAL fields that were originally derived from film, the correct pairs of odd and even fields must be weaved together. Hence, some form of 2:2 mode detection must be performed to detect when a sequence of fields was originally generated from film with the 2:2 pull-down technique and subsequently identify the correct pair of fields that will be weaved together to compose each display frame.
Given increasing popularity of progressive scan display systems, as well as the need to maintain compatibility with systems generating interlaced display data, new 2:2 detection techniques are required. In particular, these deinterlacing techniques should allow for the generating of progressive scan displays from interlaced data created by 2:2 pull-down.
The principles of the present invention are embodied in systems and methods for detecting video data generated using pull-down techniques. In one particular embodiment, a method is disclosed for detecting a stream of video data generated utilizing a pull-down technique includes receiving a sequence of fields of interlaced video data. For each pair of a plurality of pairs of the fields of interlaced video in the sequence, pixel values corresponding to pixel positions of pixel lines of a first parity of a first field of the pair of fields are weaved with pixel values corresponding to pixel positions of pixel lines of a second parity of a second field of the pair of fields. For each pixel value corresponding to each pixel position of each pixel line of the second field of the pair of fields after weaving, a check is made for feathering at the corresponding pixel position due to motion. The pixel positions with feathering due to motion in the second field of the pair of fields are counted, and, for the plurality of pairs of fields in the sequence, a check is made for a pattern in the number of counted pixel positions with feathering due to motion in the second field of each pair of fields to determine if the video data comprises data generated utilizing a pull-down technique.
Embodiments of the present principles are particularly advantageous when applied to systems and methods for detecting a stream of video data which has been generated by a 2:2 pull-down process. Additionally, when a stream video data fields generated by 2:2 pull-down is detected, these principles allow for pairs fields to be properly selected for generating progressive scan video frames by deinterlacing. Furthermore, vertical spatial high frequency data are preserved during deinterlacing of 2:2 pull-down generated fields.
For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
The principles of the present invention and their advantages are best understood by referring to the illustrated embodiment depicted in
In video encoder 100, the composite analog input video is converted into composite digital video in the YC format by analog to digital converters (ADCs) 101. The digitized YC video data are then passed through automatic gain control (AGC) and filter block 102. A sync detector 103 detects the vertical synchronization (
Y/C separator block 104 next separates the digital Y and C components of the digitized composite video data stream. The C component is demodulated into U and V color components by color subcarrier recovery block 105 and chroma demodulation block 106, described in further detail below. The Y, U, and V components are passed directly to video processor 107 for further processing to generate the Y, Cr and Cb components of the output YCrCb digital component video signal.
In the illustrated embodiment of audio-visual recording and playback system, deinterlacing circuitry within video processor 107 performs interlaced video to progressive scan video conversion (deinterlacing). Additionally, video processor 107 performs 2:2 stream detection and allocates 2:2 generated interlaced fields into the proper interlaced or deinterlaced display frames as discussed further below. In alternate embodiments, a dedicated post-processor or similar subsystem performs deinterlacing and 2:2 detection if the input stream is already digitized.
In the example shown in
As shown in
For discussion purposes, procedure 200 will be described by assuming current interlaced field FIN is the reference field such that pixel lines Y and Y+2 of interlaced field FIN provide lines Y and Y+2 of progressive scan frame FPN. In this example, line Y+1, of progressive scan frame FPN, and in particular pixel value PPXY+1 corresponding to pixel position PPXY+1, is being tested during the current iteration of procedure 200.
Procedure 200 begins at block 201, and at block 201a, the reference interlaced field is set as FIN, the previous interlaced field becomes field FIN−1, and the next interlaced field becomes field FIN+1. Thus, for example if reference interlaced field FIN has even parity (i.e. is composed even numbered pixel lines), then preceding interlaced field FIN−1 and following interlaced field FIN+1 both have odd parity (i.e. are composed of odd numbered pixel lines). On the other hand, if reference field FIN has odd parity, then preceding field FIN−1 and following field FIN+1 are both of even parity.
A motion map MMN is generated at block 202 by comparing at least one of the chrominance and luminance data of each pixel value of each pixel position of preceding field FIN−1 with the luminance and chrominance data of the corresponding pixel value of each pixel position of following field FIN+1. Motion map MMN is preferably a one (1) bit per pixel map, with each bit mapping to a given pixel position set to a logic one (1) to indicate a change in corresponding pixel value, having a magnitude greater than a selected threshold, between fields FIN−1 and FIN+1 (i.e. motion detected). On the other hand, if no change, having a magnitude greater than the selected threshold, in either luminance or chrominance, is detected for the given pixel value between fields FIN−1 and FIN+1, then the motion map bit for the corresponding pixel position is set to a logic zero (0) (i.e. no motion).
As discussed further below, motion maps generated at block 202 have lines of motion mapping bits of either an odd or even parity, depending on the parity of the lines of preceding field FIN−1 and following field FIN+1 being compared by parity check. In particular, if interlaced fields FIN−1 are FIN+1 are odd, the lines of motion mapping of motion map MMN represent odd pixel lines, and if the lines of interlaced fields FIN−1 are FIN+1 are even, the lines of motion mapping bits of motion map MMN represent even pixel lines.
At block 203, pixel value PPXY+1 corresponding to pixel position PPXY+1 of following field FIN+1 is weaved with pixel values PPXY and PPXY+2 corresponding to pixel positions PPXY and PPXY+2 of reference field FIN. After weaving is performed at block 203, a check is made at block 204 for feathering at pixel position PPXY+1. Feathering is checked by comparing the characteristics of pixel value PPXY+1 with the characteristics of pixel values PPXY and PPXY+2 after weaving. Specifically, the luminance and chrominance data of pixel value PPXY+1 are compared against the luminance and chrominance data of pixel values PPXY and PPXY+2, as well as a threshold value. A check for feathering is required because the single motion map parity check at block 203 can miss motion which occurs during the time between the generation of preceding field FIN−1 and following field FIN+1. On the other hand, feathering could also be a representation of vertical spatial high frequency. Advantageously, the principles of the present invention allows for an accurate determination if any detected feathering represents actual motion, due to field splitting of the original progressive frame, or vertical spatial high frequency.
At decision block 205, feathering is detected when the magnitude of any change, in at least one of the luminance or chrominance values, between pixel value PPXY+1 and both pixel values PPXY and PPXY+2 exceeds a selected threshold, and the direction of change between pixel values PPXY and pixel PPXY+1 and the direction of change between pixel values PPXY+1 and PPXY+2 is the same. Otherwise, if any of these three conditions is not met, an absence of feathering is detected at pixel position PPXY+1.
If no feathering is detected at pixel position PPXY+1, then at block 206, the pixel index X and/or the line index Y is incremented and procedure 200 returns to block 203 for the testing of the next pixel in current (reference) field FIN.
In contrast, if feathering is detected at pixel position PPXY+1 at decision block 205, then historical motion detection must be performed to determine if the feathering is the result of actual motion or the result of vertical spatial high frequency. Vertical spatial high frequency occurs when pixel values rapidly change between vertical pixel positions because of sharp spatial changes in the display objects being generated, such as at the boundaries checker-board patterns or with fine resolution pixel patterns, for example fine vertical display object lines.
To detect historical motion at pixel position PPXY+1, motion maps MMN−Z to MMN, including current motion map MMN generated at block 202 and the motion maps generated during the testing of Z−1 number of previous fields FIN−Z to FIN−1, are combined into single motion map. Specifically, a determination is made at decision block 207 as to the parity of the line containing current pixel position PPXY+1. If pixel position PPXY+1 has odd parity (i.e. is part of an odd line Y+1), then the odd map lines of even-line motion maps MMN−Z to MMN−1 are generated by interpolation of the existing even motion map lines at block 208. If pixel PPXY+1 has even parity (i.e. is part of an even line Y+1), then the even map lines of odd-line motion maps MMN−Z to MMN−1 are generated by interpolation of the existing odd motion map lines at block 209.
The interpolation operations performed at blocks 208 and 209 ensure that all Z+1 number of motion maps MMN−Z to MMN have map lines of the same parity as the line including pixel position PPXY+1 and therefore can be appropriately combined at block 210, such that a detection of historical motion over Z+1 number of interlaced fields can be made. Combination of memory maps MMN−Z−1 to MMN, is preferably accomplished by performing a logical-OR operation on a mapping-bit by mapping-bit basis. Consequently, at decision block 211, a determination can be made as to whether, over a history of the generation of Z+1 number of fields FIN−Z to FIN, motion has occurred at pixel position PPXY+1.
If historical motion has not occurred at decision block 211, then at block 212, pixel value PPXY+1 corresponding to pixel position PPXY+1 is considered static with feathering (i.e. represents vertical spatial high frequency). In this case, at block 216, the pixel index X and/or the line index Y is incremented, and procedure 200 loops-back to block 203 to test the next pixel position in reference field FIN.
If instead motion is detected at decision block 211, then at block 213, pixel PPXY+1 value corresponding to pixel position PPXY+1 is considered as a feathered pixel with motion due to feathering and a feathered pixel count increments at block 214. Procedure 200 then loops, at decision block 215 and index incrementation block 216, until all the pixel positions in reference field FIN have been tested.
Once all the pixel positions in reference field FIN have been tested at block 215, the number of feathered pixels is compared against a threshold at block 217. If at decision block 218 the feathered pixel count for field FIN is below the threshold, then at block 219, the feathered pixel count for reference field FIN is logged as being low (i.e. F_LOW). While F_LOW ideally is zero (0), in actual implementations, this count will normally be a small positive number. Otherwise, if the feathered pixel count is above the threshold, at block 218, then the feathered pixel count is logged as high (i.e. F_HIGH) at block 220.
After observing a sufficiently long sequence of fields at decision block 221, a determination is made at decision block 222 as to whether the current sequence of fields were generated using 2:2 pull-down. Ideally, since each original frame of film is converted into a pair of odd and even fields with the same information content, a pattern of feathered pixel counts logged at blocks 219 and 220 with each iteration of procedure 200 should be:
On the other hand, if 2:2 pull-down frames are detected at decision block 222, then the principles of the present invention allow for the allocating of the interlaced fields into the proper frames. Specifically, at block 224 a check is made to determine if the feathered pixel count for reference field FIN has the value F_LOW or F_HIGH. If the number of feathered pixels is F_LOW, reference field FIN and following field FIN+1 belong in the same output frame. Otherwise, it the feathered pixels count is F_HIGH, then reference field FIN and following field FIN+1 belong in different progressive scan frames. A typical progressive scan output generated from 2:2 pull-down generated interlaced fields follows the sequence:
Advantageously, the principles of the present detection provide for accurate 2:2 detection, and subsequent identification of field pairs, even in view of interlaced fields with spatial high frequency.
Although the invention has been described with reference to specific embodiments, these descriptions are not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. It should be appreciated by those skilled in the art that the conception and the specific embodiment disclosed might be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.
It is therefore contemplated that the claims will cover any such modifications or embodiments that fall within the true scope of the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5339109 | Hong | Aug 1994 | A |
| 5467439 | Lee et al. | Nov 1995 | A |
| 5519451 | Clatanoff et al. | May 1996 | A |
| 5532751 | Lui | Jul 1996 | A |
| 5602654 | Patti et al. | Feb 1997 | A |
| 5708474 | Hong | Jan 1998 | A |
| 5832143 | Suga | Nov 1998 | A |
| 6133957 | Campbell | Oct 2000 | A |
| 6295091 | Huang | Sep 2001 | B1 |
| 6400762 | Takeshima | Jun 2002 | B2 |
| 6459455 | Jiang et al. | Oct 2002 | B1 |
| 6577435 | Bang et al. | Jun 2003 | B1 |
| 6614484 | Lim et al. | Sep 2003 | B1 |
| 6757022 | Wredenhagen | Jun 2004 | B2 |
| 6795123 | Gotanda et al. | Sep 2004 | B2 |
| 7206028 | Yeh et al. | Apr 2007 | B2 |
| 7286185 | Wong et al. | Oct 2007 | B2 |
| 20050168655 | Wyman et al. | Aug 2005 | A1 |
