The present invention relates to wireless communications, and in particular, to transmission of uncompressed video over wireless communication links.
With the proliferation of high quality video, an increasing number of electronic devices (e.g., consumer electronic devices) utilize high-definition (HD) video. Conventionally, most devices compress the HD video, which can be over 1 Gbps (gigabits per second) in bandwidth, to a fraction of its size to allow for transmission between devices. However, with each compression and subsequent decompression of the video, some video information can be lost and the picture quality is degraded.
The High-Definition Multimedia Interface (HDMI) specification defines an interface for uncompressed HD transmission between devices through the HDMI cables (wired links). Three separate channels are used to transmit three pixel component streams (R, B, G or Y, Cb, Cr). For each channel, pixels are transmitted in a pixel-by-pixel order for each video line, and line-by-line for each video frame or field. The HDMI provides pixel-repetition functionality which repeats each pixel one or multiple times. The copies of each pixel directly follow the original pixel during the transmission at each pixel component channel.
Existing Wireless Local Area Networks (WLANs) and similar technologies do not have the bandwidth needed to carry uncompressed HD video, such as providing an air interface to transmit uncompressed video over a 60 GHz bandwidth. Further, existing networks can suffer from interference issues when several devices are connected, leading to video signal degradation.
The present invention provides a method and a system for bit reorganization and packetization of uncompressed video, such as HD video, for transmission over wireless communication channels. The bit stream of the uncompressed video is reorganized according to importance levels of video bits. Then, various unequal protection methods are applied to bits at different video importance levels.
After the bit reorganization, encoding processes such as run-length coding (RLC) and forward error correction (FEC) are applied to more important bits to provide better protection for transmission, compared to less important bits. This improves the transmission robustness of uncompressed video over wireless channels, with the same data rate using bit reorganization, RLC and FEC. The reorganized and encoded information is then transmitted from a transmitter to a receiver over a wireless channel. The receiver performs the reverse steps of the transmitter, along with error detection/correction as needed, to recover the uncompressed video information.
These and other features, aspects and advantages of the present invention will become understood with reference to the following description, appended claims and accompanying figures.
In the drawings, like references refer to similar elements.
The present invention provides a method and a system for communication of uncompressed video, such as HD video, over a wireless channel. This involves performing bit reorganization and packetization for pixels of uncompressed video frames, for transmission over a wireless channel.
The bits in an uncompressed video bit stream are reorganized according to video importance levels of the bits. Bits at higher video importance levels have more significant contribution in terms of video information than bits at lower video importance levels.
After the bit reorganization, various unequal (different) protection levels are applied to the bits at different video importance levels. Preferably, the bits that provide more significant video information are provided with more protection for transmission. Encoding processes such as RLC and FEC can be applied to more important bits to provide stronger transmission protection compared to less important bits.
The reorganized and encoded bits are transmitted from one wireless communication station (the sender) to another wireless communication station (the receiver) over a wireless channel. The receiver then performs reverse steps of the sender to regenerate the uncompressed video bit stream and the corresponding uncompressed video frames.
Example implementations of the present invention for communication of uncompressed HD video frames are now described. A typical uncompressed HD video frame includes M rows and N lines (columns) of pixels, totaling M×N video pixels. The video pixels can be encoded in either a Red, Green, Blue (RGB) pixel component format, or different Y, Cb, Cr pixel component formats such as 4:4:4 or 4:2:2. For a pixel having T bits, and with a frame updating frequency of f frame/second, the data rate required for transmission of an HD video frame is M×N×T×f bits per second. For example, in the 1080p video format, each frame has 1920 rows and 1080 lines, and each pixel has 24 bits, such that if the frame updating frequency is 60 frames/second, then the data rate is 1920×1080×24×60=2,985,984,000 bps.
For each video pixel, if the RGB (or YCbCr 4:4:4) format is used, usually each R, B, or G (or Y, Cb, Cr) pixel component has T/3 bits. If other YCbCr formats except YCbCr 4:4:4 are used, usually each pixel component has more bits than T/3. For example, in the HDMI specification where T is 24, each R, B, or G (or Y, Cb, Cr in 4:4:4) pixel component has 8 bits per pixel. However, each Y, Cb or Cr pixel component has 12 bits in HDMI YCbCr 4:2:2.
An example bit reorganization process according to the present invention reorganizes and groups together (packetizes) bits of different pixels based on bit importance levels. For example, the MSBs of multiple pixels are grouped together and the LSBs of those pixels are grouped together.
The example bit reorganization and packetization process includes the steps of:
Specifically, the bits from the R pixel component of different pixels that are at the same importance level are placed into the same bit plane sub-stream by the parser 102. As such, the parser 102 reorganizes the R (or Y) pixel component bits of pixels 1, 2, . . . , etc. (
The PMP module 108 then multiplexes the sub-streams 103, 105 and 107, based on a transmission priority for each sub-stream, as described above. As such, the PMP module 108 generates 1, . . . , n transmission streams 109, wherein each transmission stream carries bits of certain priority (i.e., transmission stream 1 carries data bits at Priority 1, . . . , transmission stream n carries data bits at Priority n).
The packet scheduler 110 then places the multiplexed bits in different streams 109 into different transmission packets 111 with different priority levels (i.e., packets P1, . . . , Pn corresponding to priority P1, . . . , priority Pn, respectively). The packets 111 are then transmitted over a wireless channel, from a sender to a receiver.
Upon receiving the packets, the receiver performs de-packetization and assembly of the bits in the received packets by reverse steps of the sender.
Then, a de-multiplexing module 206 de-multiplexes the bits in the n buffers 204 back bit plane grouping. In this example, the de-multiplexing module 206 de-multiplexes the bits in the n buffers 204 back into D bit plane sub-streams for each pixel components. Specifically, in this example, the de-multiplexing module 206 de-multiplexes the bits in the n buffers 204 back into: (1) D bit plane sub-streams forming a bit stream 203 for the pixel component R (or Y), (2) D bit plane sub-streams forming a bit stream 205 for the pixel component B (or Cb), and (3) D bit plane sub-streams forming a bit stream 207 for pixel component G (or Cr).
As such, the stream 203 includes sub-streams 1, . . . , D for pixel component R or Y according to the bit plane of
After the information bits of all of the bit planes for a pixel component stream are received from the sender and processed by the PDDP module 202 and the de-multiplexing module 206, bit reordering modules 208, 210, and 212 perform bit reordering for the streams 203, 205, and 207, respectively. The bit reordering modules 208, 210, and 212, recover the original bit order of each pixel component stream such that different bits belonging to the same pixel are grouped together. Then, the recovered pixel component streams can be further processed (e.g., displayed) as uncompressed video.
According to further features of the present invention, after bit organization, an encoding process is applied to the more important bits (e.g., MSBs) before transmission by the sender, in order to provide stronger protection for more important bits in each pixel component stream. Examples of encoding include RLC and FEC encoding. An example of the bit reorganization and encoding process includes the steps of:
The parsers 102, 104 and 106 operate as described in relation to
The multiplexing and packetizing module 308 then multiplexes the sub-streams 303, 305 and 307 of the different pixel components and places the multiplexed information into n transmission packets 310 with different priority levels (i.e., packets P1, . . . , Pn corresponding to priority P1, . . . , priority Pn, respectively). In this example, the module 308 performs a similar function as a combination of the PMP module 108 and the scheduler module 110 in
Upon receiving the packets, the receiver performs de-packetization and assembly of the bits in the received packets by reverse steps of the sender.
The bits in each packet 401 belong to different pixel component streams, and are processed by a module 402 which performs de-packetizing by packet decomposition, bit assembly by data placement, and de-multiplexing by de-multiplexing (PDP). In this example, the PDP module 402 performs the functions of a combination of the module PDDP module 202, the buffers 204 and the de-multiplexing module 206 in
The 1st to the Kth sub-streams and the decoded (K+1)th to the Dth sub-streams, of each of the streams 403, 405 and 407, form bit streams 409, 411 and 413, respectively. The bit streams 409, 411 and 413, correspond to the pixel components R (or Y), B (or Cb), and G (or Cr), respectively, with bits placed in their proper places in separate pixel component buffers based on bit plane position of the information.
As such, the stream 409 includes sub-streams 1, . . . , D for pixel component R or Y according to the bit plane of
After information bits of all bit planes for a pixel component stream are received from the sender, then bit reordering modules 410, 412, and 414 (which function similar to bit reordering 208, 210, and 212 in
The transmitter 502 further includes: a parsing module 510 implementing the function of the above-mentioned parsers 102, 104, and 106 (
The PHY/MAC layers of the transmitter 502 add PHY and MAC headers to packets and transmit the packets to the receiver 504 over the wireless channel 501. In the wireless receiver 504, the PHY/MAC layers 514 and 516 process the received packets. The receiver 504 further includes a de-packetizing module 521, a de-multiplexing module 519, optionally a decoding module 517, and a bit reorganization (de-parsing) module 518.
The depacketizing module 521 and the de-multiplexing module 519 together implement the function of the PDP module 402 (
Although in relation of
As is known to those skilled in the art, the aforementioned example architectures described above, according to the present invention, can be implemented in many ways, such as program instructions for execution by a processor, as logic circuits, as an ASIC, as firmware, etc.
The present invention has been described in considerable detail with reference to certain preferred versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
This application claims priority from U.S. Provisional Patent Application Ser. No. 60/773,828, filed on Feb. 15, 2006, incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5214656 | Chung et al. | May 1993 | A |
5233348 | Pollmann et al. | Aug 1993 | A |
5680129 | Weinberger et al. | Oct 1997 | A |
5978545 | Kato et al. | Nov 1999 | A |
6108382 | Gringeri et al. | Aug 2000 | A |
6201834 | Zhu | Mar 2001 | B1 |
6519005 | Bakhmutsky et al. | Feb 2003 | B2 |
6532268 | Morisawa et al. | Mar 2003 | B1 |
6539448 | Deng | Mar 2003 | B1 |
6597402 | Butler et al. | Jul 2003 | B1 |
6618397 | Huang | Sep 2003 | B1 |
6747993 | Rinchiuso | Jun 2004 | B2 |
6756921 | Kimura et al. | Jun 2004 | B2 |
6801665 | Atsumi et al. | Oct 2004 | B1 |
6868186 | Sadeh | Mar 2005 | B1 |
6879266 | Dye et al. | Apr 2005 | B1 |
6915016 | Varshney et al. | Jul 2005 | B2 |
6966065 | Kitazato et al. | Nov 2005 | B1 |
7016545 | Schwartz et al. | Mar 2006 | B1 |
7043210 | Zhu et al. | May 2006 | B2 |
7161957 | Wang et al. | Jan 2007 | B2 |
7184588 | Proteau et al. | Feb 2007 | B2 |
7313810 | Bell et al. | Dec 2007 | B1 |
7340764 | Kubota et al. | Mar 2008 | B2 |
7444023 | Mossakowski | Oct 2008 | B2 |
7492789 | Shvodian | Feb 2009 | B2 |
7499565 | Fujii et al. | Mar 2009 | B2 |
7512180 | Florencio | Mar 2009 | B2 |
7752647 | Fung | Jul 2010 | B2 |
7792375 | Mizuno | Sep 2010 | B2 |
7868879 | Rizko | Jan 2011 | B2 |
8006168 | Reznic et al. | Aug 2011 | B2 |
20020042917 | Townsend et al. | Apr 2002 | A1 |
20020071485 | Caglar et al. | Jun 2002 | A1 |
20020118763 | Harris et al. | Aug 2002 | A1 |
20020184384 | Simmon et al. | Dec 2002 | A1 |
20030118107 | Itakura et al. | Jun 2003 | A1 |
20030174243 | Arbeiter et al. | Sep 2003 | A1 |
20040253980 | Lane | Dec 2004 | A1 |
20050036546 | Rey et al. | Feb 2005 | A1 |
20050089105 | Zhang et al. | Apr 2005 | A1 |
20050123042 | Park et al. | Jun 2005 | A1 |
20050175251 | Taketa et al. | Aug 2005 | A1 |
20050208961 | Willenegger | Sep 2005 | A1 |
20070014356 | Han et al. | Jan 2007 | A1 |
20070047485 | Gorokhov et al. | Mar 2007 | A1 |
20070097257 | El-Maleh et al. | May 2007 | A1 |
20070098063 | Reznic et al. | May 2007 | A1 |
20070110396 | Itoh et al. | May 2007 | A1 |
20070189383 | Shao et al. | Aug 2007 | A1 |
20070195893 | Kim et al. | Aug 2007 | A1 |
20070202842 | Shao et al. | Aug 2007 | A1 |
20070202843 | Niu et al. | Aug 2007 | A1 |
20070206875 | Ida et al. | Sep 2007 | A1 |
20070223823 | Islam et al. | Sep 2007 | A1 |
20070230461 | Singh et al. | Oct 2007 | A1 |
20070234134 | Shao et al. | Oct 2007 | A1 |
20070268972 | Kim | Nov 2007 | A1 |
20080037624 | Walker et al. | Feb 2008 | A1 |
20080043832 | Barkley et al. | Feb 2008 | A1 |
20080063069 | Sekiguchi et al. | Mar 2008 | A1 |
20080123739 | Reznic et al. | May 2008 | A1 |
20080144553 | Shao et al. | Jun 2008 | A1 |
20080204461 | Hutchins et al. | Aug 2008 | A1 |
20090021646 | Shao et al. | Jan 2009 | A1 |
20090063935 | Singh et al. | Mar 2009 | A1 |
20090086638 | Niu et al. | Apr 2009 | A1 |
20090086699 | Niu et al. | Apr 2009 | A1 |
20090109866 | Kim et al. | Apr 2009 | A1 |
20090241147 | Kim et al. | Sep 2009 | A1 |
20090265744 | Singh et al. | Oct 2009 | A1 |
Number | Date | Country |
---|---|---|
2003152733 | May 2003 | JP |
2004034654 | Apr 2004 | WO |
2004073201 | Aug 2004 | WO |
Entry |
---|
U.S. Non-Final Office Action for U.S. Appl. No. 11/833,919 mailed May 4, 2011. |
U.S. Final Office Action for U.S. Appl. No. 12/107,685 mailed Apr. 21, 2011. |
Hitachi Ltd. et al. High-Definition Multimedia Interface (HDHI) Specifications version 1.2, Aug. 22, 2005. |
Wang, Y. et al., “Error Control and Concealment for Video Communciation: A Review,” Proceedings of the IEEE, vol. 86, No. 5, May 1998, pp. 974-997, United States. |
Gharai, L. et al., “RTP Payload Format for Uncompressed Video,” Network Working Group, The Internet Society, Sep. 2005, pp. 1-17, United States. |
Maruhashi, K. et al., “Wireless Uncompressed-HDTV-Signal Transmission System Utilizing Compact 60-GHz-band Transmitter and Receiver,” Microwave Symposium Digest, 2005 IEEE MTT-S International, Jun. 12-17, 2005, pp. 1867-1870, United States. |
IEEE Draft P802.15.3/D16, “Draft Standard for Telecommunications and Information Exchange Between Systems—LAN/MAN Specific Requirements, Part 15.3: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for High Rate Wireless Personal Area Networkers (WPANs),” LAN/MAN Standard Committee, IEEE Computer Society, Feb. 2003, pp. 1-362, New York, NY, United States. |
EIA/CEA-861-B, “A DTV Profile for Uncompressed High Speed Digital Interfaces,” EIA Standard, EIA Engineering Standards and Publications, Electronic Industries Alliance Technology Strategy & Standards Department, May 2002, pp. i-117, Arlington, VA, United States. |
Mboa et al., “Distributed Medium Access Control (MAC) for Wireless Networks,”0 Draft Version 0.99, MBOA/WiMedia Alliance, Nov. 1, 2005, pp. ii-176, United States. |
International Search Report and Written Opinion dated Jul. 9, 2007 for International Application No. PCT/KR2007/001527 from Korean Intellectual Property Office, filed Mar. 29, 2007, pp. 1-8, Seo-gu, Daejeon, Republic of Korea. |
International Search Report dated May 26, 2007 for International Application No. PCT/KR2007/000827 from Korean Intellectual Property Office, filed Feb. 15, 2007, pp. 1-2, Seo-gu, Daejeon, Republic of Korea. |
International Search Report for dated Mar. 10, 2008 International Application No. PCT/KR2007/006148 from Korean Intellectual Property Office, filed Nov. 30, 2007, p. 1, Seo-gu, Daejeon, Republic of Korea. |
Modiano, E., “An Adaptive Algorithm for Optimizing the Packet Size Used in Wireless ARQ Protocols,” Wireless Networks, vol. 5, No. 4, Jul. 1999, pp. 279-286, United States. |
Lettieri, P. et al., “Adaptive Frame Length Control for Improving Wireless Link Throughput, Range, and Energy Efficiency,” INFOCOM Seventeeth Annual Joint Conference of the IEEE Computer and Communication Societies, vol. 2, Mar.-Apr. 1998, pp. 564-571, United States. |
LG Electronics, Inc., “WirelessHD Specification Version 1.0 Overview,” Oct. 9, 2007, pp. 1-77, United States. |
“NEC Develops Compact Millimeter-Wave Transceiver for Uncompressed HDTV Signal Transmission”, NE Asia Online, Apr. 1, 2005, (Downloaded from http://neasia.nikkeibp.com/topstory/000913 on Sep. 29, 2006), pp. 1-2, United States. |
Hachman, M., “CE Giants back Amimon's Wireless HDTV Tech,” PCMAG.com, Jul. 23, 2008, p. 1, United States. |
International Standard ISO/IEC JTC 1/SC 29, “Information Technology, JPEG 2000 Image Coding System—Part 11: Wireless,” ISO/IEC, Mar. 21, 2005, pp. 1-72, United States. |
Hitachi, Ltd. et al., “High-Definition Multimedia Interface (HDMI) Specification Version 1.3,” Jun. 22, 2006, pp. 1-237, United States. |
International Search Report dated Oct. 12, 2007 for International Application No. PCT/KR2007/003251 from Korean Intellectual Property Office, filed Jul. 4, 2007, pp. 1-2, Seo-gu, Daejeon, Republic of Korea. |
International Search Report dated May 22, 2007 for International Application No. PCT/KR2007/000802 from Korean Intellectual Property Office, filed Feb. 15, 2007, pp. 1-2, Seo-gu, Daejeon, Republic of Korea. |
International Search Report dated Jul. 9, 2007 for International Application No. PCT/KR2007/001527 from Korean Intellectual Property Office, filed Mar. 29, 2007, pp. 1-2, Seo-gu, Daejeon, Republic of Korea. |
International Preliminary Report on Patentability & Written Opinion dated Sep. 30, 2008 for International Application No. PCT/KR2007/001527 from Korean International Property Office, filed Mar. 29, 2007, pp. 1-13, Seo-gu, Daejeon, Republic of Korea. |
International Preliminary Report on Patentability & Written Opinion dated May 19, 2009 for International Application No. PCT/KR2007/003251 from Korean International Property Office, filed Jul. 4, 2007, pp. 1-9, Seo-gu, Daejeon, Republic of Korea. |
International Preliminary Report on Patentability & Written Opinion dated Jun. 16, 2009 for International Application No. PCT/KR2007/006148 from Korean International Property Office, filed Nov. 30, 2007, pp. 1-9, Seo-gu, Daejeon, Republic of Korea. |
International Preliminary Report on Patentability & Written Opinion dated Aug. 19, 2008 for International Application No. PCT/KR2007/000827 from Korean International Property Office, filed Feb. 15, 2007, pp. 1-7, Seo-gu, Daejeon, Republic of Korea. |
International Preliminary Report on Patentability & Written Opinion dated Aug. 19, 2008 for International Application No. PCT/KR2007/000802 from Korean International Property Office, filed Feb. 15, 2007, pp. 1-9, Seo-gu, Daejeon, Republic of Korea. |
Caetano, L., “SiBEAM—60 GHz Architecture for Wireless Video Display,” SiBEAM, Inc., White Paper, Mar. 2006, pp. 1-6, United States. |
U.S. Non-final Office Action for U.S. Appl. No. 12/107,685 mailed Oct. 13, 2010. |
U.S. Final Office Action for U.S. Appl. No. 11/833,919 mailed Sep. 30, 2010. |
U.S. Restriction Requirement for U.S. Appl. No. 11/728,244 mailed Feb. 24, 2010. |
U.S. Non-Final Office Action for U.S. Appl. No. 11/728,244 mailed Jun. 3, 2010. |
U.S. Non-Final Office Action for U.S. Appl. No. 11/833,919 mailed May 26, 2010. |
U.S. Notice of Allowance for U.S. Appl. No. 11/833,919 mailed Jan. 10, 2012. |
U.S. Notice of Allowance for U.S. Appl. No. 12/107,685 mailed Jan. 5, 2012. |
U.S. Final Office Action for U.S. Appl. No. 11/833,919 mailed Oct. 17, 2011. |
U.S. Non-final Office Action for U.S. Appl. No. 12/107,685 mailed Sep. 6, 2011. |
Number | Date | Country | |
---|---|---|---|
20070189397 A1 | Aug 2007 | US |
Number | Date | Country | |
---|---|---|---|
60773828 | Feb 2006 | US |