Patent Application
20100263007
This invention relates generally to Internet protocol television (IPTV) in a home network, and more particularly to using application layer forward error correction (AL-FEC) to increase the quality of IPTV.
Internet Protocol Television
Internet protocol television (IPTV) provides television contents using the Internet protocol (IP) instead of using conventional broadcast and cable channels. IPTV is typically bundled with other services such as Video on Demand (VOD), Voice over IP (VOIP), or digital phone using Internet access, collectively referred to as “triple play.”
IPTV has a number of unique features. In a conventional TV network, all available content is continuously transmitted to each customer. Then, the customer selects the specific content to watch. This consumes network bandwidth, e.g., 6 MHz per channel or 600 MHz for a 100 channel cable network.
In IPTV, only the content selected by the customer is transmitted. Moreover, IPTV has the ability for digital video recorders (DVRs) to concurrently record broadcasts. It also allows customers to have picture-in-picture (PIP) viewing without leaving the current program. IPTV also used packets, which enable error correction, which is not available for conventional TV signals.
Forward Error Correction
With forward error correction (FEC), the sender adds an error-correcting code (ECC) to the packets to be transmitted. Then, the receiver can detect and correct a most errors. One important class of FEC includes linear block error-correcting codes. In general, FEC is used at data link and physical layers of the International Organization for Standardization (ISO) protocol stack to reduce packet loss and ensure a certain Quality-of-Service (QoS) for all content.
Application Layer Forward Error Correction
One challenge for high-quality IPTV services is to reduce noticeable artifacts caused by packet loss. To achieve a high quality, application layer forward error correction (AL-FEC) can be used, in addition to the FEC protection at the link and physical layers.
For an optional protocol for AL-FEC protection of streaming media for IPTV services carried over Real-time Transport Protocol (RTP), see ETSI TS 102 034 v1.3.1, “Transport of MPEG 2 Transport Stream (TS) Based DVB Services over IP Based Networks,” DVB Blue Book A086rev5, and M. Luby, T. Stockhammer, and M. Watson, “IPTV Systems, Standards and Architectures: Part II Application Layer FEC in IPTV Services,” IEEE Communications Magazine, pp. 94-101, May 2008.
In ETSI TS 102 034, the AL-FEC protocol applies a layered structure as shown in
Interleaved Parity Code (SMPTE 2022-1 code)
If any data packet is lost during transmission, SMPTE 2022-1 decoding is performed. If only one data packet is missing in one column and the corresponding repair packet is received, then this missing data packet can be recovered by exclusive-ORing the repair packet with the data packets in this column.
For details about the SMPTE 2022-1 code, see SMPTE specification 2022-1, “Forward Error Correction for Real-time Video/Audio Transport Over IP Networks.”
Raptor Codes
Raptor codes are one kind of fountain codes. Fountain codes are also known as rateless erasure codes and they belong to linear block codes. Raptor codes have the property that for a fixed number of data symbols, the number of encoding symbols (including both data and repair symbols) can be potentially limitless, and the data symbols can be recovered from any subset of the encoding symbols of size equal to or slightly larger than the number of data symbols, see U.S. Pat. No. 6,307,487.
Although LT-codes have excellent performance, they do not have constant encoding and linear decoding complexity without compromising error performance. Raptor codes are an extension of LT-codes, which overcome this drawback, see U.S. Pat. Nos. 6,307,487 and 7,139,960.
The raptor code used in IPTV standard is a systematic linear block code. The property of systematic codes is that their code-words can be partitioned into two parts, one containing the unaltered data symbols and the other containing repair symbols. Unless specified otherwise, any reference to “raptor codes” herein refers to systematic raptor codes.
In order to apply the raptor encoder to data packets, the source packets can be partitioned into blocks. The raptor encoder is applied independently to each block. Each block is identified by a unique integer Source Block Number (SBN). Each block is partitioned into K source symbols. Each source symbol is identified by a unique integer Encoding Symbol Identifier (ESI).
Each encoding packet includes data symbols or repair symbols. The Encoding Symbol ID, X, in each data or repair packet is the Encoding Symbol ID of the first data or repair symbol carried in that packet. The subsequent data or repair symbols in the packet have Encoding Symbol IDs, X+1 to X+G−1, in sequential order, where G is the number of symbols in the packet. A packet may contain any number of symbols for the same block.
There are “pre-coding relationships” between the last L−K and the first K intermediate symbols. The pre-coding relationships are
G
pre-coding
·E=0, (1)
where Gpre-coding is an (L−K) by L binary matrix and 0 is a column vector of length (L−K).
In the binary matrix, the elements are 0 or 1, and · denotes matrix multiplication over a Galois field of two elements GF(2). An example binary matrix is
and if
β=a+b and γ=b+c.
As shown in
C=G
LT
·E, (2)
Let D denote the column vector consisting of L−K zero symbols followed by the K source data symbols. Define a matrix A over the GF(2) as
Then,
A·E=D. (4)
The intermediate symbols are
E=A
−1
·D. (5)
Matrix A is full rank and invertible.
After the intermediate symbols 631 have been generated, repair symbols 651 are produced by LT encoder 640. We denote repair symbols by
R=(R[0], R[1], . . . , R[J−1]),
where J is the number of repair symbols generated. Then L intermediate symbols and J repair symbols can be related by a J×L binary matrix G′LT, i.e.,
R=G′
LT
·E. (6)
One or more repair symbols are grouped to form a repair packet.
The decoding process is described as follows. Let V≧K be the number of received encoding symbols for a source block and let M=L−K+V. Let D′ be the column vector of M symbols with values known to the receiver, where the first L−K of the M symbols have values of zeroes, and the remaining V of the M symbols are the received encoding symbols for the source block.
An M by L binary matrix A′ can be generated based on the received encoding symbols, which is composed of three parts. The first part of A′ is Gpre-coding. The second part of A′ contains rows in GLT that correspond to the received source symbols. For example, if the 1st, 3rd, 5th, . . . , source data symbols are received at the receiver, then the 1st, 3rd, 5th, . . . , rows of GLT are selected. The third part of A′ contains rows in G′LT that correspond to the received repair symbols. Then
A′·E=D′. (7)
Decoding a source block is equivalent to solving E from A′ and D′. If and only if the rank of A′ is L, then E can be decoded and the missing source data symbols can be recovered using equation (4).
For more detailed information about raptor encoding and decoding, see ETSI TS 102 472, “Digital Video Broadcasting (DVB); IP Datacast over DVB-H: Content Delivery Protocols.”
Decoding of AL-FEC for IPTV
Decoding is carried out at the receiver side as follows:
Step 1: SMPTE 2022-1 decoding. In this step, SMPTE 2022-1 decoding is used and all missing data packets are not recovered, go to Step 2.
Step 2: raptor decoding. In this step, the raptor repair packets, together with the received data packets and any data packets which were recovered in Step 1, are processed using standard raptor decoding procedures (as described above) to recover more data packets. If there are still missing data packets after raptor decoding, perform Step 3.
Step 3: Decoding. In this step, the unprocessed SMPTE 2022-1 packets are converted to a form in which they can be added to the raptor decoding process, and raptor decoding continues. For details of the decoding, see ETSI TS 102 034 v1.3.1, “Transport of MPEG 2 Transport Stream (TS) Based DVB Services over IP Based Networks,” DVB Blue Book A086rev5, Oct. 2007.
Although current AL-FEC schemes can effectively reduce packet loss, for some home networks, especially wireless networks, additional packet loss will occur after the packet enters the home network. The packet loss at the end devices could be more severe than expected sometimes. In this case, packets reaching the home gateway (HG) from the IP network are sufficient to recover all the source information, but IPTV-STB can not do so because of considerable packet loss within the home network.
To solve this problem, QoS enhancement is required. A QoS-adjustable home gateway is described by S. Lee, C. Cho, and I. Han, “QoS-adjustable home gateway for IPTV service,” ICCE, January 2005, pp. 395-396. They describe changing priority level of received data streams and do not deal with packet loss within home network. One solution to the problem is that the receiver 340 can request the source 310 to send additional FEC, see US Patent Publications 20080028279, 20080028280.
However, the source is often far away from IPTV-STB, and communicating with the source causes delays. Thus, a scheme for reducing packet loss and that operates entirely in the home network is desired.
This invention operates in an error-prone home network, especially, when additional packet loss is caused by the home network so that source data can not be totally recovered by the IPTV-STB using error control codes embedded in the transmitted packets.
The embodiments of the invention provide QoS enhancement components for both the home gateway (HG) and the STB, and enable the STB to send the HG statistics signals, which characterize an operation of the home network, and the packets. If the home network has severe packet loss and the quality of IPTV services can not meet requirements, then the QoS enhancement components is activated.
The embodiments of the invention use real-time transport control protocol (RTCP) for STB to transmit the feedback signal to the HG. RTCP is related to real-time transport protocol (RTP). The RTCP provides out-of-band control information for an RTP flow, and also feedback on the QoS provided by the RTP.
The QoS enhancement components in the HG and the STB have a layered structure. The first layer enhancement is to add more protection to the SMPTE repair packets. And the second layer adds more protection to the raptor packets. When QoS enhancement components are activated, one or more appropriate layers are selected based on the statistics fed back from the STB.
The interface acts as a router, which distributes different services to corresponding devices. IPTV multi-media content is forwarded from the gateway through a residence network 132 via wireless or wired connections, then received and processed by an IPTV set-top box (IPTV-STB) 133, and displayed on, e.g., a TV 134.
The home network 132 is a local area network (LAN), which connects the devices 130 in the residence 125 through wireless LAN, cables, fiber optics or other possible media. The IPTV-STB 133, e.g., decodes the content for display on the TV. The IPTV-STB also provides two-way communication, which makes video-on-demand (VOD) and other interactive services possible.
The IPTV-STB makes use of real-time transport protocol (RTPC) 701 to transmit the statistics to the HG 131. The RTCP is defined in Request for Comment (RFC) 3550, which supersedes RFC 1889 of the Internet Official Protocol standard.
The statistics characterize on operation of the home network 132 as well as the packets received by the IPTV-STB, such as bytes sent, packets sent, lost packets, jitter, and round trip delay. A HG controller 810 of the HG analyzes the statistics information, and determines what kind of data packets 702 to send to IPTV-STB, and from there to home network devices, such as the television 134.
As described in greater detail below, the home network be subject to low, medium and high error rates, i.e., LL<MM<HH. If the error rate is low, then only source packets are sent. If the error rate is medium, then enhancement repair packets are also sent. If the error rate is high, the HG also sends enhancement raptor packets.
As shown in
The source, repair, and raptor packets 901 are always forwarded to the output 702 of the QoS enhancement encoder. When switch 903 is closed, the first layer enhancement encoder 920 is activated to send SMPTE 2022-1 enhancement repair packets. When switch 904 is closed, a second layer enhancement encoder 930 is activated to send enhancement raptor packets.
As shown in
In our first layer enhancement encoder 920, a buffer collects SMPTE 2022-1 repair packets and checks the SNBase low bits field in the header of the packets. Regard packets with SNBase low bits values i, i+f, i+2f, . . . , i+(r−1)f, where i<f, as a set. Then there are f such sets that are disjoint.
If all the r packets in the same set have been stored in the buffer, the packets are exclusive-ORed to generate our enhancement repair packet. The FEC header of our enhancement repair packet is the same as that of the SMPTE 2022-1 repair packet with the smallest SNBase low bits value in this set, except that the Offset field 520 is changed to f and the NA field 530 is changed to Dr, see
Because our additional enhancement repair packets 1030 generated by the first layer enhancement encoder 920 are compatible with the current SMPTE 2022-1 decoder, the STB can decode our new enhancement repair packets.
If error rate in the home network is relatively high, then the second layer enhancement encoder 930 is also activated. The second layer enhancement encoder also uses a buffer to store the received raptor encoding packets (including source and repair packets).
The buffer arranges the received raptor packets in a Q by y array. Then, the Q raptor packets in the same column are XORed to generate a new enhancement raptor packet. This arrangement can effectively reduce the burst errors in the home network. Because the raptor packets do not necessarily have the same size, the smallest size of the packets that participate in the XOR operation is selected as the size of the corresponding enhancement raptor packet. For the raptor encoding packets with larger size, their extra symbols are not involved in the XOR operation.
The IPTV-STB 133 has an additional process to deal with enhancement raptor packets 1140.
If the number of received enhancement packets is Ne and the number of symbols in each enhancement raptor packet is Wi for 0≦i≦Ne−1, then, the jth symbol in the enhancement raptor packet i is ei,j, where 0≦i≦Ne−1 and 0≦j≦Wi.
Let
{tilde over (D)}=[D′|e
0,0
e
0,1
. . . e
0,W
−1
e
1,0
e
1,1
. . . e
1,W
−1
. . . e
N
−1,0
. . . e
N
−1,W
−1],
i.e., the concatenation of D′ and all the symbols in the received enhancement packets. Because the length of D′ is M, the length of
For each symbol ei,j in {tilde over (D)}, a new row is added to the matrix to A′. The new row is generated as follows.
Step 1: The row a is initialized to zero.
Step 2: For 1≦l≦Q, the lth Encoding Symbol ID field of enhancement raptor packet i is read out and let it be ESIl. If ESIl<K, then the (ESIl+j)th row in GLT is XORed to a. If ESIl≧K, then the (ESIl−K+j)th row in G′LT is XORed to a.
After all the new rows are added to A′, a new matrix à is generated, which has
more rows compared to A′ and
÷E={tilde over (D)}. (9)
As long as à has a rank at least L, E can be solved. Because à has more rows than A′, it has a higher probability to meet the rank requirement.
QoS Enhancement STB Decoding
There are three cases for the STB decoder 136.
Case 1: no QoS enhancement function is activated. In this case, the STB decoder works as specified in the ETSI TS 102 034 standard.
Case 2: only the first layer enhancement encoder 920 is activated. For this case, in Step 1 of STB decoding, both the original SMPTE 2022-1 repair packets 430 and the enhancement repair packets 1030 generated by the first layer enhancement encoder are decoded by the conventional SMPTE 2022-1 decoder. Step 2 is the same as that in ETSI TS 102 034. In Step 3, besides the virtual raptor packet generated by the unprocessed SMPTE 2022-1 repair packets 430, another virtual raptor packet can be produced by the unprocessed new enhancement repair packets 1030. Then raptor decoding continues.
Case 3: both the first and second layer enhancement encoders are activated. For this case, in Step 1 of STB decoding, both the original SMPTE 2022-1 repair packets 430 and the enhancement repair packets 1030 generated by the first layer enhancement encoder are decoded by conventional SMPTE 2022-1 decoder. In Step 2, enhancement raptor packets 1140 are involved in raptor decoding as described above. In Step 3, besides the virtual raptor packet generated by the unprocessed SMPTE 2022-1 repair packets 430, another virtual raptor packet can be produced by the unprocessed enhancement repair packets 1030. Then raptor decoding in Step 2 is continued.
Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.
