Example embodiments of the present invention relate in general to error correction technology, and more specifically, to a graceful degradation-forward error correction (GD-FEC) method capable of being applied to multimedia broadcast and multicast service, and an apparatus thereof.
In multimedia packet communication through a mobile communication network, packet loss may occur due to the characteristics of mobile communication environments. For example, when a terminal passes through a shadow region, such as the back of a building, a tunnel, etc., burst packet loss may occur since the terminal cannot receive data.
As a protection method against such packet loss, a packet redirection method using Automatic Repeat Request (ARQ) or Hybrid ARQ (HARQ) has been proposed, however, the method fails to solve a problem of service quality degradation due to delay caused by packet redirection. That is, in non-realtime service, such as replaying stored video, delay causes little problem, but in real-time service, such as interactive video or video conferencing, delay has significantly great influence on service quality.
As a method for recovering lost packets without redirection in consideration of these conditions, application layer-forward error correction (AL-FEC) has been used.
AL-FEC increases redundancy in order to increase a coding rate according to an increase of packet loss. Also, in order to increase redundancy, the AL-FEC uses a method in which a receiver decides a code rate adaptively to a channel using a rateless code such as Fountain code, or a method of feeding the state of a channel back to a transmitter terminal from a lower layer by using the concept of an reception application layer or a cross layer to adjust AL-FEC redundancy of the transmitter terminal.
However, there may occur packet loss over a long period that cannot be recovered by the AL-FEC, and accordingly, a packet recovery method capable of providing seamless service while minimizing delay is needed.
Accordingly, example embodiments of the present invention are provided to substantially obviate one or more problems due to limitations and disadvantages of the related art.
An example embodiment of the present invention provides a graceful degradation-forward error correction (GD-FEC) method capable of recovering continuous data loss while minimizing delay.
Another example embodiment of the present invention also provides a GD-FEC apparatus for performing the GD-FEC method.
In some example embodiments, a graceful degradation-forward error correction (GD-FEC) method includes: deciding a data group for encoding in a plurality of data streams being successively received over time, the data group for encoding configured with a predetermined number of time durations; deciding at least one reference data group to be used for generating a parity frame in the individual time durations of the data group for encoding; generating a parity frame using the at least one reference data group; and distributing the parity frame into at least one time duration of a plurality of time durations including the time durations configuring the data group for encoding, according to a predetermined rule.
The deciding of the reference data group may include: comparing the time lengths of frames corresponding to a reference data type having a highest priority among data types included in the individual data durations of the data group for encoding, to each other; deciding the size of a frame having a longest time length among the frames, as a size of the reference data group; and deciding the reference data group including at least one data type according to predetermined recovery priority in each time duration, based on the size of the reference data group.
The generating of the parity frame may include: concatenating the reference data group to configure source data; and performing encoding on the source data to generate the parity frame.
The performing of the encoding on the source data to generate the parity frame may include performing RaptorQ-like encoding on the source data to generate the parity frame.
The generating of the parity frame may include generating the parity frame according to a predetermined period configured in unit of the reference data group.
The distributing of the parity frame may include distributing the parity frame to a time duration before the time durations configuring the data group for encoding, or to at least one time duration among the time durations configuring the data group for encoding, according to a predetermined rule.
The distributing of the parity frame may include dividing the parity frame to generate two or more divided parity frames each having a predetermined length; and distributing the two or more divided parity frames to time durations before the time durations configuring the data group for encoding, or to two or more time durations among the time durations configuring the data group for encoding. The generating of the divided parity frames may include deciding the size of each divided parity frame, based on the number of reference data groups having to be recovered among a plurality of reference data groups each corresponding to the reference data group and the size of each reference data group.
The deciding of the data group for encoding may include: deciding a Group of Picture (GoP) for each time duration having a predetermined length in pictures successively received over time; and deciding the data group for encoding configured with a predetermined number of GoPs each corresponding to the GoP.
The deciding of the reference data group may include: comparing the lengths of I pictures among picture types included in the individual GoPs configuring the data group for encoding, to each other; deciding the length of an I picture having a longest length among the I pictures included in the individual GoPs, as a (the) size of the reference data group; and deciding the reference data group including at least one picture type according to predetermined recovery priority in each GoP, based on the size of the reference data group.
In other example embodiments, a graceful degradation-forward error correction (GD-FEC) method which is performed by a transmission apparatus for transmitting multimedia data includes: deciding a data group for encoding configured with a predetermined number of time durations in a received data stream; deciding at least one reference data group to be used for generating a parity frame in the individual time durations included in the data group for encoding; generating a parity frame using the at least one reference data group; and transmitting the parity frame before transmitting data included in the data group for encoding.
The generating of the parity frame may include: concatenating the reference data group to configure source data; and performing RaptorQ-like encoding on the source data to generate the parity frame.
The transmitting of the parity frame before transmitting the data included in the data group for encoding may include transmitting the parity frame for a time duration in which the received data stream is buffered.
The deciding of the data group for encoding configured with the predetermined number of time durations in the received data stream may include deciding the time durations of the data group for encoding as the same duration per which a specific protocol segments and processes a file.
The deciding of the data group for encoding configured with the predetermined number of time durations in the received data stream may include deciding the time durations of the data group for encoding as the same duration as a file segment duration of a file delivery over unidirectional transport (FLUTE) protocol.
The GD-FEC method may further include performing interleaving in unit of a segment packet resulting from dividing a segment file included in the file segment duration of the FLUTE protocol.
In other example embodiments, a graceful degradation-forward error correction (GD-FEC) apparatus includes: a first encoding layer configured to perform GD-FEC on received multimedia data; a transport protocol processing layer configured to perform transport protocol processing on the data subject to the GD-FEC; and a second encoding layer configured to perform application layer-forward error correction (AL-FEC) on the data subject to the transport protocol processing.
The first encoding layer may perform the GD-FEC according to the type of the received multimedia data.
As described above, according to the method and apparatus for GD-FEC, packet loss over a long period which cannot be recovered by application layer-forward error correction (AL-FEC), may be recovered, thereby providing seamless service.
Also, by minimizing the size of redundancy data to be added for recovering lost data, encoding gain may be improved, resulting in improvement of transmission efficiency.
In addition, by minimizing delay due to GD-FEC encoding and decoding, loss compensation efficiency may be improved, which contributes to quality of service (QoS).
Throughout the drawings and the detailed description, unless otherwise described, the same drawing reference numerals will be understood to refer to the same elements, features, and structures. The relative size and depiction of these elements may be exaggerated for clarity, illustration, and convenience.
The following description is provided to assist the reader in gaining a comprehensive understanding of the methods, apparatuses, and/or systems described herein. Accordingly, various changes, modifications, and equivalents of the methods, apparatuses, and/or systems described herein will be suggested to those of ordinary skill in the art. Also, descriptions of well-known functions and constructions may be omitted for increased clarity and conciseness.
Accordingly, while the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the invention to the particular forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising,”, “includes” and/or “including”, when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Hereinafter, embodiments of the present invention will be described in detail with reference to the appended drawings. In the following description, for easy understanding, like numbers refer to like elements throughout the description of the figures, and the same elements will not be described further.
A graceful degradation-forward error correction (GD-FEC) method according to an embodiment of the present invention provides an optimized parity frame generation method for protecting burst packet loss, and the GD-FEC method is combined with a payload formatting method in order to disperse influence of inevitable delay to prevent the delay from influencing service, thereby improving coding gain and packet loss compensation efficiency.
For easy description on the configuration and operation of the present invention, formats, timing diagrams, encoding methods, parity bit generation methods, frame configuration and distribution methods, parity frame arrangement methods in consideration of transmission delay and reception delay, etc. of various types of GD-FEC methods according to some embodiments of the present invention will be first described.
The following description for the various types of GD-FEC methods will be given based on video data, however, the GD-FEC methods may be applied to multimedia data, such as audio, dynamic and interactive multimedia scenes (DIMS), text, etc., as well as video data.
First Type GD-FEC
Referring to
Meanwhile, in each GoP which is an element configuring G, an I frame is preferentially selected, and then frames to be preferentially protected according to predetermined priorities, are sequentially selected to configure a macro frame Mfr corresponding to the GoP according to the length of a macro frame Mfr. Here, the predetermined priorities may be recovery priorities upon occurrence of burst packet loss, and for example, the predetermined priorities may be set to the order of I, P, and B frames. The P frame is a picture obtained by interpicture forward predictive coding, and the B picture (frame?) is a picture obtained by interpicture bi-directional predictive coding.
Then, by concatenating the macro frames Mfr to configure a source symbol, and performing GD-FEC encoding on the source symbol, a parity frame Pfr is obtained as the result of the GD-FEC encoding.
The process of generating the parity frame in the first type GD-FEC may be expressed as equation 1, below.
In equation 1, an operator “<˜” is a function of generating a parity frame Pfr, wherein the function inserts the result obtained by performing RaptorQ-like encoding on a source symbol configured by concatenating macro frames Mfr in G, into each GoP in G. Also, “F” represents the RaptorQ-like encoding function, and “i” represents a frame time. The length of the parity frame Pfr is decided by theorem 1 and corollary 1, which will be described later.
Parameters of the first type GD-FEC are L=4, D=1, and G=1. Here, D is the number of GoPs in which the parity frame Pfr acquired as the result of GD-FEC encoding is carried, and in the first type GD-FEC shown in
Hereinafter, the configuration of the first type GD-FEC will be described in more detail with reference to equation 1. By concatenating four macro frames Mfr5, Mfr9, Mfr13, and Mfr17 respectively included in four GoPs GoP-2, GoP-3, GoP-4, and GoP-5 to configure a source symbol, and performing RaptorQ-like encoding (that is, the function F) on the source symbol, a parity frame Pfr4 is generated and located in a frame time duration t4.
Meanwhile, since the first type GD-FEC relates to an example in which G=1 and a GoP is configured with four frame time durations, the second parity frame Pfr8 will be located in a frame time duration t8. As expressed in equation 1, the parity frame Pfr8 is generated by concatenating four macro frames Mfr9, Mfr13, Mfr17, and Mfr21 respectively included in four GoPs GoP-3, GoP-4, GoP-5, and GoP-6 to configure a source symbol, and performing RaptorQ-like encoding on the source symbol.
Second Type GD-FEC
Hereinafter, the configuration and method of the second type GD-FEC will be described with reference to
As shown in
In the second type GD-FEC, since L=4, G includes four GoPs, and a parity frame Pfr is generated by concatenating macro frames Mfr respectively included in the four GoPs to configure a source symbol, and performing RaptorQ-like encoding on the source symbol.
Meanwhile, in the second type GD-FEC, since G=2, a parity frame Pfr is generated every two GoPs, and since D=1, a parity frame Pfr is carried in a GoP.
The process of generating the parity frame Pfr in the second type GD-FEC may be expressed as equation 2, below.
In equation 2, the meanings of an (the) operator “<˜”, “F”, and “i” have been described above in connection with equation 1.
As expressed in equation 2, a method of generating the first parity frame Pfr4 of the second type GD-FEC, and a location to which the parity frame Pfr4 is applied, are the same as in the first type GD-FEC, however, in the second type GD-FEC, since G=2, a method of generating the second parity frame Pfr12 and a location to which the parity frame Pfr12 is applied, are different from in the first type GD-FEC.
That is, in the second type GD-FEC, since G=2, the location to which the second parity frame Pfr12 is applied, may be a frame time duration t12 of GoP-3, and the parity frame Pfr12 may be generated based on four macro frames Mfr13, Mfr 17, Mfr 21, and Mfr 25 respectively included in four GoPs (that is, GoP-4, GoP-5, GoP-6, and GoP-7) successively located starting from GoP-4.
In the configuration of the second type GD-FEC shown in
Third Type GD-FEC
Hereinafter, the third type GD-FEC method will be described in more detail with reference to
In
For example, if the individual GoPs shown in
Also, in each GoP which is an element configuring G, an I frame is preferentially selected, and then frames to be preferentially protected according to predetermined priorities, are sequentially selected to configure a macro frame Mfr corresponding to the GoP according to the length of a macro frame Mfr. Here, the predetermined priorities may be recovery priorities upon occurrence of burst packet loss, and for example, the predetermined priorities may be set to the order of I, P, and B frames.
For example, as shown in
That is, as shown in
After macro frames Mfr are decided in the individual GoPs included in G, as described above, by concatenating the macro frames Mfr to configure a source symbol and performing GD-FEC encoding on the source symbol, divided parity frames DPfr are acquired as the result of the GD-FEC encoding. Each divided parity frame DPfr is a predetermined size of a frame divided from a parity frame Pfr, the parity frame Pfr generated by concatenating macro frames Mfr of a GoP group consisting of L GoPs and performing GD-FEC encoding, in order to add the parity frame Pfr to an appropriate location (for example, the final frame time duration) of each GoP.
Successively, the divided parity frames DPfr are added to corresponding GoPs.
In the third type GD-FEC, the process of generating and adding the divided parity frames DPfr may be expressed as equation 3, below.
In equation 3, an (the) operator “<˜” is a function of generating a parity frame Pfr, wherein the function inserts the result obtained by performing RaptorQ-like encoding on a source symbol configured by concatenating macro frames Mfr in G, into each GoP in G. Also, an (the) operator “//” is a function of dividing the length of a parity frame resulting from RaptorQ-like encoding by D to generate divided parity frames DPfr. Also, “F” represents the RaptorQ-like encoding function, and “i” represents a frame time. The length of a divided parity frame DPfr is decided by corollary 2, which will be described later.
Parameters of the third type GD-FEC may be given as L=4, D=2, and G=2. Here, the meanings of the parameters D and G are the same as those described above with reference to
Meanwhile, in the third type GD-FEC, since G=2, a parity frame Pfr is generated every two GoPs, and since D=2, a parity frame Pfr is divided to two divided parity frames DPfr, and each divided parity frame DPfr is located in a GoP. As a result, a parity frame Pfr is located in two GoPs.
In the third type GD-FEC shown in
Fourth Type GD-FEC
Hereinafter, the fourth type GD-FEC method will be described in more detail with reference to
In
If the individual GoPs shown in
Also, in each GoP which is an element configuring G, an I frame is preferentially selected, and then frames to be preferentially protected according to predetermined priorities, are sequentially selected to configure a macro frame Mfr corresponding to the GoP according to the length of a macro frame Mfr. Here, the predetermined priorities may be recovery priorities upon occurrence of burst packet loss, and for example, the predetermined priorities may be set to the order of I, P, and B frames.
For example, as shown in
That is, as shown in
After macro frames Mfr are decided in the individual GoPs included in G, as described above, by concatenating the macro frames Mfr to configure a source symbol and performing GD-FEC encoding on the source symbol, divided parity frames DPfr are acquired as the result of the GD-FEC encoding. The divided parity frames DPfr are added to corresponding GoPs.
The process of generating and adding the divided parity frames DPfr in the fourth type GD-FEC may be expressed as equation 4, below.
In equation 4, an (the) operator “<˜” is a function of generating divided parity frames DPfr, wherein the function inserts the result obtained by performing RaptorQ-like encoding on a source symbol configured by concatenating macro frames Mfr in G, into each GoP in G. Also, an (the) operator “//” is a function of dividing the length of a parity frame resulting from RaptorQ-like encoding by D-T to generate the divided parity frames DPfr. Also, “F” represents the RaptorQ-like encoding function, “T” represents the number of recoverable GoPs, that is, the number of recoverable macro frames Mfr when concatenation loss has occurred between the macro frames Mfr, and “i” represents a frame time. The length of a divided parity frame DPfr is decided by corollary 2, which will be described later.
Hereinafter, theorems are defined in order to deduce corollaries, and corollaries 1 and 2 are defined.
Theorem 1
If it is assumed that a channel capacity C is a bit B for each used channel, a bit rate R satisfies the relationship of R=Bk/(k+m)≦C=B, wherein k represents the length of a message, and m represents the length of a parity.
Also, if it is assumed that loss has generated in α of the message upon bearer transmission, the length of the remaining portion of the message, in which no loss has occurred, becomes (1-α)k, wherein m>αk.
Proof of Theorem 1
In theorem 1, a lost message may be interpreted to be lost when it passes through a binary erasure channel (BEC). Accordingly, an inequality expressed as equation 5 is obtained, and as a result, an inequality expressed as equation 6 has to be satisfied.
If it is assumed that m=αk, R′=1 which is greater than C′, which invalidates the assumption. Accordingly, the inequality of equation 6 is satisfied only when m>αk.
Corollary 1
It is seen from theorem 1 that when loss has generated in a of a message having the length of k bits upon bearer transmission, the required length of parity requires more bits than the lost bits. In this case, if loss recovery is performed using codes (for example, RaptorQ codes) being considered as complete recovery codes, it is possible to configure the length of parity to be close to the length of lost bits. Accordingly, m=αk+ε with respect to a sufficiently small positive number ε, is satisfied.
Corollary 2
The length of a divided parity frame may be calculated by equation 7, below.
According to theorem 1 and corollary 1, the amount of parity bits should be set to be larger than the amount of data bits having to be recovered. This means that in order to recover the bits of at least I frames when all frames have been lost due to loss of successive GoPs, more parity bits than the amount of bits of I frames have to be generated.
Consequently, since the length of parity required for recovering lost ak bits is m=αk+ε, the length of parity required for recovering lost T macro frames is [T·length(Mfr)+ε]. If all of T GoPs in G have been lost, desired information can be recovered by preforming RaptorQ-like decoding using D-T GoPs, and accordingly, the length of a divided parity frame required to recover the loss is given as in equation 7.
Referring to
The header of the parity frame may omit redundancy fields, and allow compression. Also, other parameter information excluded from the header of the parity frame may be transmitted through out-of-band signaling.
Hereinafter, the processing procedure of the GD-FEC method will be conceptually described, and then sequentially described in detail with reference to a 3rd Generation Partnership Project (3GPP) transmission protocol.
Referring to
The media equipment 810 may be a content server, and the transmission equipment 820 may be a broadcast multicast service center (BM-SC). Also, the network 830 may be a mobile communication network supporting multimedia broadcast and multicast service (MBMS).
Even though the GD-FEC and application layer-forward error correction (AL-FEC) can improve reliability and robustness of multimedia data transmission, parity data for the GD-FEC and AL-FEC is redundancy data in view of the media equipment 810. Accordingly, neither the GD-FEC nor the AL-FEC should be performed in general-purpose media equipment 810 for provision of multimedia, and the transmission equipment 820 performs the GD-FEC and AL-FEC on multimedia data, such as audio, video, DIMS, text, etc., provided from the media equipment 810.
However, since the GD-FEC needs to distinguish I frames upon encoding and decoding, unlike the AL-FEC, the GD-FEC has to be performed before distinguishing the type or priority of data received by the transmission equipment 820 becomes impossible due to interleaving, encoding, etc.
As shown in
That is, GD-FEC encoding may be performed in the uppermost layer among protocol layers, or the upper layer at least, than a layer performing transmission, as shown in
Meanwhile, the AL-FEC is performed in a layer just below a realtime transport protocol (RTP) layer, a realtime transport control protocol (RTCP) layer, or a secure RTP (SRTP) layer. Data input to the layer that performs the AL-FEC may be data subject to interleaving and encoding in the RTP or SRTP layer, and the layer that processes the AL-FEC may group data to symbols having the same priority, regardless of the type, priority, etc. of the data, and encode the symbols.
That is, as shown in
As described above, transmission equipment that performs GD-FEC may have a protocol layer structure in the order of a GD-FEC processing layer, RTP & RTCP layers, SRTP & FEC layers, and a user datagram protocol (UDP).
Meanwhile, decoding delay of data subject to GD-FEC encoding may cause various problems such as switching time delay, etc. in the system.
Hereinafter, a method for minimizing delay in the GD-FEC method will be described.
In detail, referring to
Here, the repair packets are parity frames. As described above with reference to
Also, the received source packet stream may be buffered in the transmission equipment, and then GD-FEC encoding may be performed on the buffered source packets. In this case, transmission delay (or buffering delay) may be generated by a time duration for which the GD-FEC is performed.
After the GD-FEC encoding is performed, the transmission equipment transmits the repair packets acquired through the GD-FEC encoding through a network (S1120).
Recovery equipment receives the repair packets transmitted from the transmission equipment (S1130). The recovery equipment may receive the repair packets after a predetermined time duration (that is, a transmission delay time through the network) has elapsed. Also, the recovery equipment may receive a part of the repair packets due to loss of some repair packets.
Then, the recovery equipment buffers the received repair packets, and recovers the audio packets using the buffered repair packets (S1140).
That is, according to the current embodiment, since transmission equipment acquires repair packets by performing GD-FEC encoding for the time duration of buffering for transmitting a packet stream, and transmits the repair packets to recovery equipment for the buffering time duration through a network so that the recovery equipment preferentially recovers packets using the repair packets, no delay due to GD-FEC encoding occurs although network transmission delay occurs, when source packets are transmitted by performing GD-FEC encoding.
Referring to
Accordingly, GD-FEC encoding may be performed for a time duration in which a FLUTE file segment is processed, so that no additional delay due to GD-FEC encoding occurs.
Also, according to another embodiment of the present invention, GD-FEC decoding may be performed for a time duration in which a FLUTE file segment is processed, so that no additional delay due to GD-FEC decoding occurs.
Specifically, the performance of GD-FEC may be improved through interleaving, and by setting a time duration for interleaving in unit of a FLUTE segment packet, additional delay due to interleaving may be prevented.
In addition, by dividing a FLUTE segment file to segment packets, and transmitting the segment packets according to predetermined priorities, instead of interleaving, the same effect as interleaving may be observed.
A number of examples have been described above. Nevertheless, it will be understood that various modifications may be made. For example, suitable results may be achieved if the described techniques are performed in a different order and/or if components in a described system, architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components or their equivalents. Accordingly, other implementations are within the scope of the following claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2012-0007404 | Jan 2012 | KR | national |
10-2012-0125373 | Nov 2012 | KR | national |
10-2013-0004438 | Jan 2013 | KR | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/KR2013/000572 | 1/24/2013 | WO | 00 |