The invention relates to a method and apparatus for transmitting encoded media data.
Conventionally, in a best-effort transmission line environment such as the Internet, a large volume of encoded packet data (media data) such as streaming information or video telephony is transmitted continuously.
In the best-effort transmission line environment as described above, congestion occurs due to an increase in communication traffics in a network. Quality of media data being transmitted deteriorates noticeably due to this congestion. Examples of a technique for preventing such deterioration in quality of media data include techniques as described below.
As the technique, for example, there is a system using a redundant code such as a Forward Error Correction (FEC) system. In addition, as the technique, there is a system using retransmission such as an Automatic Repeat Request (ARQ).
In the FEC system, parity information of media data is generated and sent on a transmission side, whereby, even in the case where a packet loss occurs on a reception side, it becomes possible to perform restoration of the lost packet.
In the ARQ system, when the packet loss is detected on the reception side, the reception side requests a transmission side to retransmit the lost packet, and the transmission side retransmits the lost packet, whereby it becomes possible to prevent deterioration in quality of media data.
Further, in the best-effort transmission line environment, conventionally, the deterioration in the quality of the media data due to the packet loss is prevented by the following techniques: a technique that switches between the FEC system, which is effective for a stationary packet loss, and the ARQ system, which is effective for a burst packet loss; and a technique that uses a combination of the FEC system and the ARQ system (e.g., see patent documents 1 and 2).
Note that, as other related art to the retransmitted invention, for example, there are techniques disclosed in patent documents 3 to 5 described below.
[Patent Document 1]
Japanese Patent Application Laid-Open Publication No. 7-67175
[Patent Document 2]
Japanese Patent Application Laid-Open Publication No. 2002-330118
[Patent Document 3]
Japanese Patent Application Laid-Open Publication No. 9-214474
[Patent Document 4]
Japanese Patent Application Laid-Open Publication No. 9-298526
[Patent Document 5]
Japanese Patent Application Laid-Open Publication No. 2000-228676
However, in those techniques, for example, there are problems as described below.
First, in the technique of switching the FEC system and the ARQ system, the FEC system and the ARQ system are switched using feedback information from the reception side. However, in the best-effort transmission line environment in which plural apparatuses are interconnected, it is difficult to estimate communication traffics by other apparatuses. Therefore, the above techniques may not be able to cope with congestion that occurs suddenly.
In addition, in the case where the FEC system and the ARQ system are combined, there is a problem in that transmission bands increase due to an increase in retransmission, which adversely affects other communication traffics. In particular, when congestion occurs, the packet loss occurs suddenly (in a burst-like manner). Thus, when retransmission is performed in this state, transmission bands increase. As a result, a state of a transmission line is further deteriorated. Therefore, with the technique, a retransmitted packet is lost and the effect of the ARQ is not exhibited.
The retransmitted invention has been devised in view of the problems described above. It is a primary object of the invention to provide a technique of, in the event that a loss has occurred with respect to a packet sent into a transmission line, compensating for the loss efficiently. In addition, it is a secondary object of the invention to provide a technique for controlling congestion in a transmission line that is caused by retransmitting data compensating for the transmitted packet.
To solve the above-mentioned problems, the present invention adopts the following means. That is, the present invention may be specified as a transmission apparatus, including: a communication unit performing transmission and reception of an encoded data packet with other terminals; a redundant code generating unit generating a redundant code associated with the data packet for restoring a loss of the data packet; a redundant code control unit controlling a band used for the redundant code; a buffer for retransmission accumulating retransmission packets that are retransmitted to the other terminals at a time of the loss of the data packet; a retrieval unit retrieving a corresponding retransmission packet from the buffer for retransmission when receiving a retransmission request for the data packet from the other terminals; a retransmission control unit controlling a band used for retransmission that is used at a time of transmission of the retransmission packet; and a band control unit controlling the bandwidth of a redundant code based on the band used for retransmission and a band used for the redundant code.
In the invention, in order to compensate for data lost at the time of transmission of an encoded data packet, the transmission apparatus performs control to keep the sum of a band used for retransmission and a band used for the redundant code at a constant amount at the time of transmitting redundant code data and retransmission data.
Therefore, according to the present invention, it is possible to provide a technique for, in the event that a loss has occurred with respect to a packet sent into a transmission line, compensating for the loss efficiently.
Further, according to the present invention, the band control unit may instruct the redundant code control unit to reduce the bandwidth of the redundant code based on the band used for retransmission.
Further, according to the present invention, after reducing the bandwidth of redundant codes based on the band used for retransmission, the band control unit may instruct the redundant code control unit to increase the bandwidth.
In this way, in the invention, an amount of information to be associated with a data packet can be controlled according to a condition in a transmission line, whereby bands in the transmission line can be used effectively.
In addition, in the invention, the redundant code generating unit generates a redundant code packet, and the band control unit may control the bandwidth of the redundant code by controlling an insertion interval of the redundant code packet at the time of retransmission of the data packet.
Further, according to the present invention, the band control unit may set a threshold value for each or both of the band used for retransmission of the data packet and the band used for the redundant code, and when the band used for retransmission and the band used for the redundant code exceed the threshold value, the band control unit may stop the retransmission of the data packet and the generation of the redundant code.
In this way, according to the present invention, it is possible to provide a technique for controlling congestion in a transmission line that is caused by retransmitting data compensating for a transmitted packet.
Further, the present invention may be specified as a transmission apparatus comprising: a communication unit performing transmission and reception of an encoded data packet with other terminals; a redundant code generating unit generating a redundant code for restoring a loss of the data packet; a redundant code control unit controlling a band used for transmission of the redundant code; a buffer for retransmission accumulating retransmission packets that are retransmitted to the other terminals at a time of the loss of the data packet; a retrieval unit retrieving a corresponding retransmission packet from the buffer for retransmission when receiving a retransmission request for the data packet from the other terminals; a retransmission control unit controlling a band used for retransmission that is used at a time of transmission of the retransmission packet; and a band control unit controlling a transmission ratio of the redundant code based on the band used for retransmission and a band used for transmission of the redundant code.
Note that, the invention may be implemented a program realizing any one of the functions described above. In addition, in the invention, such a program may be recorded in a computer readable storage medium. Further, the invention may be implemented as a method of realizing any one of the functions described above.
According to the invention as described above, such an excellent effect that it is possible to provide a technique for, in the event that a loss has occurred with respect to a packet sent into a transmission line, compensating for the loss efficiently is provided. In addition, the invention can also provide a technique for controlling congestion in a transmission line that is caused by retransmitting data compensating for a transmitted packet.
A best mode for carrying out the invention will be hereinafter explained with reference to the drawings.
Constitutions of embodiments to be described below are examples, and the invention is not limited to the constitutions of the embodiments.
Apparatus Structure of a Transmission Apparatus
A functional structure of a transmission apparatus of the invention will be explained with reference to the drawings.
The transmission apparatus 10 in accordance with this embodiment includes an encoding unit 11, a packtizer 12, a data transmission unit 13 (equivalent to a communication unit of the invention), a redundant code generating unit 14, a redundant code transmission unit 15 (equivalent to the communication unit of the invention), a redundant code control unit 16, a buffer for retransmission 17, a retransmission request receiving unit 18 (equivalent to the communication unit of the invention), a retransmission unit 19 (equivalent to the communication unit of the invention), a retransmission control unit 20, a feedback receiving unit 21 (equivalent to the communication unit of the invention) and a band control unit 22.
The encoding unit 11 encodes the media data. The packetizer 12 packetizes the media data encoded by the encoding unit 11. The data transmission unit 13 transmits the media data packetized by the packetizer 12 to a client.
The redundant code generating unit 14 generates a redundant code packet that is used in the event that a loss has occurred in the sent media data. The redundant code transmission unit 15 transmits the redundant code packet generated by the redundant code generating unit 14 to the client.
The redundant code control unit 16 controls a band that is used by redundant codes to be sent. The buffer for retransmission 17 accumulates packets of media data for retransmission. The retransmission request receiving unit 18 receives a retransmission request for the data packet from a client (reception side) terminal.
The retransmission control unit 20 receives the retransmission request for the data packet from the client terminal, retrieves a corresponding retransmission packet from the buffer for retransmission 17, and transmits the retransmission packet to the retransmission unit 19. The retransmission unit 19 transmits this retransmission packet to client. In addition, the retransmission control unit 20 controls a band used by the retransmission packet.
The feedback receiving unit 21 receives feedback information such as a packet loss ratio from the client terminal. The band control unit 22 determines transmission bands for all data to be sent including media data, retransmission packets, and redundant code data from the feedback information received by the feedback receiving unit 21. In addition, the band control unit 22 controls (manages) the transmission band of all the data to be sent.
With the functions described above, in the case where a redundant code system and a retransmission system are combined in compensating for a loss that occurs at the time of transmission of the encoded data, the transmission apparatus 10 keeps the sum of a band used by retransmission packets and a band used by redundant codes at a constant amount.
In order to perform the processing, the retransmission control unit 20 of the transmission apparatus 10 sequentially notifies the band control unit 22 of bands used for retransmission. The band control unit 22, which has received the band used for retransmission, sequentially instructs the redundant code control unit 16 to reduce the attachment ratio of the redundant code data 2 according to an amount of information in the band used for retransmission. In addition, the transmission apparatus 10 reduces insertion of redundant code packets (widens an insertion interval of redundant code packets) to thereby reduce the attachment ratio of the redundant code data 2 when retransmission of media data has occurred.
In addition, with the functions described above, after reducing the attachment ratio of the redundant code data according to a band used for the retransmission of the media data, the transmission apparatus 10 returns the attachment ratio of the redundant code data to an original ratio. In a position of II in
In order to realize the processing described above, the band control unit 22 instructs the redundant code control unit 16 to reduce the attachment ratio of the redundant code data 2 according to the band to be used by the retransmission data 3 that are sequentially notified from the retransmission control unit 20. Then, after the transmission of the retransmission data 3, the band control unit 22 instructs the redundant code control unit 16 to return the attachment ratio of the redundant code data 2 to the original ratio. In this case, the transmission apparatus 10 returns the insertion interval of the redundant code packets to the original insertion interval to thereby return the attachment ratio of redundant codes to the original ratio.
In addition, a redundant code to be sent by the transmission apparatus 10 use a redundant code packet to which parity information among plural packets is attached. Therefore, even if a packet loss has occurred on the Internet, the transmission apparatus 10 is capable of recovering the lost packet using this redundant code packet.
Moreover, with the function described above, in the event that the band used for retransmission of media data and the band used by redundant codes exceed a threshold value, the transmission apparatus 10 stops the retransmission of the media data and the attachment of the redundant codes.
When the band control unit 22 stops the attachment of the redundant code data 2 and the transmission data 3a and 3b, a free band 5 is formed in the entire transmission band. By providing this free band 5, it becomes possible to prevent a situation where a transmission line is further deteriorated by an increase in transmission bands due to retransmission at the time when congestion has occurred. Then, after the condition in the transmission line has improved (IV in
Example of Processing by the Transmission Apparatus
As an example of processing by the transmission apparatus of this embodiment, a case in which moving picture code data of MPEG-2 (Moving Picture Experts Group phase 2) is transmitted by an RTP (Real-time Transport Protocol) will be explained using the drawings.
In this processing example, a feedback mechanism called an RTCP (RTP Control Protocol) specified by RTP as feedback information is used. Note that, as the RTCP, there are a Sender Report (SR), which is sent from a server to a client every fixed time, and a Receiver Report (RR), which is sent from the client to the server every fixed time.
Redundant Code Processing in this Processing Example
Processing using a redundant code of this processing example will be explained using
The encoding unit 101 applies encoding of MPEG-2 to inputted media data (e.g., moving picture data) (S101). The encoded media data is sent to the packetizer 102 (S102). The packetizer 102 packetizes this media data.
The packetizer 102 transmits the packetized media data to the RTP transmission unit 103 (S103). The RTP transmission unit 103 attaches an RTP header to the media data and transmits the media data to a client through the Internet (S104).
PT (7 bits) is a payload type and indicates an encoding system for the application data. Sequence Number (16 bits) indicates a transmission order of packets. This Sequence Number is used for identifying individual packets. Natural numbers to be given in the transmission order of packets are placed in the field of Sequence Number.
TimeStamp (32 bits) indicates the time at which the first byte of the packet was sent. SSRC (synchronous sender identifier) (32 bits) indicates a sender of the packet. CSRC contribution source identifier) (32 bits) indicates a host that has prepared the packet.
After the media data is sent, a copy of a packet of the media data is stored in the buffer for retransmission 105 as retransmission data (S105).
In addition, the band control unit 106 determines an entire transmission band and a transmission band of FEC using information (S108) sent from the RTCP transmission and receiving unit 107. In this case, as information to be used for the determination, for example, there are a packet loss ratio and jitter included in a Receiver Report (S106), which is sent from the client every fixed time. In addition, as other information to be used for the determination, for example, there are a Sender Report (S107), which is sent from the server side every fixed time, and a round trip time (RTT), which is calculated using the Receiver Report. Note that it is desirable to use an existing determination method as a specific method of determining a transmission band.
When the entire transmission band is determined, the band control unit 106 determines the transmission band of FEC in accordance with information on a packet loss included in the Receiver Report.
As the FECpacket used in this processing example, for example, a packet defined in RFC2733 “An RTP Payload Format for Generic Forward Error Correction” is used. In RFC2733, an FEC packet, for which a parity of the entire packet is calculated, is provided for plural RTP packets. Thus, according to RFC2733, when a packet loss has occurred, a lost packet can be restored using the FEC packet. Note that, in RFC2733, it is recommended to attach at least one FEC packet to twenty-four RTP packets.
The generation of the FEC packet is performed as described below. The packetizer 102 duplicates a generated encoded data packet. The packetizer 102 transmits the duplicated encoded data packet to a buffer for FEC generation (not shown) in the FEC generating unit 104 (S109).
Thereafter, when a predetermined quantity of packets (e.g., twenty-four packets) are accumulated in the buffer for FEC generation, the FEC generating unit 104 generates parity information in accordance with a transmission band of FEC (S110) instructed by the band control unit 106.
In other words, the parity information generated by the FEC generating unit 104 is generated by calculating an exclusive OR by a unit of bit of the respective RTP packets.
The RTP packet to be the object of the FEC processing is defined by the FEC generating unit, for example, as described below. First, it is assumed that an i-th bit in a Mask bit of the twenty-four packets from SN base is “1”. In this case, an (SN Base+I)-th packet is the RTP packet to be the object of the FEC processing. The FEC generating unit 104 transmits the FEC packet generated in this way to the RTP transmission unit 103 (Sill). In addition, the FEC generating unit 104 notifies the band control unit 106 of a band to be used by the FEC packet (S113). The band control unit 106 controls a band used by retransmission data to be described later according to the band to be used by the FEC packet (S114).
The RTP transmission unit 103 transmits this FEC packet together with the RTP packet (S112).
Retransmission Processing of this Processing Example
Next, a flow of processing at the time of retransmission in this processing example will be explained. The RTCP transmission and receiving unit 107 receives a retransmission request packet (S115).
The RTCP transmission and receiving unit 107, which has received the retransmission request packet, acquires a sequence number of a packet for which retransmission is requested, from the retransmission request packet. Then, the RTCP transmission and receiving unit 107 notifies the retransmission control unit 108 of the acquired sequence number together with a retransmission request (S116).
The retransmission control unit 108 retrieves an RTP packet corresponding to the notified sequence number from the buffer for retransmission 105 (S117). The retransmission control unit 108 transmits the RTP packet with PTin the header replaced to the RTP transmission unit 103 as a retransmission packet (S118).
The retransmission control unit 108 notifies the band control unit 106 of a size of the retransmission packet simultaneously with transmitting the retransmission packet to the RTP transmission unit 103 (S119).
The band control unit 106 has a retransmission counter (not shown) that records the number of times retransmission is performed. The band control unit 106 increments this retransmission counter by one every time retransmission of media data is performed. Then, the band control unit 106 notifies the FEC generating unit 104 to change an insertion interval of the FEC packet based on a value of this retransmission counter (S120).
The notification of an insertion interval of the FEC packet is performed, for example, as described below. When a value of the retransmission counter is larger than 0, the band control unit 106 notifies the FEC generating unit 104 to transmit the FEC packet at the insertion interval twice as large as an initial value. Then, after transmitting the FEC packet at the interval twice as large as the initial value, the band control unit 106 decrements the retransmission counter by one. If the retransmission counter changes to 0 as a result of decrementing a value of the retransmission counter, the band control unit 106 notifies the FEC generating unit 104 to return the insertion interval of the FEC packet to the initial value.
In addition, the band control unit 106 has a threshold value counter (not shown) indicating upper limits of bands to be used by the FEC packet and the retransmission packet. This threshold value counter compares an upper limit value of a bandwidth, which indicates how many bits of the FEC packet and the retransmission packet can be sent per one second, with the current bands. A value of the threshold value counter is reset to the initial value at every predetermined interval (e.g., every one second).
In this processing example, in order not to change the entire transmission band, a band used by the FEC determined by the band control unit 106 is set as an initial value of the threshold value counter.
The band control unit 106 decrements a numerical value corresponding to a transmission size from the threshold value counter every time the FEC packet and the retransmission packet are sent. If the threshold value counter changes to 0 as a result of this decrementing, the band control unit 106 notifies the FEC generating unit 104 and the retransmission control unit 108 to stop the transmission of the FEC packet and the retransmission packet (S121, S114) Upon receiving the notification to stop the transmission, if the retransmission counter is larger than 0, the retransmission control unit 108 resets the value to zero.
After issuing an instruction to stop the transmission or after a predetermined time interval has elapsed, the band control unit 106 resets a value of the threshold value counter (return a value of the threshold value to the initial value). After it is confirmed that the threshold value counter has been reset, the FEC generating unit 104 transmits again the FEC packet with its insertion interval returned to the initial value (S111).
Effects of this Embodiment
According to the method and apparatus for transmitting media data in accordance with this embodiment, it is possible to perform error correction for media data efficiently without affecting other communication traffics when congestion has occurred on the best-effort Internet (IP/UDP) or a radio transmission line.
In addition, according to the transmission apparatus and the transmission method in accordance with this embodiment, bands in a transmission line can be utilized effectively.
Further, according to the transmission apparatus and the transmission method in accordance with this embodiment, compensation for media data can be performed effectively while suppressing inconveniences such as congestion caused by data compensating for a loss of transmitted media data.
Number | Date | Country | Kind |
---|---|---|---|
2004-004453 | Jan 2004 | JP | national |