Patent Application
20090135724
The real-time transport protocol (RTP) provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. The data transport is augmented by a real-time control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers.
An example embodiment of the present invention may be implemented in the form of a method or corresponding apparatus which provides end-to-end reception quality feedback between a sending end system and a receiving end system. The method and corresponding apparatus according to one embodiment of the present invention includes: (i) tracking changes to real time transport protocol (RTP) packets of the RTP session caused by media processing of the RTP packets to produce tracked changes; (ii) modifying RTP packet information of the RTP packets based on the tracked changes; (iii) correcting RTP control protocol (RTCP) packets corresponding to the RTP session based on the tracked changes to produce corrected RTCP packets reports, the corrected RTCP packets being a measure of the end-to-end reception quality of the RTP session; and (iv) reporting the end-to-end reception quality of the RTP session by forwarding the corrected RTCP packets.
The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention.
A description of example embodiments of the invention follows.
One measure of quality of a network is reception quality. Intuitively, if what was received by a receiver matches what was sent by a sender, then the reception quality of a network is “good.” Conversely, if what was received by a receiver differs from what was sent by a sender, for example, the receiver received fewer packets than were sent by the sender, the reception quality of a network is “poor.” However, in a network, such as the network 105 of
As an example, when a media processor or other RTP intermediate systems changes the RTP packet size of an RTP packet, such as to change an RTP packet carrying an echo into an RTP packet carrying comfort noise, a sender's byte count field in an RTCP sender report does not reflect the changes in packet size. In another example, when an RTP intermediate system changes the number of RTP packets transported, such as to remove an RTP packet carrying an echo as contrasted with carrying a voice, sequence numbers in RTP headers and a sender's packet count field in an RTCP sender report do not reflect changes in packets transported. If reception reports from a receiving end system are forwarded to a sending end system by the RTP intermediate system with the reception report's contents intact, that is unchanged, an inconsistency between the two end systems may cause the reception quality feedback in the reception report to be invalid.
One way to avoid the foregoing problem of reception quality feedback is for an RTP intermediate system to discard all reception reports from a receiving end system. This approach, however, makes no reception quality feedback available.
Another way is for an RTP intermediate system to generate reception reports based on reception by the RTP intermediate system itself. However, this approach is inadequate because the reception quality feedback is only available for either a link between a sending end system and the RTP intermediate system or a link between the RTP intermediate system and a receiving end system, but not between the sending end system and the receiving end system.
In yet another way, one that addresses the aforementioned inadequacies and reflects changes caused by media processing, a reception quality feedback technique may: (1) track changes to packets caused by media processing of the packets; (2) modify packet information of the packets based on the tracked changes; (3) corrects report packets based on the tracked changes; and (4) report the end-to-end reception quality by forwarding the corrected report packets. The corrected packets of this reception quality feedback technique may be considered a valid a measure of end-to-end reception quality.
One of ordinary skill in the art will readily recognize that the foregoing reception quality feedback technique and example embodiments thereof may be employed by an intermediate system, such as the media processor 110. Alternatively, the technique and example embodiments thereof may be employed by another intermediate system separate and distinct from the media processor 110. The particulars of the last technique and example embodiments thereof will now be described.
In TABLE 1, an embodiment tracks changes caused by media processing by updating both a send sequence number (sn_send) 230 and a total packet count of packets sent (tpcps) 235 by a number of packets sent to a receiver after media processing by a media processor.
In a convenient embodiment illustrated by TABLE 1, for a first packet 225a sent after media processing, the embodiment sets the send sequence number 230 to a sequence number of the first packet 225a, i.e., sn_send=sn_first. In some embodiments, it may be advantageous to store the sequence number of the first packet 225a. For each packet sent thereafter, after media processing, the embodiment increments the send sequence number 230 by one, i.e., sn_send=sn_send+1.
For example, for a second packet 225b, the send sequence number 230 is the next sequence number after the send sequence number 230. A third packet 225c, however, is not sent after media processing, but rather is dropped. The embodiment does not increment the send sequence number 230, i.e., sn_send=sn_send.
The above example illustrated in TABLE 1 highlights an important effect or result of media processing. A sequence number for a packet, as is known to the media processor 110 and the sender 120, differs from a sequence number for the packet after media processing, as is known to the media processor 110 and the receiver 130. As such, it may be said that there are two “streams” of sequence numbers: one stream for packets between the processor 110 and the sender 120 and another stream for packets between the media processor 110 and the receiver 130. There being two streams of sequence numbers has significant implications when measuring and reporting reception quality, as will be described later. In the interim, because media processing results in there being two streams of sequence numbers, sequence numbers for packets before media processing and sequence numbers for packets after media processing are not the same, but are rather corresponding.
In another convenient embodiment also illustrated by TABLE 1, for the first packet 225a, the embodiment sets a total packet count of packets (tpcps) 235 to one. For each packet sent thereafter, after media processing, the embodiment increments the total packet count of packets sent 235 by one, i.e., tpcps=tpcps+1.
For example, after sending the first packet 225a, sending the second packet 225b increases the total packet count of packets sent 235 by one. The third packet 225c, however, is not sent after media processing, but rather is dropped. The embodiment does not increment the total packet count of packets sent 235, i.e., tpcps=tpcps.
Additionally, the embodiment tracks changes caused by media processing by updating a total octet count of packets sent (tocps) 240 by a number of octets sent to a receiver after media processing by a media processor. For the first packet 225a, the embodiment sets the total octet count of packets sent 240 to the octet count of the first packet 225a. For each packet sent thereafter, after media processing, the embodiment increments the total octet count of packets sent 240 by the octet count of the respective sent packet, i.e., tocps=tocps+octet count of packet sent.
For example, after sending the first packet 225a, sending the second packet 225b increases the total octet count of packets sent 240 by the octet count of the second packet 225b. The third packet 225c, however, is not sent after media processing, but rather is dropped. The embodiment does not increment the total octet count of packets sent 240, i.e., tocps=tocps.
As described above, the embodiment reflects changes to packets caused by media processing, such as changes in a number and a size of packets sent to a receiver, by updating a send sequence number (sn_send), a total packet count of packets sent (tpcps), and a total octet count of packets sent (tocps). Subsequently, these updated values may be further used by the embodiment to modify packet information and to correct packets that are a measure of reception quality. In this way, the embodiment modifies packet information and corrects packets based on the changes caused by media processing.
Depending on the changes caused by media processing, the payload portion 340 of the packet 315 sent from the sender 320 and a payload portion 350 of the media processed packet 325 sent to the receiver 330 after media processing may be different. For example, media processing causes the size of a packet to change. In such an example, a payload portion of a packet sent from a sender differs from the payload portion after media processing.
While a receiver's understanding of what was received reflects changes caused by media processing, this understanding differs or is otherwise inconsistent with a sender's understanding of what was sent. Without correcting this inconsistency in understanding, a measure of end-to-end reception quality of a network in which packets are media processed and changed cannot be valid.
In a network in which packets are media processed and thus changed, a sender's understanding of what was sent and a receiver's understanding of what was received are different. Because this difference is not due to reception quality, or lack of, necessarily, a measure reception quality based on a sender report, such as the sender report 425, and a receiver report, such as the received report 435, is not entirely valid.
The embodiment corrects the sender's 420 understanding of what was sent by correcting the sender report 425. A resulting corrected sender report 440 reflects changes caused by media processing. As such, a measure of reception quality based on the corrected sender report 440 is valid. A difference between the corrected sender report 440 and the receiver's 430 understanding of what was received stems from reception quality and not from changes caused by media processing.
In a similar manner, the embodiment corrects the receiver's 430 understanding of what was received by correcting the receiver report 435. A resulting corrected receiver report 445 reflects changes caused by media processing. As such, a measure of reception quality based on the corrected receiver report 445 is valid. A difference between the corrected receiver report 445 and the sender's 420 understanding of what was sent is due to reception quality and not changes caused by media processing.
In a corrected RTCP sender report 465, a total packet count of packets sent to a receiver (tpcps) 470 and a total octet count of packets sent to a receiver (tocps) 475 replaces the sender's packet count 455 and the sender's octet count 460, respectively. It should be appreciated that the total number of packets sent by a sender (sender's packet count 455) and a total number of packets sent to a receiver (tpcps 470) are not necessarily the same because of media processing of packets. For the same reason, it should also be appreciated that the total number of octets sent by a sender (sender's octet count 460) and a total octet count of packets sent to a receiver (tocps) 475 are also not necessarily the same.
As described in reference to TABLE 1, the total packet count of packets sent 470 reflect changes caused by media processing, namely, a change in a number of packets sent to a receiver after media processing. The total octet count of packets sent 475 reflects a change in the size of packets sent to a receiver after media processing. As such, the corrected RTCP sender report 465 corrects an inconsistency caused by media processing between a sender's understanding of what was sent from the sender and what was sent to a receiver. Having considered or otherwise accounted for changes caused by media processing with the corrected RTCP sender report 465, a difference between a sender's understanding and what was sent to a receiver is not the result of media processing, but is rather a valid measure of reception quality.
In a corrected RTCP receiver report 490, an estimated extended highest sequence number received (est_ehsnr) 495 replaces the extended highest sequence number received 485. Because RTP packets may be discarded as a result of media processing, resulting in packet information of packets sent to a receiver being modified with updated sequence numbers, as described in reference to TABLE 1 and
It should be appreciated that the highest sequence number of a packet sent to a receiver and thus received by the receiver (ehsnr 485) and the highest sequence number of a packet sent from a sender are not the same necessarily because of media processing.
As such, the corrected RTCP receiver report 490 corrects an inconsistency caused by media processing between what was sent from a sender and a receiver's understanding of what was sent to the receiver. Having considered or otherwise accounted for changes caused by media processing with the corrected RTCP receiver report 490, a difference between what was sent from a sender and a receiver's understanding is not the result of media processing, but is rather a valid measure of reception quality.
While described within the context of the RTCP receiver report 480, one of ordinary skill in the art will recognize that the foregoing embodiment also applies to the RTCP sender report illustrated in
As alluded to in the above description, estimating an extended highest sequence number received provides meaning to a case in which RTP packets are discarded as a result of media processing resulting in packet information of packets sent to a receiver being modified with updated sequence numbers. However, because of media processing, an extended highest sequence number received cannot be estimated from a sequence number. Instead, the extended highest sequence number received is estimated by calculating a time when a last RTP packet sent from a sender was received (ts_lrtp) according to the following:
delay—rt=ts—rr−ts—sr−DLSR; and (1)
ts
—
lrtp=ts
—
rr−delay—rt−delay—mp (2)
where,
ts_rr denotes a timestamp from an RTCP receiver report record representing when the RTCP receiver report was received;
ts_sr denotes a timestamp from an RTCP sender report record representing when the RTCP sender report having the same synchronization source as the RTCP receiver report was received;
delay_mp denotes a mean delay caused by media processing; and
DLSR denotes a delay between receiving the last RTCP sender report and the sending an RTCP packet, e.g., a delay between a time a receiver receiving an RTCP sender report and the receiver sending an RTCP receiver report.
It is useful to note that the ts_rr and ts_sr denoting when the RTCP receiver report and the RTCP sender report were received, respectively, are not the same as a network time protocol (NTP) timestamp of an RTCP packet. The network time protocol (NTP) timestamp represents when the RTCP packet was sent, e.g., from a sender (i.e., an RTCP sender report) or from a receiver (i.e., an RTCP receiver report).
The estimated extended highest sequence number received is a sequence number of an RTP record received at the calculated time ts_lrtp. In this way, it may be said that the extended highest sequence number received is estimated from time measurements, namely, (i) a time when the RTCP receiver report was received (ts_rr), (ii) a time when the RTCP sender report was received (ts_sr), (iii) the mean delay caused by media processing; and (iv) the delay between receiving the last RTCP sender report and the sending an RTCP packet (e.g., the RTCP receiver report or the RTCP sender report).
The process 500 starts (501). The process 500 searches (505) RTCP sender report records to find those RTCP sender report records with the same synchronization source (SSRC) as a subject RTCP receiver report record which is to be corrected, i.e., ssrc_sr=ssrc_rr. In this way, RTCP packets of interest are limited to those packets belonging to the same RTP session or call.
The process 500 searches (510) the SSRC matching RTCP sender report records to find a subject RTCP sender report record with the same network time protocol (NTP) timestamp (ntp_sr) as the subject RTCP receiver report record (ntp_rr), i.e., ntp_sr=ntp_rr. This further limits the RTCP packets of interest found by the process 500 at (505) to just the subject RTCP packet sender report. As such, the NTP timestamp serves as a unique identifier identifying the subject RTCP packet receiver report and sender report.
The process 500 estimates (515) a round-trip transmission delay to and from a receiver (delay_rt) and the RTP intermediate system from the following time measurements: (i) a time when the RTP intermediate system received the subject RTCP receiver report (ts_rr), (ii) a time when the RTP intermediate system received the RTCP sender report record (ts_sr) (as found by the process 500 at (505)), and (iii) a delay between the receiver receiving the last RTCP sender report and the receiver sending the subject RTCP receiver report (DSLR), i.e., delay_rt=ts_rr−ts_sr−DSLR.
Because of media processing, the highest sequence number of RTP packets received by a receiving end-system (e.g., the receiver) up to a time when an RTCP packet (e.g., the RTCP receiver report) is generated, as reported in the RTCP packet as an extended highest sequence number received, has no significance or meaning to a transmitting end-system (e.g., the sender). Recall, however, RTP packets after media processing correspond to RTP packets before media processing. Accordingly, there may be an RTP packet corresponding (i.e., a corresponding RTP packet) to the RTP packet whose sequence number is reported as the extended highest sequence number in the RTCP packet. The sequence number of the corresponding RTP packet, unlike the highest sequence number reported in the RTCP packet, does have meaning to the transmitting end-system. Accordingly, the RTCP packet may be corrected (to account for media processing) by finding the sequence number of the corresponding RTP packet.
Continuing to refer to
The process 500 continues and searches (525) RTP records to find those RTP records (ssrc_rtp) with the same SSRC as the subject RTCP receiver report (ssrc_rr), i.e., ssrc_rtp=ssrc_rr.
The process 500 searches (530) the SSRC matching RTP records to find the last RTP record received at the time ts_lrtp. The process 500 sets an extended highest sequence number received (ehsnr) to the sequence number of the found RTP record (sn_rtp), i.e., ehsnr=sn_rtp.
The process 500 ends (536) with the extended highest sequence number estimated.
In a convenient embodiment, in an event an RTP packet is received by an RTP intermediate system, the process 500 stores (not shown) the following information: a synchronization source identifier identifying a source of the RTP packet (ssrc_rtp), a sequence number of the RTP packet (sn_rtp), and a timestamp representing when the RTP packet was received (ts_rtp). In an event an RTCP sender report is received, the process 500 stores (not shown) the following information: a synchronization source identifier identifying a source of the RTCP sender report (ssrc_sr), an NTP timestamp of the RTCP sender report (ntp_sr) representing when the RTCP sender report was sent, and a timestamp from the RTCP sender report record (ts_sr) representing when the RTCP sender report was received. In an event an RTCP receiver report is received, the process 500 stores (not shown) the following information: a synchronization source identifier identifying a source of the RTCP receiver report (ssrc_rr), a last sender report timestamp (LSR) representing when the last RTCP sender report was received, and a timestamp (ts_rr) representing when the RTCP receiver report was received.
Based on the tracked changes 707, the correcting unit 710 corrects (denoted by an arch with an arrowhead) an RTCP packet 711 resulting in a corrected RTCP packet 712, in accordance with example embodiments described. Also based on the tracked changes 707, the modifying unit 715 modifies (denoted by an arch with an arrowhead) an RTP packet 716, resulting in a modified RTP packet 717, in accordance with example embodiments described above. The reporting unit reports the end-to-end reception quality of the RTP session by forwarding the corrected RTCP packet 712.
In a convenient embodiment, the example apparatus 700 has an interface (not shown) to interface the apparatus 700 to an RTP network (not shown). The interface is in communication with the correcting unit 710 to receive the RTCP packet 711 from the RTP network and is in communication with the reporting unit 720 to forward the corrected RTCP packet 712 to the RTP network and thus report end-to-end reception quality. The interface is also in communication with the modifying unit 715 to receive the RTP packet 716 from the RTP network and to transmit the modified RTP packet 717 to the RTP network.
The estimating unit 830 estimates an extended highest sequence number 835, in accordance with example embodiments described above. The estimating unit 830 estimates the extended highest sequence number 835 from input 840. The input 840 includes: a time when an RTP intermediate system received the RTCP receiver report (ts_rr); (ii) a time when the RTP intermediate system received the RTCP sender report (ts_sr); (iii) an estimate of mean delay for media processing (delay_mp); and (iv) a delay between a receiver receiving the last RTCP sender report and the receiver sending the RTCP receiver report (DSLR).
In a convenient embodiment, the correcting unit 810 also includes a storing unit (not shown) to store: (i) in an RTP record, in an event an RTP packet is received, a synchronization identifier source identifying a source of the RTP packet (ssrc_rtp), a sequence number of the RTP packet (sn_rtp), and a timestamp representing when the RTP packet was received (ts_rtp); (ii) in an RTCP sender report record, in an event an RTCP sender report is received, a synchronization source identifying a source of the RTCP sender report (ssrc_sr), an NTP timestamp of the RTCP sender report (ntp_sr) representing when the RTCP sender report was sent, and a timestamp (ts_sr) representing when the RTCP sender report was received; and (iii) in an RTCP receiver report record, in an event an RTCP receiver report is received, a synchronization source identifying a source of the RTCP receiver report (ssrc_rr), a last sender report timestamp (LSR) representing when the last RTCP sender report was received, and a timestamp (ts_rr) representing when the RTCP receiver report was received.
While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.
It should be understood that the network, flow, and block diagrams may include more or fewer elements, be arranged differently, or be represented differently. It should be understood that implementation may dictate the network, flow, and block diagrams and the number of network, flow, and block diagrams illustrating the execution of embodiments of the invention.
It should be understood that elements of the network, flow, and block diagrams described above may be implemented in software, hardware, or firmware. In addition, the elements of the network, flow, and block diagrams described above may be combined or divided in any manner in software, hardware, or firmware. If implemented in software, the software may be written in any language that can support the embodiments disclosed herein. The software may be stored on any form of computer readable medium, such as random access memory (RAM), read only memory (ROM), compact disk read only memory (CD-ROM), and so forth. In operation, a general purpose or application specific processor loads and executes the software in a manner well understood in the art.
