Patent Application
20070071026
The present invention relates generally to a system for allowing devices connected to a network to transmit and receive compressed video data packets without impairment on the network.
Ethernet and packet-switched Internet Protocol (IP) networks are systems for transmitting data between different points. These systems are known as “contention-based” systems. That is, all transmitters contend for network resources. In such a system, multiple transmitters may transmit packets at such a time that the packets arrive at a network port simultaneously. When this happens, the network resources may become oversubscribed, resulting in lost or delayed data, and network impairment.
A conventional network comprises endpoints, such as computers, connected through a Local Area Network (LAN) or Wide Area Network (WAN). Packets are sent across the network from one endpoint to another, over packet-switching devices such as LAN switches or WAN routers. For example, in
Interactive video, interactive voice, and other real-time applications require delivery of data, accurately, the first time. Real-time video is comprised of a sequence of individual images, or video frames, which are sent in compressed form over the network and displayed in rapid succession by the receiver as they arrive. With real-time video or real-time voice applications, a receiver does not have to wait until a large file is downloaded before seeing the video or hearing the sound. Instead, the media is sent in a continuous stream and is played as it arrives. The receiver needs a player which is a special program that decompresses and sends video data to the display and audio data to the speakers. Real-time video is usually sent from pre-recorded video files, but can also be distributed as part of a live broadcast ‘feed’.
The DV compression standards are common formats used in real-time media applications. DV formats define digital video reduction and compression, used by applications that require real-time video capability over a network. In the DV formats, the video is divided into individual video frames and then compressed using a Discrete Cosine Transform. DV uses intraframe compression, meaning each compressed video frame depends entirely on itself, and not on any data from preceding or following video frames. However, it also uses adaptive interfield compression; if the compressor detects little difference between the two interlaced fields of a video frame, it will compress them together, freeing up some of the “bit budget” to allow for higher overall quality.
For real-time video applications using DV compressed video, high network traffic can cause connection, download and playback problems. When the player encounters high traffic, it degrades quality in an attempt to maintain continuous playback. For these applications to operate well, even the speed of light causes undesired delay. Thus, for DV real-time video applications, it is often not feasible use TCP or any method involving a retransmission delay.
The problem, therefore, is determining how to provide reliable, first-time delivery on a contention-based network. Various approaches have been tried. The most commonly proposed system relies on prioritization of data in the network. With this approach, data having real-time constraints is identified with priority coding so that it may be transmitted before other data.
Prioritization seems at first to be a good solution. However, on reflection it suffers from the same difficulty. Prioritization only provides a delivery advantage relative to the lower-priority data. It provides no advantage against the other priority data. Analysis and testing shows that this approach can work in certain circumstances, but only when the amount of priority data is small. When using prioritization with high-volume transmissions, such as real-time video applications, the amount of priority data is very high. Further, multiple video flows transmitting data at the same high priority level will contend with one another. Thus, in real-time video applications, prioritization will likely fail to prevent contention and packet loss.
Further, some networks and devices cannot support multiple priority levels for data packets. For example, some packet switches may support only one level of packet priority (i.e., two queues: one for prioritized packets and another for non-prioritized packets), making such a scheme difficult to implement.
Another approach is to multiplex the data. With this method the bursts of data associated with one flow of data are separated from the burst of another. Multiplexing usually uses some type of time-domain system (known as Time Domain Multiplexing (TDM)) to separate flows. TDM organizes the network so that specific time slots are assigned to individual flows. In other words, each potential transmitter on the network is guaranteed a slot of time to transmit, even if the transmitter is unlikely to use its assigned slot. These frequently unused time slots make TDM inefficient compared to Ethernet and IP networks.
Asynchronous Transfer Mode (ATM) is another technology for multiplexing a data network, to reduce contention. ATM breaks all data flows into equal length data blocks. Further, ATM can limit the number of data blocks available to any flow or application. The result is a virtual TDM multiplex system. ATM also has a limited address space, and thus is not as scalable to large networks as is Ethernet and IP.
Both TDM and ATM provide contention reduction, but at the cost of considerable added complexity, cost, components, and lost bandwidth performance. Other approaches rely on specialized hardware to schedule packet delivery, driving up hardware costs.
The invention provides a method for transmitting compressed video packets in an Ethernet or IP packet network by scheduling them for delivery based on communications between the transmitting node and the receiving node, which are evaluated to determine a preferred delivery schedule.
In one variation, a transmitting node transmits a query to the intended receiving node, indicating the intent to transmit compressed video. The receiving node responds with a reception map indicating what transmission time slots have already been allocated by other transmitting nodes (or, alternatively, what transmission time slots are available). The transmitting node then proposes a transmission map to the receiving node, taking into account any time slots previously allocated to other transmitting nodes. The receiving node either accepts the proposed transmission map or proposes an alternate transmission map. Upon agreement between the nodes, the transmitting node begins transmitting the compressed video according to the proposed transmission map, and the receiving node incorporates the proposed transmission map into its allocation tables. Because the proposed delivery schedule has been agreed to between the two endpoints, uncoordinated contention that might otherwise overflow network switches near the endpoints is avoided. Because the schedule is determined by the two endpoints, no network arbiter is needed to coordinate among network resources.
In another variation, a transmitting node transmits the bandwidth requirement of the compressed video transmission to the intended recipient node. The intended recipient node, after evaluating time slots previously allocated to other transmitters, responds with a proposed delivery schedule indicating time slots during which the transmitter should transmit the compressed video packets in order to avoid contention with other previously scheduled packets while maintaining the necessary bandwidth for the transmitter. The transmitter thereafter transmits packets according to the proposed delivery schedule.
In another variation, a transmitting node transmits a proposed delivery schedule to an intended recipient, indicating time slots corresponding to times during which it proposes to transmit compressed video packets. The intended recipient either agrees to the proposed delivery schedule, or proposes an alternate delivery schedule that takes into account the transmitter's bandwidth requirements. Upon agreement between the nodes, transmission occurs according to the agreed-upon delivery schedule. The schedule can be released at the end of the transmission.
In yet another variation, a transmitting node having the need to transmit compressed video packets according to a known data rate transmits a series of test packets over the network to the intended receiver using different delivery times. The test packets are evaluated to determine which of the delivery times suffered the least latency and/or packet loss, and that delivery time is used to schedule the packets for the duration of the transmission. Other endpoints use a similar scheme, such that each endpoint is able to evaluate which delivery schedule is best suited for transmitting packets with the least likely packet loss and latency. Different priority levels are used to transmit the compressed video data; the test packets; and other data in the network.
Turning briefly to
Depending on the packet size and underlying network bandwidth, some varying fraction of each time slot would be actually used to transmit a packet. Assuming a packet size of 125 bytes (1,000 bits) and a 10BaseT Ethernet operating at 10 mbps, a single 100-microsecond time slot would be used to transmit each packet. Assuming a packet size of 1,500 bytes, twelve of the 100-microsecond intervals would be consumed by each packet transmission.
According to one variation of the invention, the scheduled delivery scheme applies to prioritized packets in the network; other non-prioritized packets are not included in this scheme. Therefore, in a system that supports only priority traffic and non-priority traffic, the scheduled delivery scheme would be applied to all priority traffic, and ad-hoc network traffic would continue to be delivered on a nonpriority basis. In other words, all priority traffic would be delivered before any nonpriority traffic is delivered.
The delivery schedule of
The methods taught by the invention may apply to real-time video compressed with DV compression standards. This standard, which includes the DV25, DV50, DV100, MPEG, and H260 variants, etc., is a common format for digital video reduction and compression. Under the DV standards, before video is transmitted over the network to the receiver, the video is divided into video frames and then compressed using a Discrete Cosine Transform. The DV standards use intraframe compression, meaning each compressed video frame depends entirely on itself, and not on any data from preceding or following video frames. However, they also use adaptive interfield compression. That is, if the DV compressor detects little difference between the two interlaced fields of a video frame, it will compress them together, freeing up some of the “bit budget” to allow for higher overall quality.
One example of a DV standard is the DV25. Under the DV25 format, video information is carried in a nominal 25 megabit per second (Mbits/sec) data stream. After adding in audio, subcode (including timecode), Insert and Track Information (ITI), and error correction, the total data stream transmits about 30 Mbits/sec.
Suppose that a transmitting node needs to support a real-time video connection over the network. For a single real-time video connection transmitting DV25 compressed video, a bandwidth of 30 Mbits/second might be needed. Assuming a packet size of 2048 bytes, or 16,384 bits, this would mean that approximately 1,800 packets per second must be transmitted, which works out to (on average) two packets every millisecond. In the example of
Returning to
In one embodiment, a reception map (see
In step 302, the intended receiving node responds with a reception map such as that shown in
In step 303, the transmitter sends a proposed transmission map to the intended receiving node. The proposed transmission map preferably takes into account the allocated time slots received from the intended receiving node, so that previously allocated time slots are avoided. The transmitter allocates enough time slots to support the required bandwidth of the transmission while avoiding previously allocated time slots.
In step 304, the intended recipient reviews the proposed transmission map and agrees to it, or proposes an alternate transmission map. For example, if the intended recipient had allocated some of the proposed time slots to another transmitter during the time that the transmitter was negotiating for bandwidth, the newly proposed delivery schedule might present a conflict. In that situation, the intended recipient might propose an alternate map that maintained the bandwidth requirements of the transmitter.
In step 305, the transmitter repeatedly transmits packets, or bursts of packets, to the intended recipient according to the agreed delivery schedule, each packet containing a compressed video frame or video frame portion. To support a real-time video connection, for example, the transmitter could transmit six 80-byte packets every 10 milliseconds. For a higher definition real-time video connection, the transmitter could transmit at a more frequent rate. Finally, in step 306 the receiver's map is deallocated when the transmitter no longer needs to transmit.
Note that for two-way communication, two separate connections can be established: one for node A transmitting to node B, and another connection for node B transmitting to node A. Although the inventive principles will be described with respect to a one-way transmission, it should be understood that the same steps would be repeated at the other endpoint where a two-way connection is desired.
In step 403, the transmitter agrees to the proposed transmission map, causing the intended receiver to “lock in” the agreed time slots (this step could be omitted), and in step 404 the transmitter transmits packets according to the agreed-upon schedule. Finally, in step 405 the transmission map is deallocated upon termination of the connection.
In step 503, the transmitter transmits packets containing compressed video frames according to the agreed-upon delivery schedule, and in step 504 the transmission map is deallocated upon termination of the transmission.
In another variation, a transmitter may request bandwidth (e.g., one 1000-byte packet every 10 milliseconds) and the receiver responds with a placement message (e.g., start it at the 75th 100-microsecond slot). The receiver could also respond with multiple alternatives (e.g., start it at the 75th, the 111th, or the 376th time slot). The transmitter would respond with the time slot that it intended to use (e.g., the 111th), and begin transmission. This variation is intended to be within the scope of sending “transmission maps” and “reception maps” as those terms are used herein.
In step 602, as described in the above embodiments, a delivery schedule is partitioned into time slots according to a scheme such as that illustrated in
In step 604, a plurality of test packets are transmitted during different time slots at a rate needed to support the desired bandwidth. Each test packet is transmitted using a “discovery” level priority that is higher than that accorded to normal data packets (e.g., TCP packets) but lower than that assigned to real-time data traffic (to be discussed below). For example, turning briefly to
In step 605, the sender evaluates the test packets to determine which time slot or slots are most favorable for carrying out the connection. For example, if it is determined that packets transmitted using time slot #1 suffered a lower average dropped packet rate than the other slots, that slot would be preferred. Similarly, the time slot that resulted in the lowest packet latency (round-trip from the sender) could be preferred over other time slots that had higher latencies. The theory is that packet switches that are beginning to be stressed would have queues that are beginning to fill up, causing increases in latency and dropped packets. Accordingly, according to the inventive principles other time slots could be used to avoid transmitting packets during periods that are likely to increase queue lengths in those switches. In one variation, the time slots can be “overstressed” to stretch the system a bit. For example, if only 80-byte packets are actually needed, 160-byte packets could be transmitted during the test phase to represent an overloaded condition. The overloaded condition might reveal bottlenecks where the normal 80-byte packets might not.
Rather than the recipient sending back time-stamped packets, the recipient could instead perform statistics on collected test packets and send back a report identifying the latencies and dropped packet rates associated with each time slot.
As explained above, packet header overhead has been ignored but would typically need to be included in the evaluation process (i.e., 80-byte packets would increase by the size of the packet header). Slot selection for the test packets could be determined randomly (i.e., a random selection of time slots could be selected for the test packets), or they could be determined based on previously used time slots. For example, if a transmitting node is already transmitting on time slot 3, it would know in advance that such a time slot might not be a desirable choice for a second connection. As another example, if the transmitting node is already transmitting on time slot 3, the test packets could be transmitted in a time slot that is furthest away from time slot 3, in order to spread out as much as possible the packet distribution.
In step 606, a connection is established between the two endpoints and packets are transmitted using the higher “real-time” priority level and using the slot or slots that were determined to be more favorable for transmission. Because the higher priority level is used, the connections are not affected by test packets transmitted across the network, which are at a lower priority level. In one variation, the IP precedence field in IP packet headers can be used to establish the different priority levels.
It should be appreciated that rather than transmitting test packets simultaneously during different time slots, a single slot can be tested, then another slot, and so on, until an appropriate slot is found for transmission. This would increase the time required to establish a connection. Also, as described above, for a two-way connection, both endpoints would carry out the steps to establish the connection.
In another variation, packet latencies and dropped packet rates can be monitored during a connection between endpoints and, based on detecting a downward trend in either parameter, additional test packets can be transmitted to find a better time slot in which to move the connection.
As shown in
The clock pulses may comprise a pulse according to an agreed-upon interval (e.g., one second) that is used by each node to generate time slots that are synchronized to the beginning of the pulses. Alternatively, the clock source may generate a high-frequency signal that is then divided down into time slots by each node. Other approaches are of course possible.
As another alternative, each clock source may be synchronized to a common reference signal, such as a radio signal transmitted by the U.S. Government. As shown in
Another way or means of synchronizing time slots and delivery schedules among the nodes is to have one node periodically transmit (e.g., via multicast) a synchronization packet on the node on the network. Each node would receive the packet and use it to synchronize an internal clock for reference purposes. As an alternative to the multicast approach, one network node can be configured to individually send synchronization packets to each participating network node, taking into account the stagger delay involved in such transmission. For example, a synchronization node would transmit a synchronization packet to a first node on the network, then send the same packet to a second node on the network, which would be received later by the second node. The difference in time could be quantified and used to correct back to a common reference point. Other approaches are of course possible, and any of these means for synchronizing may be used independently of the others.
It should be understood that the phase of all frames may be independent from one another; they need only be derived from a common clock. Different endpoints need not have frames synchronized with each other. In other words, each time interval need not be uniquely identified among different endpoints, as long as the time intervals remain in relative synchronicity. This principle is shown with reference to
As shown in
In short, when NCD B determines that test packet X was received with minimal delay, it informs NCD A that the test packet identified as “packet X” was empirically favorable for future transmissions. Thus, NCD A identifies the relevant time interval as interval 1, whereas NCD B identifies the relevant time interval as interval 4. Similarly, NCD A identifies the relevant time interval for packet Y as interval 3, whereas NCD B identifies the relevant time interval for packet Y as interval 6. As long as the timeline at the top of
The invention will also work with “early discard” settings in router queues since the empirical method would detect that a discard condition is approaching.
While the invention has been described with respect to specific examples including presently preferred modes of carrying out the invention, those skilled in the art will appreciate that there are numerous variations and permutations of the above described systems and techniques that fall within the spirit and scope of the invention as set forth in the appended claims. Any of the method steps described herein can be implemented in computer software and stored on computer-readable medium for execution in a general-purpose or special-purpose computer, and such computer-readable media is included within the scope of the intended invention. The invention extends to not only the method but also to computer nodes programmed to carry out the inventive principles. Numbering associated with process steps in the claims is for convenience only and should not be read to imply any particular ordering or sequence.
The application is related in subject matter to U.S. patent application Ser. No. 10/663,378, filed Sep. 17, 2003 and titled “Empirical Scheduling of Network Packets,” and U.S. patent application Ser. No. 10/679,103, filed Oct. 31, 2003 and titled “Endpoint Packet Scheduling System.”
