The invention disclosed herein relates to a system and method for delivering data streams across a data communications network.
Data communication networks often respond to congestion by arbitrarily dropping data. The discarding of data without consideration as to the importance of the data can have severe negative consequences for certain types of data transmissions.
For example, video data generally comprises key frames and predictive frames, where the key frames, since they provide the basis for the predictive frames, are more important than predictive frames when the video data is processed for display to a user. Consequently, where video data streams are transmitted through a network suffering congestion, the arbitrary dropping of data can result in the loss of key frames which would severely degrade the quality of the video display generated from the received video data stream.
The invention provides a method, system, and computer program product for delivering a data stream of multiple data types through a network so that, when congestion in the network causes data to be dropped, data of the stream of a data type that is of less importance is dropped before data of the stream of a data type that is of a greater importance. In accordance with the invention, a data stream to be delivered over a network includes data of a primary type and one or more secondary types. Primary type data is routed over the network at a first priority level and secondary type data is routed at one or more priority levels lower than the first priority level such that, when congestion in the network requires that data be discarded, secondary type data is discarded before primary type data. Then, primary and secondary type data routed over the network is combined to provide the data stream.
In an embodiment of the invention, the data stream is transmitted with data of the primary type being transmitted separately from data of the one or more secondary types. Then, data of the primary and the one or more secondary types routed over the network is combined to provide the data stream, the provided data stream being at least a representation of the transmitted data stream, i.e., the data stream if not data is discarded or a representation thereof if data is discarded.
In the preferred embodiment, the data stream is transmitted to a router, which routes primary type data at a first priority and secondary type data at one or more priority levels lower than the first priority level. According to an embodiment of the invention, where the data stream comprises a video stream, I frames of the video stream are transmitted in association with a first designation and P and B frames of the video stream are transmitted in association with one or more designations other than the first designation.
According to another embodiment of the invention where the data stream comprises a video stream, I frames of the video stream are transmitted, e.g., to the router in association with a first TCP/UDP port number, and P and B frames of the video stream are transmitted, e.g., to the router, in association with one or more TCP/UCP port numbers other than the first port number. Data, e.g., received at the router, associated with a first port number is routed through a network at a first priority level and data, e.g., received at the router, in association with one or more port numbers other than the first port number is routed through the network at one or more priority levels lower than the first priority level such that, when congestion in the network requires that data received, e.g., at the router, be discarded, data associated with the one or more port numbers other than the first port number is discarded before data associated with the first port number. Then, data routed over the network, e.g., from the router, associated with the first port number is combined with data received through the network from the router associated with the one or more port numbers other than the first port number to provide a representation of the transmitted video stream.
The invention is illustrated in the figures of the accompanying drawings which are meant to be exemplary and not limiting, in which like references are intended to refer to like or corresponding parts, and in which:
Preferred embodiments of a method, system, and article of manufacture containing software programs in accordance with the present invention are described with reference to the drawings.
As shown in
In an embodiment of the invention, Server 100 comprises a Data Stream (“DS”) Provider System 110 and a Data Stream Transmitter (“DST”) System 120. The DS Provider System 110 may comprise any computer system capable of providing data streams as described above. For example, where the data stream to be provided comprises a video stream, DS Provider System 110 may comprise any known computer system for providing a video stream. Moreover, DS Provider System 110 may provide the video stream in any number of ways, such as, for example, based on live video data or previously recorded video files. In an embodiment of the invention, DS Provider System 110 comprises computer code for performing the function of providing the data stream as described above.
The DST System 120 may comprise any computer system capable of (a) separating the different types of data of the data stream and (b) transmitting the data of the primary type in association with a first designation and transmitting the data of the one or more secondary types in association with one or more designations other than the first designation. For example, where the data stream to be transmitted is a video stream, DST System 120 may comprise a computer system capable of separating the I frames, P frames, and B frames of the video stream and transmitting the I frames in association with a first TCP/UDP port number and transmitting the P and B frames in association with one or more TCP/UDP port numbers other than the first port number. For instance, P and B frames may both be transmitted in association with a second port number or the P and B frames may be transmitted in association with second and third port numbers, respectively. In an embodiment of the invention, DST System 120 comprises computer code for performing the data stream separation and transmission as described above.
Client 200 receives and processes the data streams generated and transmitted by Server 100. As such, Client 200 may comprise any computer system capable of (a) receiving data related to a given data stream and associated with a fourth designation and data related to the given data stream and associated with one or more designations other than the fourth designation, and combining the received data related to the given data stream and associated with the fourth designation and the received data related to the given data stream and associated with the designations other than the fourth designation to provide a data stream representative of the given data stream, and (b) processing the representative data stream.
In an embodiment of the invention, Client 200 comprises a Data Stream Receiving (“DSR”) System 210 and a Data Stream (“DS”) Processing System 220. DSR System 210 may comprise any computer system capable of performing the receiving operation described above. For example, where the data to be received and processed is video data, DSR System 210 may comprise any computer system capable of receiving I frames related to a given video stream and associated with a fourth port number and P and B frames related to the given video stream and associated with one or more port numbers other than the fourth port number (e.g., the P and B frames may both be received in association with a fifth port number or the P and B frames may be received in association with a fifth and sixth port number, respectively), and combining the received I, P, and B frames related to the given video stream to provide a video stream representative of the given video stream (e.g., where the representative stream may have fewer frames than the originally transmitted stream due to data loss from congestion, but where the frames remaining in the representative stream have identical counterparts in the originally transmitted stream). In an embodiment of the invention, DSR System 210 comprises computer code for performing the receiving and combining operations described above.
DS Processing System 220 may comprise any computer system capable of processing a data stream as described above. For example, where the representative stream is a video stream, DS Processing System 220 may comprise any known computer system capable of processing a video stream for display to a user. In an embodiment of the invention, DS Processing System 220 comprises computer code for performing the processing described above.
The term “computer system” here is used broadly and encompasses computer hardware and computer software or computer software only. For example, DS Provider System 110 and DST 120 may each comprise computer code that resides on the same computer hardware of Server 100. Alternatively, DS Provider System 110 and DST 120 may each be implemented in distinct computer hardware.
It should be noted that client and server functionality may be combined in a single system. In an embodiment of the invention, shown in
Returning to
Data may be routed over the any or all of Networks 300, 400 and 500 by any suitable hardware and/or software capable of performing the routing functions disclosed herein, and data may be discarded as disclosed herein by any hardware and/or software capable of performing the discarding functions disclosed herein. For example, such functions may be performed by Routers 600 and 700, which each may comprise any router capable of servicing data associated with a plurality of designations according to a plurality of priority levels such that, when congestion requires that data within the router be discarded, data associated with designations to be serviced according to a lower priority level is discarded before data associated with designations to be serviced according to a higher priority level. For example, Routers 600 and 700 each may comprise a Quality of Service (“QoS”) capable router capable of classifying received data associated with a plurality of port numbers into a plurality of levels of service such that, when congestion requires that data within the QoS router be discarded, data associated with port numbers classified into a lower level of service is discarded before data associated with port numbers classified into a higher level of service.
In an embodiment of the invention, Server 100, Client 200, and Routers 600 and 700, all are statically configured, e.g., through computer hardware or computer code, to transmit, receive, and service data, respectively, based on the same designations. For example, where the data steam at issue is a video stream, Server 100 may be configured to transmit the I frames of the video stream in association with a first destination port number (e.g., destination port number 5151) and transmit the P and B frames of video stream in association with one or more destination port numbers other than the first destination port number (e.g., P and B frames both in association with destination port number 5153 or P and B frames in association with destination port numbers 5153 and 5155, respectively). Also, continuing with the example, Client 200 may be configured to receive I frames of video streams on a fourth destination port number that is the same as the first destination port number (e.g., destination port number 5151) and the P and B frames for video streams on one or more destination port numbers other than the fourth destination port number that are the same as the one or more destination port numbers other than the first destination port number (e.g., P and B frames both in association with destination port number 5153 or P and B frames in association with destination port numbers 5153 and 5155, respectively).
Also, referring to the same example, Routers 600 and 700 may be configured to service data received in association with a first destination port number (e.g., destination port number 5151) according to a first priority level and one or more destination port numbers other than the first destination port number (e.g., destination port number 5153 or destination port numbers 5153 and 5155) according to a one or more priority levels lower than the first priority level such that, when congestion requires that data within the router be discarded, the data associated with the destination port numbers to be serviced according to a lower priority level is discarded before data associated with the destination port numbers to be serviced according to a higher priority level.
For instance, where Server 100 and Client 200 are configured to transmit and receive, respectively, I frames in association with a first destination port number (e.g., 5151) and P and B frames both in association with a single destination port number other than the first destination port number (e.g., destination port number 5153), then Routers 600 and 700 may be configured to service data received in association with the first destination port number according to a first priority level and service data received in association with the other port number according to a second priority level lower than the first priority level such that, when congestion requires that data within the router be discarded, the data associated with the other destination port number (e.g., P and B frames associated with destination port number 5153) is discarded before the data associated with the first destination number (e.g., I frames associated with destination port number 5151).
Alternatively, where Server 100 and Client 200 are configured to transmit and receive, respectively, I frames in association with a first destination port number (e.g., 5151), P frames in association with a second destination port number (e.g., 5153), and B frames in association with a third destination port number (e.g., 5155), then Routers 600 and 700 may be configured to service data received in association with the first, second, and third destination port numbers according to first, second, and third priority levels, respectively, such that, when congestion requires that data within the router be discarded, the data associated with the third destination port number (e.g., B frames at port number 5155) is discarded before data associated with the first and second destination port numbers (e.g., I frames at port number 5151 and P frames at port number 5153, respectively), and data associated with the second destination port number is discarded before data associated with the first destination port number.
In another embodiment of the invention, Server 100, Client 200, and Routers 600 and 700 may transmit, receive, and service data, respectively, based on various user definable designations. In this embodiment, shown in
Although Control System 800 is shown in
Next, the data of the provided data stream is separated according to the different data types, as represented in block 1100. For instance, where the provided data stream is a video stream, DST System 120 of Server 100 may perform the operations represented in block 1100 by, for example, separating the I, P, and B frames from the provided video stream.
The separated data of the data stream is then transmitted such that data of the primary type is transmitted in association with a first designation and data of the one or more secondary types is transmitted in association with one or more designations other than the first designation, as represented in block 1200. For instance, where the data stream is a video stream and it is desired to have I frames transmitted with high priority and P and B frames transmitted with the same priority lower than high priority, DST System 120 of Server 100 may perform the operations represented in block 1200 by, for example, transmitting I frames of the video stream in association with a first destination port number (e.g., 5151) and both P and B frames of the video stream in association with a destination port number other than the first destination port number (e.g., 5153). Alternatively, where the data stream is a video stream and it is desired to have I frames transmitted with high priority, P frames transmitted with medium priority, and B frames transmitted with lower priority, then DST System 120 of Server 100 may performed the operations represented in block 1200 by, for example, transmitting I, P, and B frames of the video stream in association with a first (e.g., 5151), a second (e.g., 5153), and a third (e.g., 5155) destination port number.
The transmitted data of the data stream is received at Router 600 which, as represented in block 1300, routes the transmitted data of the data stream associated with the first designation according to a first priority level and the transmitted data of the data stream associated with the one or more designations other than the first designation according to one or more priority levels lower than the first priority level such that, when congestion in Third Network 500 requires that data within the router be discarded, the data associated with the designations to be routed according to a lower priority level is discarded before data associated with the designations to be routed according to a higher priority level. For example, where the data stream is a video stream, Router 600 may be configured to route data associated with a first destination port number (e.g., I frames at port number 5151) according to a first priority level and data associated with a second destination port number (e.g., P and B frames both at port number 5153) according to a lower priority level, such that, when congestion in Third Network 500 requires that data within the router be discarded, the data associated with the second destination port number (e.g., P and B frames at port number 5153) is discarded before data associated with the first destination port number (e.g., I frames at port number 5151). In another example, Router 600 may be configured to route data associated with a first, a second, and a third destination port number (e.g., I frames at port number 5151, P frames at port number 5153, and B frames at port number 5155) according to a first, a second, and a third priority level, respectively, such that, when congestion in Third Network 500 requires that data within the router be discarded, the data associated the third destination port number (e.g., B frames at port number 5155) is discarded before data associated with the first and second destination port numbers (e.g., I frames at port number 5151 and P frames at port number 5153, respectively), and data associated with the second destination port number is discarded before data associated with the first destination port number.
Data of the data stream associated with the first designation and the one or more designations other than the first designation that is not discarded by Router 600 due to congestion in Third Network 500, is routed through Third Network 500 and received at Router 700, which then routes the data of the data stream in a similar manner as described above with respect to Second Network 400. For example, Router 700 routes data of the data stream received by it from Third Network 500 such that data of the data stream associated with the first designation is routed according to a first priority level and data of the data stream associated with the one or more designations other than the first designation is routed according to one or more priority levels lower than the first priority level such that, when congestion in Second Network 400 requires that data within the router be discarded, the data associated with the designations to be routed according to a lower priority level is discarded before data associated with the designations to be routed according to a higher priority level.
Returning to
The data of the data stream that was received separately in association with the first designation and the one or more designations other than the first designation are then combined to provide a representation of the data stream, as represented in block 1500. For example, where the data stream comprises a video stream, the operations represented in block 1500 may comprise DSR 210 of Client 200 combining the I frames received on the first destination port number (e.g., 5151) and related to a given video stream (e.g., data from a given IP address and having a given source port number) with P and B frames received on one or more destination port numbers other than the first port number (e.g., P and B frames both on 5153 or P and B frames on 5153 and 5155, respectively) and related to the same video stream to provide a representation of the given video stream, as previously described.
Finally, the representation of the data stream is processed for presentation to a user, as represented in block 1600. For example, where the representation of the data stream comprises a video stream, DS Processing System 220 of Client 200 may process the representative video stream to present to the user operating Client 200 the video encoded within the representative video stream.
In the embodiment of the invention described above, Server 100 transmits data to Client 200 and Routers 600 and 700 route data according to two or more priority levels with respect to congestion in Third Network 500 and Second Network 400, respectively. In the embodiment of the invention shown in
The operations represented in blocks 2000, 2100, and 2200 maybe accomplished by, for example, Control System 800 obtaining a policy (e.g., by retrieving the policy from a policy server) providing that data of data streams of the primary type and the one or more secondary types be transmitted, received, and routed, as described above. Then Control System 800 may communicate with each of Server 100 and Client 200 (or End User Systems 151 and 152) and Routers 600 and 700 to configure each of them to transmit, receive, and route, respectively, using the designations and corresponding priority levels provided for in the policy.
Next, as represented in block 2300, data streams are delivered from Server 100 (or one of the End User Systems) to Client 200 (or another End User System) through Routers 600 and 700 in accordance with the policy. For example, data streams may be delivered from Server 100 (or one of the End User Systems) to Client 200 (or another End User System) through Routers 600 and.700 in the same manner as described previously in connection with
While the invention has been described and illustrated in connection with preferred embodiments, many variations and modifications as will be evident to those skilled in this art may be made without departing from the spirit and scope of the invention, and the invention is thus not to be limited to the precise details of methodology or construction set forth above as such variations and modification are intended to be included within the scope of the invention.
This application claims priority from U.S. Provisional Application No. 60/341,817 filed Dec. 15, 2001, which is hereby incorporated by reference into this application.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US02/39918 | 12/12/2002 | WO |
Number | Date | Country | |
---|---|---|---|
60341817 | Dec 2001 | US |