Patent Grant
6954796
This invention relates to the transmission of real time data between a sender and receiver over a transmission network.
The current delivery of real-time data depends very much on the feed back channel from the receiver to the sender which provides a report on the received data transmission. Based on this feedback report, the sender will take appropriate action for the transmission of subsequent data to the receiver. The action taken by the sender may require reservation of more network resources for end-to-end data delivery and re-transmission of lost data. Re-transmission of lost data requires the receiver end to report specifically the amount of lost data encountered before the entire data can be of use to the applications at the receiver end. For a large system having a large number of servers and clients, the report messages between the terminals utilise a considerable amount of bandwidth.
In both connection and connectionless data networks, data packets at the lower layers such as the network layer differentiate data only by priority or time-to-live parameters. The distinction made between data packets at the network layer does not relate the coupling nature of the application layer data fragmented into data packets for network layer delivery. Some of these data packet parameters used to distinguish the various classes of service between data packets are listed in the IETF (Internet Engineering Task Force) RFC 791 on Internet Protocol version 4 and IETF's RFC 2460 on Internet Protocol version 6. In the current specifications for Differentiated Service as mentioned in IETF's RFC 2474, RFC 2475, RFC 2597 and RFC 2958, only a means to provide data packet marking is provided. IP data packets with special marking allow the network components such as routers or switches to treat the IP data packets based on the packet's different Quality of Service fields or DSCP, as an indicator on how data flow should be determined. The re-transmission and dropping of data packets belonging to a specific stream is performed at the IP packet level without any knowledge of the application data. Data packet differentiation is only performed at the packet level and selective dropping of a group of data packets is almost impossible if this process has to be performed at the network level.
Without the additional information from the application software, a data packet that makes it though the network may consist of only part of the data the application software needs. With the re-transmission of the missing data that finally arrives at the receiver end, a strict real-time requirement in most instances cannot be met. A data object that arrives not adhering to the real-time requirement is said to be of no presentation value because the time for decoding the piece of information has already passed. As a result, network resources are used in delivering real-time data of no value to the end user. In addition, the non-presentation value group of data packets may cause unnecessary network congestion at the already strained intermediate network nodes. In many instances the congestion of network traffic may be prolonged and in a worst case scenario the traffic congestion may spread to a wider area.
For most real-time data streaming across a packet data network, the decoding rate for the data is only made known to the end devices, for instance the server and the end client. In many instances, the server can only deliver data at the rate based on the decoding rate of the data and a report sent from the client giving statistics of the data received. In some implementations, a large data buffer or an arbitrary size of data buffer needs to be pre-allocated at the client side before the streaming of data from the server to the client can begin. In the scenario where a wide area network is constructed based on a number of intermediate network nodes such as routers and switches which have no information on the type of data and how the data should be treated for end-to-end delivery, the intermediate nodes have no real-time streaming knowledge of the network packets arriving and leaving the nodes. In the event of network congestion at one of the intermediate nodes there is no means to deliver or re-schedule the delivery of network data packets that makes sense from the view of the application data. In most cases only a partial object data is successfully delivered to the end terminals and re-transmission of lost data may be performed automatically by the network layer.
According to the invention in a first aspect, there is provided a method of sending a data object through a communications network from a sender to a recipient via at least one network node comprising the steps of:
According to the invention in a second aspect, there is provided a method of creating synchronised time stamped multimedia objects comprising the steps of:
According to the invention in a third aspect, there is provided a method for scheduling the transmission time of multimedia objects comprising the steps of:
According to the invention in a fourth aspect, there is provided an apparatus for interconnecting an object streaming apparatus into existing network interconnection apparatus to control data traffic generated by the multimedia object streams comprising:
The described embodiment of the invention uses a system architecture that allows end-to-end streaming of time based objects. An input data object from an object stream entering the network is marked and disassembled by an ingress object traffic controller.
In the ingress object traffic controller, a large data object is fragmented into a plurality of smaller network data packets as specified by the chosen network layer for delivery. Each network data packet is marked according to the precedence level desired by the application entities at the source of the object stream. To provide the network data packets with linkage and real-time information, each object has an object descriptor created. Based on the real-time requirement and the network resources allocated for the delivery of the object stream, a time for an object frame is computed and created for the delivery of time-stamped object data. The data information of the object descriptor is mapped to the header of the network data packet where possible or is encoded into the payload.
At each of the intermediate nodes such as routers and switches connected with the apparatus, the data packet is received. The data packet is re-routed to the host using an egress object traffic controller if the data object has reached the final destination, otherwise the packet is processed by an exchange object traffic controller for re-transmission to another network node. In the egress or exchange object traffic controller, the object descriptor for each stream is collected and a timer circuit is spawned for each object stream.
Based on the timing information, a system object frame is created dynamically and the number of time slots per object frame is computed. The time slots are used to mark the delivery time of the object data. The time schedule for the delivery of the object based on time slots is performed based on a traffic shaper or filter adopted for the stream. Object data meeting the criteria of the traffic shaper is put in the object stream queue and object data not meeting the requirement of the traffic shaper is stored in a cache memory for a generalised lower priority object data scheduling. An object decoding period is determined from the set of object stream descriptors received. One or more object decoding time slots are allotted for the object data to be delivered to meet real-time criteria in the stream object queue. After completing scheduling the object data that will expire within the object time frame in the stream object queue and data objects in the stream object cache memory are scheduled next. After the object has been scheduled, excess time slots are saved for future object frames. If there are insufficient time-slots, additional time-slots are allocated from the reserves to meet the need. If there are insufficient time-slots for delivery, object data that expires in the next object frame is discarded.
Data object packets scheduled for re-transmission are then re-assembled based on the output network criteria and a network header is mapped accordingly. The process is repeated for the data object in the exchange object traffic controllers of the intermediate nodes within the service provider domain until the final destination has been reached.
The described embodiment of the invention provides an apparatus and a method for the delivery of scalable multimedia objects to meet real-time applications on a heterogeneous network. These scalable multimedia objects may contain synchronised audio, video and data information for interactive, communicative and broadcast applications over a wide area network. The described embodiment provides a multimedia objects streaming method based in network resource availability at each network component and it allows real-time end to end delivery of multimedia objects with minimal effect on the presentation quality of the multimedia object over networks with dynamically changing network resource parameters such as varying bandwidth, delay and latency jitters. The described embodiment provides a method and an apparatus that allows fast high level application data context switching and routing at the network layer.
The described embodiment of the invention further allows the delivery of data without additional feedback from the receiver and thus cuts down on the complexity in the protocol needed by both ends of the users to handle a feedback report. The processing power needed for the client to perform the reporting based on the data received from the server is also reduced.
The described embodiment further provides a means to deliver real-time and near real-time multimedia object delivery of different decoding rates in a connectionless oriented connection accurately based on individual decoding rate. The described embodiment further allows synchronisation of the time based object streams such that real-time delivery specified for the object stream can be meet accurately without an accurate timer being implemented by the host network node.
The described embodiment provides a mechanism to perform network congestion avoidance at each intermediate network node dynamically at the rate equivalent to the smallest decoding rate from a set of decoding rates from the object streams and also allows link information between data packets belonging to the same data object to be retained as the data is being fragmented and delivered across the network.
For real time delivery, the intermittent delay is kept at the maximum constant delay such that end to end delay for the arrival of all data objects is kept at a constant rate over time.
The content data switching can be performed at the network level, thus freeing the application level from the rudimentary work of re-transmitting of content object data received.
With the ability to couple fragmented data packets originating from a single data object, the described embodiment enables routers or intermediate network nodes between end-to-end users to drop data packets of the data objects with no presentation value. A data object without presentation value is one where the data object cannot arrive at the end user at the decoding rate. This prevents non-presentation data objects causing network congestion by dropping the said data objects.
A timer to improve synchronisation without the need for a central control time or network time protocol to keep synchronisation between network nodes is provided. The described embodiment allows an accurate means to estimate the decoding time of each object belonging to different object streams.
The described embodiment also allows a single network traffic controller to control object streams with different traffic shaper or filtering algorithms.
The described embodiment allows the existing network configuration to use the implementing apparatus to control real-time traffic over wide area network such as the internet or corporate wide intranet. The described embodiment also provides an infrastructure for the content provider to stream AV content to the users in real-time and the amount of data objects to be delivered to end used can varies depending on the profile of the users on the data packet network such as the internet For a virtual leased line, the server can choose to stream content to their valued clients with better quality of service over the non-valued clients.
An embodiment of the invention will now be described, by way of example, with reference to the accompanying drawings, in which:
The described embodiment of the invention provides a system and an apparatus for, and methods of, delivering multimedia objects for improved streaming at the multimedia content provider, improving the quality presentation at content consumer end and improving the network congestion at the service provider domains. The embodiment is described in the following in detail with reference to accompanying drawings. In the following description, numerous details are set forth in order to provide for thorough understanding of the described embodiment of the present invention by one skilled in the art. In some of the explanations, some obvious details that forms part of the embodiment are left out in order to not unnecessarily obscure the description.
The service provider may contain a network of interconnected switching elements that support different network protocols and architecture. Each of the switching elements is connected to a content traffic exchange hop, 18, to perform the function of object data interchange and to control the flow of object traffic out the switching element. The connectivity between the switching element and the content traffic exchange hop 18 is via a standard link, 17, that allows data to be exchanged. The link 17 allows selected data stream flow from different end-to-end users to be controlled and managed by the content traffic exchange hop 18 before being passed back to the switching element 16.
In the content provider domain 11, traffic is generated by each server entity, 12, and is moderated by an ingress gateway, 13 to control the traffic entering the service provider domain. For brevity, the term “server” is used to describe a logical entity which is the source of content to be delivered across the network. The last switching element 16 in the service provider domain is connected to an egress gateway 111 of the content consumer domain 110, via an interconnect, 19, that shares the same type of network protocol and architecture.
The content consumer domain 110 may consist of clients with different network access profiles. These clients can either have high bandwidth, 114, fixed line access, 112, or low bandwidth, 115, mobile access, 113. The egress gateway 111 will regulate the traffic streamed to the client and the requests from the server carried out by the client.
Reference numeral 21 of
The flowchart in
Another source of timing information and object reference number can be extracted from the payload by the network payload de-packetizer function of functional block 46. The timing and object reference are only extracted from the network packet payload if the network header of the network data packets does not carry timing information and object reference number. Encoding of the timing information and the object reference number in the payload of the data packet follows the object descriptor shown in FIG. 2A.
The timer circuit, 44, contains a time stamp for each object data packet received within the object decoding frame. A handle reference is created to provide a distinction between the various object data packets received based not only on the object reference number extracted from the object descriptor but also the source and destination network address and the numeral representative of the network service access point. For the timer for the object received to work accurately within each frame, the timers in block 44 are updated or created by timing information from an object stream timer circuit, 412. The timing information is also passed to a data de-fragmentor, 42 for accurate construction of a data stream if the data stream received has reached the final destination. For the object stream received, the stream is further processed by an object stream classifier, 410, to determine a final destination of the object stream based on the unique handle reference. If the object stream has reached the final destination, the data packets are passed to the object de-fragmentor, 42. In order for the object to be re-assembled and later fed to the buffer used for object decoding, timing information from the timer circuit 412 which keeps the timers for the object stream received within one object decoding time frame is required.
In order for the host network components such as switches and routers to perform the function of a content exchange traffic controller, the functional sub-blocks in the object stream scheduler block, 47, are required. The internal clocking of the timer referenced by the unique handle reference providing all the accurate time keeping is kept by the object stream timer circuit, 412. This contains timing information for the object stream to be re-routed to another destination. For each stream received, the delivery time slots for each object data packet, based on each individual object stream decoding timing requirement and the transmission time required, are computed by a time slot divider, 411. In addition, divider 411 also provides an object packet stream traffic control, 48 with the object descriptor constructed from the data packet received via the network parameter mapper and decoder functional block, 45. Based on all of the information computed for each object stream of the time slot divider, the object stream traffic control, 48, then decides the time required for the final delivery of the object stream. Based on the precedence of the object stream carried in the object descriptor of each object stream, the time slot allocator 49 performs the final decision as to how the object data fragmented into data packets is delivered out to the network via the functional block 46. Timing information of the data object stream that is scheduled to be transmitted is performed by a system time slot divider, 43, which gathers all the information of each time slot divider 411 assigned to each object stream to determine a minimum object decoding time. The number of time slots for the node is determined by equation eq. 2 below. An object frame is determined by the system time slot divider in accordance with equation eq. 1 below.
Object Frame=Min(DecodingRateObject Stream 1, DecodingRateObject Stream 2, . . . , DecodingRateObject Stream N) (EQ. 1)
System time slot divider 43 performs and maintains a universal object frame for the content exchange traffic controller based on the network resources allocated for transmission. The network resources may include bandwidth, latency and total packet group delay. Information generated by divider 43 is also synchronised with the time slot allocator 49, such that the same timing information can be mapped to the network data header or encoded in the payload of the network packet carried out by the functional block 45.
To enable the exact transmission time of the object data to be transmitted out at the egress node or content traffic exchange hop, a time frame called the object frame is assigned for the object streams to be transmitted out. The pre-allocated time period for the object frame is equivalent to the time for decoding one object data as described in the object descriptor. In the case where there is a requirement to use more or less bandwidth, the object frame period can be increased and reduced. Each object frame may consist of more than one time slot depending on the bandwidth allocated for the delivery of the object data. Depending on the amount of transmission bandwidth allocated, the object frame may contain more than one time slot. The number of time slots per object frame can be computed as follows:
No. of Time Slots (n)=Transmission rate/Object Decoding rate (EQ. 2)
For each object frame, only an integer number of time slots is allowed. In the case of a non-integer value as a result of eq. 2, an additional time slot may be allocated for each time object frame if the sum of the fractional portion of time slots calculated forms a value more than one accumulated over a number of object frames. The rate of additional time slots in terms of object frame is inversly proportional to the fractional portion of the computed time slots, n, derived from eq. 1.
The calculation of an elastic delay is performed by equation 1 and 2. With the preset delay being set by higher application software or set at pre-configured time, the time slot can choose which decoding frame the object can start transmitting. The subsequent object of the same reference must be transmitted within the object decoding time or less. Each node can have a pre-configured delay for the object stream of a specificed object reference.
The preferred implementation of the timer as mentioned in blocks 44 and 412 of
Based on the information from the system time slot divider, time slot information for each individual object stream, and the scheduling of the data object to be streamed out to another destination is performed as shown in FIG. 7. Reference numeral 71 represents an arbitrary number of input object traffic streams entering the node to be re-transmitted to another destination network node. Information such as the stream object descriptor and the information from the output of the time slot divider that provides description of the object are multiplexed by block 72. A full scan for k object streams, 71, by the multiplexer is performed at the maximum rate of the system object frame generated by the system time slot divider, 45. This can be performed at a faster rate if not all the streams have objects for delivery. Besides streaming the information to the stream object slot scheduler, the multiplexer also keeps the statistic of the object stream, as shown in the data structure below. This allows feedback for the dynamic mechanism required by the traffic shaper or filter.
In a preferred form, the multiplexing sequence starts from the object stream with the least decoding rate of 71 and the other precedence classifications then follow.
Memory Size (Queue & Cache)=Group Delay×(max. Object Size/Decoding Rate) (EQ. 3)
Depending on the amount of delay to be supported, the group delay value of each node need to be updated such that re-allocation of the memory size can be recomputed. The recomputed memory size using eq. 3 determines the maximum group delay per hop.
For each object frame computed dynamically by the functional blocks 43, for the system implementing the exchange object traffic controller, a new number of time slots per object are computed using the conditions set forth in eq. 1 and eq. 2. The rate of change is performed at the object frame rate.
The thus described embodiment of the invention provides system architecture for the apparatus as described in the object traffic controller to be configured and connected to any legacy network structure for end-to-end delivery of real-time data. The apparatus provides a real-time mechanism for the content switching based on timing requirement of individual data streams at the network node performing the routing and switching functions. Network congestion is avoided by discarding object data which may consists of a number of network data packets if the real-time requirement failed to be observed due to heavy network traffic. Discarding objects failing real-time requirement allows efficient use of network resources and also provide an indicator to higher layer software that the bandwidth is insufficient for the session. It also enables the network components to operate in a collaborative effort in streaming real-time traffic through a large network. It also frees up the additional resources required at the client terminals to implement sophisticated protocol stack for providing feedback based on the traffic received. In addition, the system allows real-time with high timing accuracy to be delivered across a wide area network without drastic change to the existing network architecture. The client can receive quality content with graceful degradation when there is network congestion or when the bandwidth to the client reduces due to heavy utilisation of network resources or server resources. On the service and content provider ends, it provides an infrastructure and means to differentiate the real-time services provided for the client based on client's profile. This invention is very useful in delivering real-time data across heterogeneous network to light-weight client with limited network resources and processing power such a mobile PDA or mobile video terminal.
The present disclosure relates to subject matter contained in priority Singapore Patent Application No. 200000085-1, filed on Jan. 7, 2000, the contents of which is herein expressly incorporated by reference in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 200000085 | Jan 2000 | SG | national |
This application is a division of U.S. patent application Ser. No. 09/709,726, filed on Nov. 13, 2000 now U.S. Pat. No. 6,917,984, the content of which is hereby incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5481312 | Cash et al. | Jan 1996 | A |
| 5648958 | Counterman | Jul 1997 | A |
| 5864540 | Bonomi et al. | Jan 1999 | A |
| 5918020 | Blackard et al. | Jun 1999 | A |
| 6414960 | Kuhn et al. | Jul 2002 | B1 |
| 6418140 | Matsui | Jul 2002 | B1 |
| 6578092 | Lau et al. | Jun 2003 | B1 |
| 6693909 | Mo et al. | Feb 2004 | B1 |
| 6721271 | Beshai et al. | Apr 2004 | B1 |
| 6760877 | Lappetelainen et al. | Jul 2004 | B1 |
| Number | Date | Country |
|---|---|---|
| 0701376 | Mar 1996 | EP |
| 0738083 | Oct 1996 | EP |
| Number | Date | Country | |
|---|---|---|---|
| 20040215805 A1 | Oct 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09709726 | Nov 2000 | US |
| Child | 10848379 | US |
