Communications scheduler

Abstract

A System for providing communications over a communications network includes a communications interface and processor. The communications interface communicates over the communications network. The processor directs a communications scheduler to determine at least one metric for a path within the communications network. The processor also selects a data flow for the path and determines whether to transmit a packet in the selected data flow based on the at least one metric. The processor then directs a communications protocol handler to generate the packet for the selected data flow.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computer configured to transmit network data in the prior art.

FIG. 2 is an illustration of a network environment in an exemplary implementation of the invention.

FIG. 3 is an illustration of a network device in an exemplary implementation of the invention.

FIG. 4 is a flow chart for the transmission of network data in an exemplary implementation of the invention.

FIG. 5 is a flow chart for the determination of the bandwidth estimate in an exemplary implementation of the invention.

FIG. 6 is a block diagram of the network device in an exemplary implementation of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The embodiments discussed herein are illustrative of one example of the present invention. As these embodiments of the present invention are described with reference to illustrations, various modifications or adaptations of the methods and/or specific structures described may become apparent to those skilled in the art. All such modifications, adaptations, or variations that rely upon the teachings of the present invention, and through which these teachings have advanced the art, are considered to be within the scope of the present invention. Hence, these descriptions and drawings should not be considered in a limiting sense, as it is understood that the present invention is in no way limited to only the embodiments illustrated.

A system for providing communications over a communications network includes a communications interface and a processor. The communications interface communicates over the communications network. The processor directs a communications scheduler to determine at least one metric for a path within the communications network. The processor also selects a data flow for the path and determines whether to transmit a packet in the selected data flow based on the at least one metric. The processor then directs a communications protocol handler to generate the packet for the selected data flow.

The system can advantageously provide reliable transmission of packets through a communications network more efficiently and more fairly than that available in the prior art. Instead of increasing speeds to the point of network congestion before slowing transmission, packets may be transmitted through the communications network at speeds that more closely approximate the capacity of the packet path. As a result, network congestion is reduced and high data transmission rates can be maintained.

Further, the system can advantageously provide fair transmission of packets through the communications network. Instead of transmitting packets based on the conservation of bandwidth over long distances, packets can be transmitted based on an equitable policy. As a result, packets from each user may be transmitted over the communication network at a rate equal to that of other users regardless of the RTT of the transmission. Further, packets may be transmitted at a faster rate based on the relative importance of the packets, the identity of the user providing network data, or the type of digital device generating the packets. Other fairness policies are also possible.

FIG. 2 is an illustration of a network environment 200 in an exemplary implementation of the invention. The network environment 200 includes a source 210, an optional source network 220, a source network device 230, a communications network 240, a destination 250, an optional destination network 260, and a destination network device 270. The source 210 is coupled to the source network device 230 over the source network 220. The source network device 230 is coupled to the communications network 240. The communications network 240 is coupled to the destination network device 270, which is coupled to the destination 250 over the destination network 260.

Source 210 can be any digital device configured to transmit network data. Similarly, the destination 250 is any digital device configured to receive network data. Examples of digital devices include, but are not limited to, computers, personal digital assistants, and cellular telephones. The source network 220 and the destination network 260 are any networks that couple a digital device (e.g., source 210) to a network device (e.g., source network device 230). The source network 220 and the destination network 260 can be a wired network, a wireless network, or any combination.

The communications network 240 can be any network configured to carry network data and/or packets between the source network device 230 and the destination network device 270. In one example, the communications network 240 is the Internet.

The embodiments in FIGS. 2-5 depict an example of packets being transmitted from the source 210 to the destination 250 through the source network device 230, the communications network 240, and the destination network device 270. Other embodiments may include packets being transmitted directly from the destination 250 to the source 210. In an example, the source 210 and the destination 250 may comprise the source network device 230 and the destination network device 270, respectively. While there are numerous variations in where packets are generated and transmitted, the figures below describe one example of packet transmissions from the source 210 to the destination 250 for the sake of simplicity.

The source network device 230 and the destination network device 270 are any device or system configured to process and exchange packets over the communications network 240. A path is any route the packet may take from the source network device 230 to the destination network device 270. The configuration of the source network device 230 and the destination network device 270 are described in further detail below in FIG. 6. One example of the source network device 230 and the destination network device 270 is an appliance in a network memory architecture, which is described in U.S. patent application Ser. No. 11/202,697 entitled “Network Memory Architecture” filed on Aug. 12, 2005, which is hereby incorporated by reference.

FIG. 3 is an illustration of a network device 300 in an exemplary implementation of the invention. The network device 300 may be the source network device 230 (FIG. 2) or the destination network device 270 (FIG. 2). The network device 300 includes a communications protocol handler 310, a communications scheduler 320, and a communications interface 330. The communications protocol handler 310 is coupled to the communications scheduler 320 over scheduler link 340. The communications scheduler 320 is further coupled to the communications interface 330 over communications link 350. The communications interface 330 is coupled to the communications protocol handler 310 over handler link 360, the communications network 240 (FIG. 2) over network link 370, and the source network 220 over the source link 380. The communications protocol handler 310, the communications scheduler 320, and the communications interface 330 may be software modules. Software modules comprise executable code that may be processed by a processor (not depicted).

The communications protocol handler 310 is any combination of protocols configured to organize network data into packets. In one example, the communications protocol handler 310 is a TCP/IP stack. In other examples, the communications protocol handler 310 is a User Datagram Protocol (UDP) stack or a Real-time Transport Protocol (RTP) stack. The communications protocol handler 310 may receive network data from an application (not depicted). The communications protocol handler 310 organizes the network data from the application into packets which are to be transmitted over the communications network 240. The communications protocol handler 310 may receive the network data from the application directly. Alternately, the application can reside on the source 210 outside of the network device 300. In an example, the communications interface 330 of the network device 300 receives the network data from the application over the source link 380. The communications interface 330 then forwards the application data over the handler link 360 to the communications protocol handler 310.

The communications scheduler 320 is configured to control the transmission of the packets from the communications protocol handler 310. The communications scheduler 320 can determine at least one metric for a path (discussed in FIG. 2, herein) on the communications network 240 and then control the flow of packets on that path based on the one or more metrics. The metric is any measured value related to a quality, operator, or performance of the path. In one example, the metric is a bandwidth estimate for the path. The bandwidth estimate is a value that estimates the number of packets that may be sent over a path during a predetermined time (e.g., the capacity of the path to transmit packets without congestion). If the bandwidth estimate of the path is high, the path may be capable of carrying a large number of packets. Otherwise, if the bandwidth estimate is low, the path may be capable of carrying a smaller number of packets.

The communications scheduler 320 can determine the bandwidth estimate of any number of paths on the communications network 240. In one example, the communications scheduler 320 transmits probe packets through the communications interface 330 over the communications network 240 to another network device 300. The communications scheduler 320 of the other network device 300 receives the probe packet and transmits a monitor packet back to the originating network device 300. The communications scheduler 320 of the originating network device 300 receives the monitor packet and determines a bandwidth estimate for the path. The determination of the metric is further discussed in FIG. 5.

The communications scheduler 320 can control the transmission of the packets from the communications protocol handler 310 based on the metric of the path. In one example, the communications scheduler 320 limits the number of packets transmitted to the capacity of the path based on the metric. This process if further discussed in FIG. 4. Although the communications protocol handler 310 may comprise a protocol that controls the transmission of network data to avoid congestion (e.g., TCP/IP stack methodology), the communications scheduler 320 may override this function.

By determining the capacity of the path and controlling the flow of packets over the communications network 240, the communications scheduler 320 can increase or optimize the speed in which network data flows across the communications network 240. The prior art protocols typically begin at slow-start and increase speed until congestion appears. Subsequently, the prior art protocols slow down the rate of transmission and slowly increase again. The communications scheduler 320 can maintain speeds that the path will allow. The ultimate throughput the communications scheduler 320 achieves may be faster than the average speed of the prior art protocols.

In some embodiments, the communications scheduler 320 pulls packets from the communications handler 310 obviating the need for buffers. The communications handler 310 can generate a packet at the command of the communications scheduler 320. In one example, the speed at which packets are generated and transmitted is equivalent to the bandwidth estimate. Since the communications scheduler 320 is pulling packets from the communications handler 310 rather than determining transmission rates after packet generation, the packets need not be buffered before transmission. As a result, buffering may be reduced or eliminated which can increase the speed of transmission and/or reduce hardware costs.

The communications interface 330 is coupled to the network device 300, the source 210, the source network 220, and/or the communications network 240. The communications interface 330 can transmit the packets received over the communications link 350 from the communications scheduler 320 to the communications network 240. The communications interface 330 also provides packets received from the communications network 240 to the communications protocol handler 310 over the handler link 360. In some embodiments, the communications interface 330 sends any monitor packets received from another network device 300 to the communications scheduler 320. Further, the communications interface 330 may send any network data received from an application over the source link 380 to the communications protocol handler 310 where the network data will be subsequently organized into packets to prepare for further transmission over the communications network 240. In some embodiments, the communications interface 330 is linked to both the source network 220 and the communications network 240 over a single link (e.g., the network link 370 or the source link 380).

FIG. 4 is a flow chart for the transmission of network data in an exemplary implementation of the invention. FIG. 4 begins in step 400. In step 410, the communications scheduler 320 (FIG. 3) selects a path from eligible paths. An eligible path can be any path with a bandwidth estimate, a path with data to send, or any path that meets a predetermined quality of service criteria. Alternatively, the eligible path can be any path with a bandwidth estimate equal to or greater than a predetermined estimate. In another example, the eligible paths can be determined to be a predetermined percentage of paths with a higher bandwidth estimate than the others.

The communications scheduler 320 can select a path from the eligible paths based on the bandwidth estimate. In one example, the communications scheduler 320 selects the path with the highest bandwidth estimate. In some embodiments, the paths may be prioritized. Specific paths may be weighed based on the properties of one or more networks within the communications network 240. In one example, the path may extend through a virtual private network (VPN) or a network with a bandwidth guarantee.

In step 420, the communications scheduler 320 retrieves a bandwidth estimate associated with one or more paths. The communications scheduler 320 can continue to retrieve or receive bandwidth estimates during any step of FIG. 4. The process of determining a bandwidth estimate is further discussed in FIG. 5.

In step 430, the communications scheduler 320 determines if the number of packets generated over the selected path for a predetermined time is less than the bandwidth estimate of the selected path. In one example, the communications scheduler 320 tracks the number of packets that have been transmitted over each path as well as when the packets where transmitted. If the number of packets transmitted over the selected path during a predetermined period of time is greater than the bandwidth estimate, the communications scheduler 320 retrieves the bandwidth estimate associated with other paths in step 420. In some embodiments, the communications scheduler 320 subsequently selects a new path from eligible paths before returning to step 430. In other embodiments, if the number of packets transmitted over the selected path during a predetermined period of time is greater than the bandwidth estimate, FIG. 4 ends. If the number of packets generated for a selected path is less than the bandwidth estimate, then the communications scheduler 320 can prioritize the data flows for the selected path in step 440.

In one example, the communications scheduler 320 queries the communications protocol handler 310 (FIG. 3) for available data flows. A data flow comprises related network data or packets. In one example, packets belonging to the same data flow comprise the same source IP address, destination IP address, protocol, source port, and destination port. There may be a separate data flow for separate applications, sessions, or processes. Each data flow may also be prioritized based on the purpose of the network data within the data flow or the source of the data flow (e.g., the digital device that generated the data flow or a user of the digital device that generated the data flow). In some embodiments, the communications protocol handler 310 notifies the communications scheduler 320 of all data flows without being queried.

In exemplary embodiments, the data flows are weighted depending upon the application that originated the data flow, the user of the application, the number of data flows already sent from the application (or the user), and the number of packets already sent from that data flows. In one example, the data flows are all given equal weight and a packet is sent from each eligible data flow in turn (e.g., a round robin approach). In another example, certain applications or users are given priority over other applications or other users (e.g., by weighing certain applications higher than others). Packets generated by a particular source IP address or transmitted to a particular destination EP address may also be given priority. There may be many different methodologies in weighing the data flows.

In step 450, the communications scheduler 320 selects the data flows for the selected path. In one example, the data flows are selected based on an assigned weight or priority. In some embodiments, the data flows are re-weighted (i.e., re-prioritized) after a packet is transmitted.

Instead of transmitting packets based on the round trip time of packets (e.g., the distance that packets are transmitted), packets can be transmitted based on a configurable fairness policy. A fairness policy is any policy that allows for equitable transmission of packets over the communications network. In one example, the fairness policy dictates that every data flow be given equal weight. In another example, the fairness policy dictates that certain users or data flows are more important (e.g., time sensitive) than others and therefore are given greater weight. The fairness policy can base fair transmission of packets on the saliency of users and/or data rather than the preservation of bandwidth over long distances within the communications network 240 (FIG. 2).

In step 460, the communications scheduler 320 performs a function call to the communications protocol handler 310 to generate a packet from the selected data flow. In one example, the communications protocol handler 310 receives network data from an application (not depicted). The network data is organized into data flows. The communications scheduler 320 prioritizes the data flows and selects a data flow. Subsequently, the communications scheduler 320 provides a function call to command the communications protocol handler 310 to organize the network data into packets.

In some embodiments, the communications protocol handler 310 does not generate packets without a function call from the communications scheduler 320. In one example, the packets are not buffered prior to transmission over the network link 370 (FIG. 3). As a result of the communications scheduler 320 pulling packets from the communications protocol handler 310, buffering the packets prior to transmission may be reduced or eliminated.

In step 470, the communications scheduler 320 transmits the packet of the selected data flow. In one example, the communications scheduler 320 commands the communications interface 330 to transmit the packet over the network link 370. The communications scheduler 320 then retrieves a new bandwidth estimate associated with one or more paths in step 420 and the process can continue. In other embodiments, FIG. 4 ends after step 470.

In some embodiments, the communications scheduler 320 overrides the behavior of the communications protocol handler 310 to transmit packets at the bandwidth estimate. In one example, the communications scheduler 320 overrides the cwnd behavior to control the size of the congestion window of the TCP/IP stack (i.e., the communications protocol handler 310). As a result, the communications scheduler 320 can replace or modify the cwnd behavior (or any behavior that influences the congestion window) to cause the communications protocol handler 310 to transmit packets at the rate based on the bandwidth estimate.

FIG. 5 is a flow chart for the determination of the bandwidth estimate in an exemplary implementation of the invention. FIG. 5 begins in step 500. In step 510, the source network device 230 (FIG. 2) generates and transmits one or more probe packets. A probe packet is a packet sent by the source network device 230 to a destination network device 270 (FIG. 2) to determine a metric for a particular path. The metric may be a bandwidth estimate. In an example, the communications scheduler 320 (FIG. 3) of the source network device 230 generates and subsequently transmits one or more probe packets over the communications network 240 (FIG. 2). In some embodiments, the probe packets are stamped with a transmission timestamp based on the time of transmission. Further, the probe packets may be stamped with a selected path over which the probe packet is to be sent.

In step 520, the destination network device 270 receives the probe packets from the source network device 230. In an example, the communications scheduler 320 of the destination network device 270 receives the probe packets. The destination network device 270 marks the arrival of the one or more probe packets with a timestamp in step 530. In one example, the destination network device 270 may collect probe information associated with the one or more probe packets including, but not limited to, the source network device 230 that sent the one or more probe packets, the path over which the probe packet(s) was sent, and/or the transmission timestamp of the probe packet(s).

In step 540, the destination network device 270 determines the bandwidth estimate of the selected path based on the timestamp(s) of the one or more probe packets. In some embodiments, the destination network device 270 determines the bandwidth estimate by determining the number of eligible probe packets received over a predetermined time. Eligible probe packets can be probe packets with timestamps within the predetermined time. In some embodiments, the destination network device 270 determines the bandwidth estimate based on the inter-arrival time between probe packets (e.g., the time between receipt of successive probe packets).

In step 550, the destination network device 270 generates and transmits a monitoring packet with the bandwidth estimate to the source network device 230. In one example, the communications scheduler 320 of the destination network device 270 generates and transmits the monitoring packet to the communications scheduler 320 of the source network device 230.

In step 560, the source network device 230 receives the monitoring packet from the destination network device 270. In some embodiments, the destination network device 270 transmits the monitoring packet over the same selected path as the one or more probe packets. The source network device 230 can confirm the bandwidth estimate contained within the monitoring packet or modify the bandwidth estimate based on the time when the monitoring packet was received. In one example, the destination network device 270 transmits the monitoring packet with a timestamp to allow the source network device 230 to re-calculate the bandwidth estimate for the selected path. In other embodiments, the destination network device 270 transmits the monitoring packets with the timestamp over a different path to allow the source network device 270 to receive the bandwidth estimate for the selected path and calculate the bandwidth estimate for the different path.

In step 570, the source network device 230 determines the bottleneck based on the bandwidth estimate. In one example, the communications scheduler 320 pulls packets from the communications protocol handler 310 based on the bandwidth estimate. The pulled packets are subsequently transmitted over the communications network 240 by the communications interface 330.

In other embodiments, the source network device 230 transmits probe packets without timestamps to the destination network device 270 over a selected path. The destination network device 270 receives the probe packets and transmits monitoring packets with timestamps to the source network device 230 over the same path. The source network device 230 receives the monitoring packets and then determines the bandwidth estimate of the path based on the timestamps of the monitoring packets.

Many different probe packets may be sent over many paths from a source destination network device 230 to many different destination network devices 270 during a given time. By continuously discovering new paths and modifying the bandwidth estimates of existing paths, the source network device 230 can increase the through-put of packets to the destination 250 (FIG. 2) without continuously decreasing and increasing the speed of packet transmission when congestion occurs.

FIG. 6 is a block diagram of a network device 600 in an exemplary implementation of the invention. The network device 600 may have a similar configuration as the source network device 230 (FIG. 2) and/or the destination network device 270 (FIG. 2). The network device 600 includes a processor 610, a memory 620, a network interface 630, and an optional storage 640 which are all coupled to a system bus 650. The processor 610 is configured to execute executable instructions.

The memory 620 is any memory configured to store data. Some examples of the memory 620 are storage devices, such as RAM or ROM.

The network interface 630 is coupled to the communications network 240. (FIG. 2) and the source 210 (FIG. 2) via the link 660. The network interface 630 is configured to exchange communications between the source 210, the communications network 240, and the other elements in the network device 600. In some embodiments, the network interface 630 may comprise a Local Area Network interface for the source 210 and a Wide Area Network interface for the communications network 240.

The optional storage 640 is any storage configured to retrieve and store data. Some examples of the storage 640 are hard drives, optical drives, and magnetic tape. The optional storage 640 can comprise a database or other data structure configured to hold and organize data. In some embodiments, the network device 600 includes memory 620 in the form of RAM and storage 640 in the form of a hard drive.

The above-described functions can be comprised of executable instructions that are stored on storage media. The executable instructions can be retrieved and executed by the processor 610. Some examples of executable instructions are software, program code, and firmware. Some examples of storage media are memory devices, tape, disks, integrated circuits, and servers. The executable instructions are operational when executed by the processor to direct the processor to operate in accord with the invention. Those skilled in the art are familiar with executable instructions, processor(s), and storage media.

The above description is illustrative and not restrictive. Many variations of the invention will become apparent to those of skill in the art upon review of this disclosure. The scope of the invention should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims along with their full scope of quivalents.

Claims

1. A system for providing communications over a communications network, the system comprising: a communications interface configured to communicate over the communications network; anda processor configured to direct a communications scheduler to determine at least one metric for a path within the communications network, select a data flow for the path, determine whether to transmit a packet in the selected data flow based on the at least one metric, and direct a communications protocol handler to generate the packet for the selected data flow.

2. The system of claim 1 wherein the communications interface is further configured to transmit the packet in the selected data flow.

3. The system of claim 1 wherein the communications protocol handler comprises a transmission control protocol/internet protocol stack.

4. The system of claim 1 wherein the processor is further configured to direct the communications protocol handler to receive information from an application to be included in the packet.

5. The system of claim 1 wherein the at least one metric comprises a bandwidth estimate.

6. The system of claim 1 wherein the processor is further configured to direct the communications scheduler to determine if the flow has information to send prior to selecting the data flow for the path.

7. The system of claim 1 wherein the processor is configured to direct the communications scheduler to select the data flow comprises the processor configured to direct the communications scheduler to determine a priority of data flows and determine the data flow to generate the packet based on priority of data flows determination and the at least one metric.

8. The system of claim 7 wherein the priority of data flows is based on a fairness policy.

9. A system for providing communications over a communications network, the system comprising: a communications scheduler module configured to determine at least one metric for a path within the communications network, select a data flow for the path, determine whether to transmit a packet in the selected data flow based on the at least one metric, and direct a communications protocol handler to generate the packet for the selected data flow; anda communications network handler module configured to receive the direction from the communications scheduler to generate the packet in the selected data flow and generate the packet based on the direction.

10. The system of claim 9 wherein the communications network handler module is further configured to transmit the packet in the selected data flow.

11. The system of claim 9 wherein the communications protocol handler module comprises a transmission control protocol/internet protocol stack.

12. The system of claim 9 wherein the communications protocol handler module is further configured to receive information from an application to be included in the packet.

13. The system of claim 9 wherein the at least one metric comprises a bandwidth estimate.

14. The system of claim 9 wherein the communications scheduler module is further configured to determine if the data flow information to send prior to selecting the data flow for the path.

15. The system of claim 9 wherein the communications scheduler module is configured to select the data flow comprises the communications scheduler module configured to determine a priority of data flows and determine the data flow to generate the packet based on the priority of data flows determination and the at least one metric.

16. The system of claim 15 wherein the priority of data flows is based on a fairness policy.

17. A method for providing communications over a communications network, the method comprising: in a communications scheduler, determining at least one metric for a path within the communications network;in the communications scheduler, selecting a data flow for the path;in the communications scheduler, determining whether to transmit a packet in the selected data flow based on the at least one metric; andin the communications scheduler, directing a communications protocol handler to generate the packet for the selected data flow.

18. The method of claim 17 further comprising in the communications protocol handler, transmitting the packet in the selected data flow.

19. The method of claim 17 wherein the communications protocol handler comprises a transmission control protocol/internet protocol stack.

20. The method of claim 17 further comprising in the communications protocol handler, receiving information from an application to be included in the packet.

21. The method of claim 17 wherein the at least one metric comprises a bandwidth estimate.

22. The method of claim 17 further comprising in the communications scheduler, determining if the path has information to send prior to selecting the data flow for the path.

23. The method of claim 17 wherein selecting the data flow comprises in the communications scheduler, determining a priority of data flows and determining the data flow to generate the packet based on the priority of data flows determination and the at least one metric.

24. The method of claim 23 wherein the priority of data flows is based on a fairness policy.

25. A software product for providing communications over a communications network, the software product comprising: a communications scheduler software operational when executed by a processor to direct the processor to determine at least one metric for a path within the communications network, select a data flow for the path, determine whether to transmit a packet in the selected data flow based on the at least one metric, and direct a communications protocol handler to generate the packet for the selected data flow; anda storage medium configured to store the communications scheduler software.

26. The software product of claim 25 wherein the communications scheduler software is further operational when executed by the processor to direct the processor to transmit the packet in the selected data flow.

27. The software product of claim 25 wherein the communications protocol handler comprises a transmission control protocol/internet protocol stack.

28. The software product of claim 22 wherein the at least one metric comprises a bandwidth estimate.

29. The software product of claim 25 the communications scheduler software is further operational when executed by the processor to direct the processor to determine if the path has information to send prior to the selection of the data flow within the path.

30. The software product of claim 25 wherein the communications scheduler software is further operational when executed by the processor to direct the processor to determine a priority of data flows and determine the data flow to generate the packet based on the priority of data flows determination and the at least one metric.

31. The software product of claim 30 wherein the priority of data flows is based on a fairness policy.