Stateless distributed load-balancing

Abstract

Aspects of the subject technology provide state-less load-balancing using sequence numbers to identify traffic flows. In some implementations, a process of the technology can include steps for receiving, by a load-balancer, a first packet from a source device including a request to access the service provided by a server coupled to the load-balancer, determining a load for each of the servers, wherein each server is associated with a unique set of sequence numbers, and forwarding the request to a target server selected based on its corresponding load, and wherein the request is configured to cause the target server to issue a reply to the source device. Systems and machine-readable media are also provided.

Description

TECHNICAL FIELD

The present technology pertains to load-balancing, and more specifically to stateless load-balancing achieved using sequence numbers to identify and balance traffic flows.

BACKGROUND

The ubiquity of Internet-enabled devices has created an enormous demand for Internet services and content. This Internet-connected revolution has created significant challenges for service and content providers who often struggle to service a high volume of user requests without falling short of user performance expectations. For example, providers typically need large and complex datacenters to keep up with network and content demands from users. These datacenters are generally equipped with server farms configured to host specific services, and include numerous switches and routers configured to route traffic in and out of the datacenters.

Not surprisingly, such large volumes of data can be difficult to manage and create significant performance degradations and challenges. Load-balancing solutions may be implemented to improve performance and service reliability in a datacenter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only exemplary embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:

FIG. 1 illustrates an example network environment in which some aspects of the technology can be implemented;

FIG. 2 illustrates an example process for implementing a load-balancing technique of the subject disclosure;

FIG. 3 conceptually illustrates a timing diagram of example network communications used to implement a load-balancing technique of the disclosed technology;

FIG. 4 illustrates an example network device in accordance with various embodiments; and

FIG. 5 illustrates an example computing device in accordance with various embodiments.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the disclosure.

Overview:

Additional features and advantages of the disclosure are set forth in the description which follows, and in part can be learned by practice of the disclosed principles. The features and advantages of the disclosure can be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the disclosure are fully apparent from the following description and appended claims, or can be learned by the practice of the principles set forth below.

Disclosed are systems, methods, and computer-readable media configured for providing stateless load-balancing between multiple servers. In some aspects, a process of the technology can include steps for receiving, by a load-balancer, a first packet from a source device, the first packet comprising a request to access a service provided by a server coupled to the load-balancer, determining, by the load-balancer, a load for each of a plurality of servers, wherein each of the plurality of servers is associated with a unique set of sequence numbers. In some aspects, the process can further include steps for forwarding the request to a target server selected from among the plurality of servers based on the determined load for each of the servers, and wherein the request is configured to cause the target server to issue a reply to the source device, the reply comprising a sequence number selected from the unique set of sequence numbers corresponding with the target server.

A process of the technology can further include operations for receiving, by the load-balancer, a second packet from the source device, wherein a header of the second packet contains the sequence number, and routing, by the load-balancer, the second packet to the target server based on the sequence number in the header of the second packet. In some instances, the sequence number of the second packet may be a value that is incremented based on an amount of data transmitted from the target server to the source device, for example, in a previous transmission frame.

As explained in further detail below, the load-balancer can be configured to determine when a sequence number for a corresponding stream exceeds (or is likely to exceed) a range of sequence numbers corresponding with the target server and, in response, to begin monitoring server states for the corresponding flow. Depending on the desired implementation, the sequence numbers can correspond with Transmission Control Protocol (TCP) sequence numbers.

Description:

Conventional load-balancers typically either use a hash-based packet destination assignment process, or make routing decisions based on server loads. Both techniques have potential drawbacks. On one hand, hash-based destination assignments fail to take consideration of server loads, and can therefore be inefficient for traffic flows that are substantially larger (or smaller) and other flows being balanced. On the other hand, performing load-balancing based on measured server loads requires that the load-balancers store and routinely update information regarding server usage levels (e.g., CPU utilization), which can become onerous in highly dynamic network settings, such as implementations where a large number of servers are used, or where server loads are quickly changing.

Aspects of the disclosed technology address the foregoing problems of conventional load-balancer implementations by providing stateless load-balancing that is performed using traffic flow identifiers to parse flows between available servers. In some aspects, initial packet routing for a new flow is performed based on server load information, which can include, but is not limited to: CPU utilization for one or more servers, memory utilization, network-path information, and/or traffic flow information, etc. Additionally, in some aspects, server load information can include information regarding loads attributed to a specific process, container, and/or virtual machine, etc. In some aspects, server loads can relate to the use of memory resources, such as the fill level of one or more queues, or the number, size and/or type of requests queued. It is understood that server loads can be determined or measured using various metrics or network characteristics. By way of non-limiting example, server loads can be based on measures of: a number of requests per second (RPS), average response time (ART), peak response time (PRT), uptime, a number of threads, and/or a count of Open Files Descriptors (OFD), etc.

After an initial packet is received by the selected server (e.g., the target server) chosen by the load balancer, the target server responds to the source device and provides a unique identifier that is associated with the target server. The unique identifier can then be inserted, by the source device, into subsequent packets transmitted to the server. In some aspects, the unique identifier inserted by the source device is represented by a value that is based on the unique identifier received from the target server and a number of TCP bytes transmitted by the target server in a previous frame. By way of example, the unique identifier provided by the server may be 2000, with 1300 total bytes transmitted in the previous frame. Therefore, the unique identifier inserted by the source device may be 3300 (i.e., 2000+1300).

When the subsequent packets are received from the source device at the load-balancer, routing decisions can be made (i.e., to provide the same flow to the target server), based on the unique identifier in the packet metadata (header). As discussed in further detail below, each server can be associated with a predetermined range of unique identifiers (e.g., TCP sequence IDs), for example, that can be used to match flows/servers. As such, server-state information at the load balancer can be used to associate a specific flow with a given instance of an application on a server, without the need of the load balancer to persist and constantly update server-load information.

Additionally, in some aspects, the load balancers can be configured to detect when an upper bound of sequence numbers sent by the source device (e.g., client) are likely to exceed a range for the associated server. In such instances, the load balancer may again begin using server loads to perform balancing, and identifying flows based on other flow-identifying information, such as their associated 5-tuple.

By using TCP sequence IDs to perform flow/server matching, the load-balancers can achieve fast and efficient load-balancing for small and/or short-lived flows. This type of balancing achieves better flow optimization, as compared to purely state-based balancing, by making state-based decisions for new and large traffic flows, while avoiding the need to persist server-load information for smaller flows.

FIG. 1 illustrates a schematic diagram of an example network environment 100 in which a load-balancing technique of the disclosed technology can be implemented. Network environment 100 includes a data center 120, which can represent one or more data centers and/or networks. For example, data center 120 can be a single data center or may include multiple of data centers. Data center 120 can be physically located in a same geographic location or distributed throughout multiple locations. Moreover, data center 120 can include forwarder-side and server-side architectures or components as will be further described.

Client 102 can connect with routers 106-1 through 106-N (collectively “106” hereinafter) in the data center 120 via a network 104. Client 102 can be any computing device, such as a laptop, a desktop, a tablet computer, a mobile phone, a server, a smart device (e.g., smart television, smart watch, etc.), an internet of things (IoT) device, a remote network or data center, etc. Additionally, network 104 can include any number or type of networks, such as a private network (e.g., local area network), a public network (e.g., the Internet), a hybrid network (e.g., virtual private network), a cloud network, etc.

Routers 106 can serve as edge devices in data center 120, and route traffic to and from data center 120. Thus, routers 106 can connect data center 120 with network 104, client 102, and any other external networks or devices. Routers 106 can serve as egress and ingress points for data center 120. Routers 106 can also route traffic internally within data center 120 to other routers or switches, network devices or services (e.g., appliances, firewalls, load-balancers, etc.), and application servers 110-1 through 110-N (collectively “110” hereinafter) in data center 120.

Application servers 110 can include physical machines or resources hosting applications, isolated environments, or services in data center 120. For example, application servers 110 can be physical servers running various applications in data center 120. Application servers 110 can run some or all of their applications in isolated environments, such as in VMs or software containers. In some cases, an application can by hosted by, and/or run on, multiple application servers 110 in data center 120. For example, application servers 110 can run instances of an application (e.g., virtual instances, replicas, parallel instances, mirror instances, etc.). To illustrate, an application can run on multiple application servers 110, to allow the multiple application servers 110 to load-balance application traffic, and/or provide redundancy (e.g., backup or standby), fault-tolerance, high-availability, scalability, etc., for the application. Application servers 110 can run a full application or instance of the application, or a portion of the application, such as a function in a service chain configuration.

Application servers 110 can include a physical network interface, e.g., a Network Interface Card (NIC) to communicate with other devices or services (e.g., devices or services in the network environment 100). The physical network interface can be assigned a physical prefix or network address for such communications. Application servers 110 can also include one or more virtual interfaces (e.g., vNICs) that provide virtualized or abstract representations of network interfaces and connections. Virtual interfaces can provide added flexibility and network capabilities, as well as various other benefits or services, such as aggregation of links or data, isolation of data or networks, decoupling of application and system traffic, expansion of network interfaces, network redundancy, dedicated links, and so forth. Virtual interfaces can be assigned virtual addresses (e.g., VIPs) in data center 120. The virtual addresses can identify the virtual interfaces as well as any applications or isolated environments associated with the virtual addresses on application servers 110.

For example, an application can be assigned a virtual address in the data center 120, which can be used to identify the application in data center 120 and route traffic to and from the application. The virtual address can be used to steer traffic to and from a virtual instance of the application running on one or more of the application servers 110. In some cases, the virtual address can be mapped to the same application on multiple application servers 110, and can be used to communicate with an instance of the application on any of the multiple application servers 110.

Application servers 110 can include a virtual switch, which can route traffic to and from the application servers 110. For example, a virtual switch can route traffic between physical and virtual network interfaces on an application server, between applications and/or isolated environments on the application server, and between the application server and devices or applications outside of the application server. To illustrate, an application server can run multiple workloads, such as, applications in different virtual machines or containers, assigned to different virtual interfaces and virtual addresses. A virtual switch on the application server can route traffic to and from different workloads by translating the virtual addresses of the workloads and communicating with the virtual interfaces as well as other network interfaces such as the physical network interface(s) on the application server.

Data center 120 can also include load-balancers 108-1 through 108-N (collectively “108” hereinafter). Load-balancers 108 can communicate traffic between routers 106 and application servers 110. Moreover, load-balancers 108 can provide load-balancing and forwarding services for traffic associated with application servers 110. Load-balancers 108 can select application servers 110 for a given flow to distribute flows and loads between the application servers 110 and steer traffic accordingly. As discussed in further detail below, application servers 110 can steer traffic flows based sequence number identification information contained in packet-header information for packets received from client (source device) 102.

FIG. 2 illustrates a flow diagram of an example process 200 for implementing a load-balancing technique of the disclosed technology. Process 200 begins when a first packet is received, at a load balancer, from a source device (202). As discussed above, the source device can be a client device, such as client device 102, discussed above with respect to FIG. 1. Additionally, the first packet can be received at a load-balancer, such as any of load-balancers 108, described above.

The first packet can include a request from the source device to access a service (e.g., an application) provided by a server that is communicatively coupled to the load-balancer. That is, the first packet can include information including at least a portion of a request for resources/services provided by a server reachable by the receiving load-balancer. Further to the environment illustrated in FIG. 1, the source device (e.g., client 102), can request resources/services provided by any of application servers 110.

After the first packet is received by the load-balancer, the load-balancer determines a server-load for one or more of the connected servers (204). As discussed above, determinations of server-loads can be based on absolute or relative measures of CPU utilization, and/or based on other network characteristics, such as network path or traffic flow information. Determinations of loads for each of the connected servers can be triggered in response to the receipt of one or more packets for a new traffic flow, such that server load determinations are not performed for all traffic received at the load-balancer. In practice, each of the servers connected with the load-balancer are associated with a unique set of sequence numbers, such as, Transmission Control Protocol (TCP) sequence ID numbers.

Subsequently, the load-balancer forwards the request to a target server that is selected from among the available servers, based on the corresponding load for each server. That is, the request received by the load-balancer is forwarded to a selected target (destination) server based on an initial determination of relative server capacity. In this manner, initial destination routing for a new request (flow) is made by the load-balancer based on actual server load states.

After the request is received from the load-balancer at the target server, the target server can reply to the source device directly, i.e., without communicating via the load-balancer. In practice, the reply of the target server includes a unique sequence number selected for insertion into the packet/s transmitted in the reply. The selected sequence number can correspond with an unallocated identifier that is selected from among the available sequence numbers in the target server's associated sequence ID set. As such, the sequence number (or sequence ID) can correspond with a particular traffic flow between the source device and the server. In some aspects, the sequence ID can be a TCP sequence ID that is embedded in packet header information of the packets for the corresponding flow.

Once the source device receives the server's reply, subsequent packet transmissions by the source device are provided to the load-balancer and automatically routed to the correct target server based on sequence ID information (208, 210). That is, one or more subsequent packets received by the load-balancer, from the source device, can be routed by the load-balancer (to the correct target server) by identifying the sequence ID in the packet header information.

In some aspects, the sequence ID provided in the packets transmitted by the source device can be a different value than that of the sequence ID received by the source device from the target server. For example, the source device can be configured to derive or compute a new sequence ID value based on the original sequence ID value received from the server as well as a number/size of packets received from the server. By way of example, the sequence ID value placed in packet header information by the source device can equal the original sequence ID value incremented by a number of TCP data bytes transmitted by the server. If, for example, the initial sequence ID value transmitted by the server to the source device was 2600, and 858 TCP data bytes were transmitted by the server in a previous frame, then the new sequence ID value that is used by the source device can be 3458 (2600+858).

In some aspects, the routing of incoming packets based on sequence ID information can be performed by the load-balancer using a table of server-state values that map sequence ID information to destination servers. As such, the load-balancer can make fast routing decisions for incoming packets of existing flows by quickly reference sequence IDs in packet header information, without the need to store (or frequently update) server-load information.

As discussed above, sequence ID values may eventually exceed a range associated with any given server. In some implementations, the load-balancer is configured to detect flows using sequence IDs having values that are likely to exceed a designated range for the associated server. In such instances, the load-balancer can begin identifying the flow using other types of information, including but not limited to the flow's corresponding 5-tuple. In some aspects, this transition can also cause the load-balancer to perform balancing for packets of the related flow based on server-load information. That is, detections of out-of-range sequence ID values can re-trigger load-based balancing. As such, large flows can be selectively identified and managed based on server-load information, whereas smaller flows can be efficiently routed using in-range ID values, e.g., based on server-state information.

FIG. 3 conceptually illustrates an example timing diagram of device communications used to implement a load-balancing technique of the disclosed technology. In the illustrated example, source device 301 can represent any network device from which a server request can originate (e.g., client device 102, discussed above); load-balancer 302 can represent any load-balancing network node, including physical boxes and/or virtually instantiated devices (e.g., load-balancers 108, discussed above); and target server 303 can represent a single network device or a combination of physical/virtual devices, for example, that is/are configured to provide information or processing to a requesting source device 301.

In practice, source device 301 transmits a request 305 to target server 303 via load-balancer 302. After request 305 is received at load-balancer 302, load-balancer 302 reads packet header information of the request to determine if the request corresponds with a new traffic flow (e.g., based on the absence of sequence ID information), or if the request corresponds with a legacy flow for which routing has been previously determined (e.g., based on the presence of a sequence ID in the packet header).

If a sequence ID exists in the packet header, load-balancer 302 can route the request to the corresponding server, for example, by referencing a table of server-state entries that associate server/sequence ID pairs. Alternatively, if no sequence ID information exists in the request, then the load-balancer can determine server-loads for two or more available servers to determine the optimal target/destination server for routing of the request.

In the example of FIG. 3, request 305 represents a new traffic flow from source device 301, and is routed to available target server 303 by load-balancer 302. The initial routing of source device traffic to a target server can be performed by the load-balancer based on server-load information for one or more servers. As such, initial balancing of new flows is based on server CPU usage or capacity information for the available servers. After target server 303 receives request 305, reply 309 is generated by target server 303 that includes a TCP sequence ID, for example, that is selected by target server 303 from among a pre-allocated set of TCP sequence ID's. Reply 309 is then forwarded directly to source device 301, e.g., without the need of routing by load-balancer 302.

After the TCP sequence ID is received at source device 301, subsequent packets transmitted from source device 301 to target server 303 include the TCP sequence ID to facilitate routing by load-balancer 302. That is, source device 301 inserts the TCP sequence ID indicated by target server 303 and the subsequent traffic is communicated to target server 303 in the same flow. In the example of FIG. 3, a subsequent (second) packet 311 transmitted by source device 301 to target server 303 is first received at load-balancer 302. Load-balancer 302 then reads the packet heading information and routes the second packet 311 to target server 303.

The TCP sequence ID 311 selected by source device 301 can include a new value based on a value of the TCP sequence ID in reply 309 received from target server 303. In some aspects, the value corresponds with a value of the received TCP sequence ID 309 that is incremented based on the size of the data frame received from target server 303. By way of example, if a value associated with TCP sequence ID 309 is 2000, and a previously transmitted frame was 1300 TCP bytes, the TCP sequence ID 311 provided by source device 301 be 3300 (i.e., 2000+1300).

By relying on sequence number identification information in the packet header, load-balancer 302 can make fast routing decisions for incoming packets without the need to frequently/constantly update server-load information. This technique has the advantage of performing efficient traffic balancing between servers that is (initially) informed by server-load information, yet avoids the need to persist load information at load-balancer 302 for smaller and/or short-lived flows.

The disclosure now turns to FIGS. 4 and 5, which illustrate example network devices and computing devices, such as switches, routers, load-balancers, client devices, and so forth.

FIG. 4 illustrates an example network device 410 suitable for performing switching, routing, load-balancing, and other networking operations. Network device 410 includes a central processing unit (CPU) 462, interfaces 468, and a bus 415 (e.g., a PCI bus). When acting under the control of appropriate software or firmware, the CPU 462 is responsible for executing packet management, error detection, and/or routing functions. CPU 462 preferably accomplishes all these functions under the control of software including an operating system and any appropriate applications software. CPU 462 may include one or more processors 463, such as a processor from the INTEL X86 family of microprocessors. In some cases, processor 463 can be specially designed hardware for controlling the operations of network device 410. In some cases, a memory 461, e.g., non-volatile Random Access Memory (RAM), Read Only Memory (ROM), etc. also forms part of CPU 462. However, there are many different ways in which memory could be coupled to the system.

Interfaces 468 may be provided as modular interface cards (sometimes referred to as “line cards”). As such, they can control the sending and receiving of data packets over the network and sometimes support other peripherals used with the network device 410. Among the interfaces that can be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, Digital Subscriber Line (DSL) interfaces, token ring interfaces, and the like. In addition, various very high-speed interfaces can be provided such as fast token ring interfaces, wireless interfaces, Ethernet interfaces, Gigabit Ethernet interfaces, Asynchronous Transfer Mode (ATM) interfaces, High-Speed Serial Interface (HSSI) interfaces, Packet over SONET/SDH (POS) interfaces, Fiber-Distributed Data Interfaces (FDDI), and the like. Generally, these interfaces may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in some instances, volatile RAM.

Independent processors may control such communications intensive tasks as packet switching, media control, signal processing, crypto processing, and management. By providing separate processors for the communications intensive tasks, these interfaces allow CPU 462 to efficiently perform routing computations, network diagnostics, security functions, etc. Although the system shown in FIG. 4 is one specific network device of the present invention, it is by no means the only network device architecture on which the present invention can be implemented. For example, an architecture having a single processor that handles communications as well as routing computations, etc., is often used. Further, other types of interfaces and media could also be used with the network device 410.

Regardless of the network device's configuration, it may employ one or more memories or memory modules (including memory 461) configured to store program instructions for the general-purpose network operations and mechanisms for roaming, route optimization and routing functions described herein. The program instructions may control the operation of an operating system and/or one or more applications, for example. The memory or memories may also be configured to store tables such as mobility binding, registration, and association tables, etc. Memory 461 could also hold various software containers and virtualized execution environments and data.

Network device 410 can also include an application-specific integrated circuit (ASIC), which can be configured to perform routing and/or switching operations. The ASIC can communicate with other components in network device 410 via bus 415, to exchange data and signals and coordinate various types of operations by network device 410, such as routing, switching, and/or data storage operations, for example.

FIG. 5 illustrates a computing system 1100 wherein the components of the system are in electrical communication with each other using a connection 1105, such as a bus. System 1100 includes a processing unit (CPU or processor) 1110 and a system connection 1105 that couples various system components including the system memory 1115, such as read only memory (ROM) 1120 and random access memory (RAM) 1125, to the processor 1110. System 1100 can include a cache of high-speed memory connected directly with, in close proximity to, or integrated as part of the processor 1110. System 1100 can copy data from the memory 1115 and/or the storage device 1130 to the cache 1112 for quick access by the processor 1110. In this way, the cache can provide a performance boost that avoids processor 1110 delays while waiting for data. These and other modules can control or be configured to control the processor 1110 to perform various actions. Other system memory 1115 may be available for use as well. Memory 1115 can include multiple different types of memory with different performance characteristics. processor 1110 can include any general purpose processor and a hardware or software service, such as service 11132, service 21134, and service 31136 stored in storage device 1130, configured to control the processor 1110 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. The processor 1110 may be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

To enable user interaction with computing system 1100, an input device 1145 can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. An output device 1135 can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems can enable a user to provide multiple types of input to communicate with system 1100. The communications interface 1140 can generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

Storage device 1130 is a non-volatile memory and can be a hard disk or other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs) 1125, read only memory (ROM) 1120, and hybrids thereof.

Storage device 1130 can include services 1132, 1134, 1136 for controlling the processor 1110. Other hardware or software modules are contemplated. Storage device 1130 can be connected to system connection 1105. In one aspect, a hardware module that performs a particular function can include the software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 1110, connection 1105, output device 1135, and so forth, to carry out the function.

For clarity of explanation, in some instances the present technology may be presented as including individual functional blocks including functional blocks comprising devices, device components, steps or routines in a method embodied in software, or combinations of hardware and software.

In some embodiments the computer-readable storage devices, mediums, and memories can include a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and/or information created during methods according to described examples include magnetic or optical disks, flash memory, Universal Serial Bus (USB) devices provided with non-volatile memory, networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprise hardware, firmware and/or software, and can take any of a variety of form factors. Typical examples of such form factors include laptops, smart phones, small form factor personal computers, personal digital assistants, rackmount devices, standalone devices, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures.

Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples, as one of ordinary skill would be able to use these examples to derive a wide variety of implementations. Further and although some subject matter may have been described in language specific to examples of structural features and/or method steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or acts. For example, such functionality can be distributed differently or performed in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

Claims

1. A computer-implemented method comprising: receiving a first packet with an access request;determining a load for each of a plurality of servers;forwarding the access request to a target server selected from among the plurality of servers based on the load for each of the plurality of servers, the access request configured to cause the target server to issue a reply with a sequence number selected from among a set of available sequence numbers corresponding with a sequence identifier of the target server, andreceiving a second packet with a header containing the sequence number, the sequence number incremented based on an amount of data transmitted from the target server.

2. The computer-implemented method of claim 1, further comprising: routing the second packet to the target server based on the sequence number.

3. The computer-implemented method of claim 2, wherein the routing of the second packet to the target server includes: identifying the target server as a destination for the second packet based on the sequence number; andforwarding the second packet to the target server.

4. The computer-implemented method of claim 1, wherein each of the plurality of servers is associated with a unique set of sequence numbers.

5. The computer-implemented method of claim 4, wherein the unique set of sequence numbers are Transmission Control Protocol (TCP) sequence numbers.

6. The computer-implemented method of claim 1, wherein the reply includes at least one packet for which a packet header contains the sequence number.

7. The computer-implemented method of claim 1, further comprising: determining when the sequence number for a corresponding stream exceeds a range of sequence numbers corresponding with the target server.

8. A system comprising: one or more processors;a network interface coupled to the one or more processors; anda computer-readable storage medium coupled to the one or more processors, the computer-readable storage medium containing instructions which, when executed by the one or more processors, cause the system to perform operations including: receiving a first packet with an access request;determining a load for each of a plurality of servers;forwarding the access request to a target server selected from among the plurality of servers based on the load for each of the plurality of servers, the access request configured to cause the target server to issue a reply with a sequence number selected from among a set of available sequence numbers corresponding with a sequence identifier of the target server; andreceiving a second packet with a header containing the sequence number, the sequence number incremented based on an amount of data transmitted from the target server.

9. The system of claim 8, wherein the operations include: routing the second packet to the target server based on the sequence number.

10. The system of claim 9, wherein the routing of the second packet to the target server includes: identifying the target server as a destination for the second packet based on the sequence number; andforwarding the second packet to the target server.

11. The system of claim 8, wherein each of the plurality of servers is associated with a unique set of sequence numbers.

12. The system of claim 11, wherein the unique set of sequence numbers are Transmission Control Protocol (TCP) sequence numbers.

13. The system of claim 8, wherein the reply includes at least one packet for which a packet header contains the sequence number.

14. The system of claim 8, wherein the operations include: determining when the sequence number for a corresponding stream exceeds a range of sequence numbers corresponding with the target server.

15. A non-transitory computer-readable storage medium with instructions stored therein which, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving a first packet with an access request;determining a load for each of a plurality of servers;forwarding the access request to a target server selected from among the plurality of servers based on the load for each of the plurality of servers, the access request configured to cause the target server to issue a reply with a sequence number selected from among a set of available sequence numbers corresponding with a sequence identifier of the target server; andreceiving a second packet with a header containing the sequence number, the sequence number incremented based on an amount of data transmitted from the target server.

16. The non-transitory computer-readable storage medium of claim 15, wherein the operations include: routing the second packet to the target server based on the sequence number.

17. The non-transitory computer-readable storage medium of claim 16, wherein the routing of the second packet to the target server includes: identifying the target server as a destination for the second packet based on the sequence number; andforwarding the second packet to the target server.

18. The non-transitory computer-readable storage medium of claim 15, wherein, each of the plurality of servers is associated with a unique set of sequence numbers, andthe unique set of sequence numbers are Transmission Control Protocol (TCP) sequence numbers.

19. The non-transitory computer-readable storage medium of claim 15, wherein the reply includes at least one packet for which a packet header contains the sequence number.

20. The non-transitory computer-readable storage medium of claim 15, wherein the operations include determining when the sequence number for a corresponding stream exceeds a range of sequence numbers corresponding with the target server.