Patent Application
20080114882
The present invention generally relates to computers and data communication and, more particularly, to communicating data between a server and multiple clients.
Client-Server applications are commonplace due to the wide availability of networked computers. The client-server concept is a type of distributed network architecture, which enables a client, often a single user computer, or a program running on the same, to make requests to a server, which is often a multi-user computer or a program running on the same. In an environment with multiple clients, each client can send requests to a single server, and the server will process the requests and typically generate responses back to the individual clients. Servers can vary in type being a file server, a web server, an application server, a terminal server, a mail server, etc. While each of these servers have a different purpose, they all have the same basic architecture.
Many computer communications use this client-server model, especially in Internet-based applications where web servers serve web pages and files to web browsers running on client computers. The terms client and server may also relate to two computer processes that are in communication with each other, where one client process typically makes a request for information, for example, a particular web page from a website, and the server process returns a response with the desired information. Often, a prerequisite step required before a client process can make request to a server process is the establishment of a connection between the client and server processes. Then, once the client process connects to the server process, both the client and server computers can use this connection to communicate data until the connection is closed.
A common way that computers communicate over a network and even over the Internet is to use TCP (Transmission Control Protocol), which was developed to get data from one networked device or computer to another. TCP/IP is a set of protocols including TCP that was developed for the Internet in the 1970s for data transmission. HTTP is a newer protocol that is used for browsing web site pages on the Internet. HTTP is a method used to transfer or convey information on the World Wide Web. A web browser running on a client utilizes protocols, such as TCP, to establish connections with the web server and will use the HTTP protocol over the connection to communicate and transfer data between the client and server computers.
The endpoints of the processes that connect to communicate between the client and server are known as sockets. Each of the client and server processes will typically establish its own socket for communication over the process connection.
The establishment of sockets and connections however, can be problematic, particularly for servers that are handling requests from large numbers of clients. Creating a new TCP socket for an inbound TCP connection (i.e., passive open) is often extremely CPU intensive and consumes considerable server memory resources. When a user requests a web page, a single client browser may initiate numerous TCP connections and thus may expend significant resources in connection with opening and closing these nonpersistent connections. HTTP/1.1 persistent connections may reduce the connection overhead by allowing multiple HTTP requests per client connection; however, allowing idle sockets to remain open for long periods of time can drain server resources. So either alternative, short nonpersistent connections and long persistent connections, can significantly affect web server performance and capacity.
A common solution for this problem is to place a TCP multiplexing device in front of the web server to consolidate incoming HTTP requests to reduce the number of times server connections have to be made. A TCP multiplexing device often significantly reduces the number of open server sockets by multiplexing numerous client connections into fewer persistent server connections. The connection multiplexing ratio can be as high as 350 client connections to 1 server connection, significantly off-loading client connection processing from the web server. Since the persistent server connections are shared by many client connections they can remain active indefinitely without draining server resources.
A conventional TCP multiplexing device uses a layer-7 reverse proxy to consolidate multiple client HTTP requests into a single connection to the web server. The proxy device receives client requests, consolidates them and applies logic to the opening and closing of server connections. HTTP/1.1 pipelining may be exploited because HTTP/1.1 allows multiple HTTP requests to be sent on a single connection without waiting for the corresponding responses. The requester then waits for the responses to arrive in the order in which they were requested. The connection between the TCP multiplexing device and the server can remain open without significantly draining server resources because the connection may be continually reused with other client requests.
The benefits of TCP multiplexing are quite obvious; however, the additional network complexity and cost may be prohibitive for less sophisticated and/or budget conscious customers. Even if the device itself is economically priced, adding technical staff to manage the increased network complexity is not inexpensive.
Software has been utilized to integrate TCP multiplexing into an operating system kernel and thus provide an alternative to an outboard TCP multiplexing device. An internal loopback interface may be used by the client software on a computer to communicate with server software on the same computer. However, multiplexing through a proxy to an internal loopback server connection allocates two TCP endpoint objects for each server connection, one for the proxy side and one for the server side. As a result, proxying through a loopback connection adds additional path length and consumes valuable server resources.
Each of the above mentioned solutions have one common problem, that problem being that each of the solutions requires additional hardware or additional server resources, adding both complexity and/or cost to the solution. Accordingly, there is a need in the art for an improved way of multiplexing client connections without adding cost or draining server resources.
The invention addresses these and other problems associated with the prior art by providing an apparatus, program product, and method that utilize socket multiplexing to multiplex and demultiplex data from and to multiple client connections established in a server. By multiplexing multiple connections through a socket, embodiments consistent with the invention are often able to increase a server's workload capacity without adding the expense, effort and complexity that is present otherwise in conventional designs that use external hardware or loopback connections. In fact, in many embodiments consistent with the invention, there are no additional hardware requirements and the solution may be fully integrated into server's kernel, resulting in a lower overall cost and reduced maintenance of overhead.
These and other advantages and features, which characterize the invention, are set forth in the claims annexed hereto and forming a further part hereof. However, for a better understanding of the invention, and of the advantages and objectives attained through its use, reference should be made to the Drawings, and to the accompanying descriptive matter, in which there is described exemplary embodiments of the invention.
These and further features of the present invention will be apparent with reference to the following description and drawings wherein:
It should be understood that the appended drawings are not necessarily to scale, presenting a somewhat simplified representation of various features illustrative of the basic principles of the invention. The specific design features of the sequence of operations as disclosed herein, including, for example, specific dimensions, orientations, locations, and shapes of various illustrated components, will be determined in part by the particular intended application and use environment. Certain features of the illustrated embodiments may have been enlarged or distorted relative to others to facilitate visualization and clear understanding. In particular, thin features may be thickened, for example, for clarity or illustration.
The embodiments described hereinafter utilize a method for multiplexing multiple client requests for multiple data connections in a single socket to a server. The server processes the requests without being encumbered by the additional overhead associated with each connection having its own socket. A server can be any device that connects with multiple devices to communicate data. Situations where this might be applicable are web server applications where servers are receiving a continued stream of client requests for data. Servers may also execute client applications that require data or information from another server. Here the server executing the client application becomes a client to a new server. A server can be both a server and client simultaneously as well, when the client application executing on the server requires data from a server application that is also executing on the same or different server.
Turning to the drawings, wherein like numbers denote like parts throughout the several views,
Computer 8 typically includes at least one processor 26 coupled to a memory 16. Processor 26 may represent one or more processors (e.g. microprocessors), and memory 16 may represent the random access memory (RAM) devices comprising the main storage of computer 8, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or backup memories (e.g. programmable or flash memories), read-only memories, etc. In addition, memory 16 may be considered to include memory storage physically located elsewhere in computer 8, e.g., any cache memory in a processor 26, as well as any storage capacity used as a virtual memory, e.g., as stored on a mass storage device (not shown) or another computer coupled to computer 8 via a network 24.
Computer 8 also typically receives a number of inputs and outputs for communicating information externally. For interface with a user or operator, computer 8 typically includes one or more user input devices 10 (e.g., a keyboard, a mouse, a trackball, a joystick, a touchpad, a keypad, a stylus, and/or a microphone, among others). Computer 8 may also include a display 12 (e.g., a CRT monitor, an LCD display panel, and/or a speaker, among others). The interface to computer 8 may also be through an external terminal connected directly or remotely to computer 8, or through another computer communicating with computer 8 via a network 24, modem, or other type of communications device.
Computer 8 operates under the control of a kernel 22 and operating system 20 (which may be separate components, or alternatively may be considered to be the same component), and executes or otherwise relies upon various computer software applications, components, programs, objects, modules, data structures, etc. (e.g. web application 18). Moreover, various applications, components, programs, objects, modules, etc. may also execute on one or more processors in another computer coupled to computer 8 via a network 24, e.g., in a distributed or client-server computing environment, whereby the processing required to implement the functions of a computer program may be allocated to multiple computers over a network.
In general, the routines executed to implement the embodiments of the invention, whether implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions will be referred to herein as “computer program code”, or simply “program code”. The computer program code typically comprises one or more instructions that are resident at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, causes that computer to perform the steps necessary to execute steps or elements embodying the various aspects of the invention. Moreover, while the invention has and hereinafter will be described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of computer readable media used to actually carry out the distribution. Examples of computer readable media include but are not limited to physical, recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., CD-ROM's, DVD's, etc.), among others, and transmission type media such as digital and analog communication links.
In addition, various program code described hereinafter may be identified based upon the application or software component within which it is implemented in specific embodiments of the invention. However, it should be appreciated that any particular program nomenclature that follows is merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature. Furthermore, given the typically endless number of manners in which computer programs may be organized into routines, procedures, methods, modules, objects, and the like, as well as the various manners in which program functionality may be allocated among various software layers that are resident within a typical computer (e.g., operating systems, libraries, APIs, applications, applets, etc.), it should be appreciated that the invention is not limited to the specific organization and allocation of program functionality described herein.
Those skilled in the art will recognize that the exemplary environment illustrated in
Socket multiplexing as described herein is capable of providing a performance competitive integrated alternative to an outboard TCP multiplexing device. Socket multiplexing consolidates multiple client connections into a single persistent socket without the need for proxying through a loopback connection. In addition, in some embodiments, socket multiplexing may support concurrent pipelining of requests from multiple clients into a single persistent socket, by exploiting HTTP/1.1, for example. With pipelining, if a client sends multiple back-to-back requests on a single client connection, subsequent requests may be queued until the response is served for the current request.
In some embodiments, load balancing algorithms may be used with socket multiplexing to evenly distribute client connections across persistent sockets and enforce any socket sharing limits. Server based socket multiplexing may also be combined with an outboard TCP multiplexing device to even further optimize the server performance and capacity.
The discussion hereinafter will focus on data flow between a client data connection and a server. On the server side, as shown in the exemplary hardware and software environment of
Referring now to
The block diagram in
In contrast to conventional TCP/IP communication stacks, where there is a one to one relationship between the socket and TCP connection, multiple client TCP connections 106a, 106b, 106c are attached to and associated with the same persistent socket 100. Note, the TCP Connection may only be associated with a socket during a request/response exchange in some embodiments. That is, a different socket may be selected for subsequent HTTP requests in embodiments that utilize load balancing.
The flow diagram in
The HTTP request is passed to a socket multiplexer incorporated in the socket in block 524, which performs blocks 526 to 532. In an alternate embodiment, a load balancer may be utilized to multiplex over more than one socket. The load balancer in block 526 selects a socket with the fewest number of active HTTP requests. If the socket share limit has been reached on all of the available sockets, a new socket will need to be created. To create a new socket, a passive open is initiated to the server, causing the server to accept a new socket (not shown). The TCP connection handle is enqueued to the socket's demultiplexer queue in block 528 to facilitate the subsequent demultiplexing of the corresponding HTTP response back to the correct TCP connection. Sockets remain open until the HTTP server closes them. The server may close a socket for a number of reasons. One reason to close the socket may be that the socket is the configured for a maximum number of requests per connected socket and that maximum has been reached. Another reason may be that the server may close a socket that has received no requests for specified period of time. Another reason may be that the HTTP server has ended which results in closing all of the sockets. Regardless of the reason, it is typically the HTTP server that determines when to close a socket.
The HTTP request is enqueued to the socket in block 532 followed by an enqueue of the socket to the I/O completion port in block 534. Note, in an alternate embodiment the HTTP Server could communicate without an I/O completion port (a.k.a., asynchronous I/O); however, asynchronous I/O significantly improves server capacity. The asynchronous I/O component dispatches an HTTP Server thread in block 536 to process the HTTP request.
The HTTP Server in block 538 parses the HTTP request, via execution of blocks 540 to 546. First the HTTP server starts processing the request in bock 540. If the request is complete (“Yes” branch from decision block 542), a CGI program (or another program suitable for handling the request) is called to generate the dynamic page (i.e., HTTP outbound data response). Otherwise in block 546, the HTTP Server waits for the remaining portion of the HTTP request to be received.
The block diagram in
The flow diagram in
The HTTP response splitter in block 608 determines the start and end of each pipelined response so that each individual response can be demultiplexed to different client TCP connections.
The demultiplexer in block 610 finds the TCP connection identified by the first TCP connection handle in the socket's demultiplexing queue. Blocks 612 to 616 are performed for the demultiplexing. Note, the TCP connection handle was enqueued to the socket's demultiplexing queue in block 528 in
In an embodiment where the inbound data request is not an HTTP/1.1 persistent request, the HTTP Response Converter in block 618 performs the necessary conversions to ensure the HTTP outbound data response is compatible with the client, executing blocks 620 to 623. When the first or only fragment of the HTTP response is processed (“Yes” branch from decision block 620), conversion in block 622 to the HTTP response header may be necessary to ensure compatibility with the client connection. For example, the Connection header value is changed to “Close”, if the original request was nonpersistent. When the last or only response fragment is processed (“Yes” branch from decision block 624) and a request was queued in block 518 in
The HTTP outbound data response is sent to the TCP protocol processing in block 628 and IP protocol processing in block 630, then the Network Driver transmits the response to the client in block 632.
The following example for the present embodiment illustrates the initialization of a socket incorporating socket multiplexing consistent with the invention. A socket with multiplexing is initialized and idled (shown in
Referring now to
Next, as illustrated by block 750, a SYN (synchronization) packet is received on port 80 to establish a new HTTP connection to the HTTP server. TCP finds the listening socket at block 752 created by the HTTP server. TCP then allocates a new TCP connection endpoint at block 754 to begin the connection establishment. TCP sends a SYN ACK packet (a packet message used in the TCP protocol to acknowledge receipt of a packet) back to the HTTP client at block 756. The HTTP client then returns an ACK packet (TCP acknowledgement) to complete the connection establishment. The new TCP connection is added to the listening socket's accept queue at block 758. Because socket multiplexing is enabled, the multiplexer is notified of the new connection at block 760 waiting in the accept queue. The multiplexer immediately accepts the new connection at block 762. Without multiplexing, the HTTP server typically accepts the connection; however, giving each new connection to the HTTP server is undesirable, because the goal is to multiplex multiple connections into a single connected socket. TCP removes the new connection from the accept queue at block 764. The new connection is returned to the multiplexer to facilitate socket multiplexing. At this point the connection is ready to receive HTTP requests from the client.
Referring now to
Next, the HTTP server's waiting accept thread must be woken up at block 812, passing the newly created connected socket. Note that unlike a conventional connected socket, multiple TCP endpoints are associated with this connected multiplexed socket. The HTTP server's accept thread dispatches a receive thread to call the socket receive function to receive subsequent HTTP requests at block 814 from multiple TCP connections. Alternative embodiments may use asynchronous (I/O completion ports) or non-blocking I/O to accelerate performance and scalability. Now that a connected socket has been created (selected), the multiplexer prepares to pass the HTTP/1.1 request to the connected socket at block 816. First, the TCP connection endpoint handle is enqueued to the tail of the socket's demultiplexer queue at block 818, to enable the HTTP/1.1 response to be properly demultiplexed back to the correct TCP connection endpoint. Second, the share count is incremented at block 820 to facilitate load balancing, in an embodiment that utilizes load sharing. Next the HTTP/1.1 request is passed to the connected socket at block 822. The receive thread is woken up to pass the HTTP/1.1 request to the HTTP server. The HTTP server processes the static or dynamic request and prepares to send an HTTP/1.1 response. The HTTP server issues another receive operation to wait for another request on the connected socked.
Referring now to
From the forgoing disclosure and detailed description of certain illustrated embodiments, it will be apparent that various modifications, additions, and other alternative embodiments are possible without departing from the true scope and spirit of the present invention. For example, it will be apparent to those skilled in the art, given the benefit of the present disclosure that a socket multiplexer can work with a variety of different communication protocols and data connections for any type of served data environment. The embodiments that were discussed were chosen and described to provide the best illustration of the principles of the present invention and its practical application to thereby enable one of ordinary skill in the art to utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the present invention as determined by the appended claims when interpreted in accordance with the benefit to which they are fairly, legally, and equitably entitled.
