Patent Application
20140019572
None.
None.
Not applicable.
None.
The invention generally relates to methods for detecting an opportunity to increase messaging efficiency in computing systems, and especially to methods for improving efficiency of Remote Direct Memory Access protocols in applications which transmit short payloads.
Remote Direct Memory Access (RDMA) is a method by which an application running in a first server may directly read, write, or both read and write data in memory which resides with a second server, usually where the first and second servers are remote to each other. By mimicking the DMA functions in a processor board in which the processor is not required to move data into the processor prior to writing it to a new location, RDMA provides similar functionality between applications on servers wherein the first application (the initiator) may access data in the target memory without disturbing the processing of the target computer. One very widely supported protocol for RDMA is Request for Comments 5040 (RFC5040) of the Network Working Group of the International Engineering Task Force (IETF), the most recent version having been drafted by R. Recio, et al.
The term “initiator” is used to describe a machine, process, application or client which wishes to access data on another machine, server, or platform.
The machine, server or platform where the data actually resides is referred to as a “target”. If a “read” operation is to be performed, the data flow is from the target to the initiator. If a “write” operation is to be performed, the data flow is from the initiator to the target.
Data which is being remotely read or written is carried within a payload area of the RDMA protocol. Referring to
Generally speaking, there are a number of headers (TCP, MPA, DDP, and RDMA) which precede transmission of the actual data in the payload, followed by the check value (cyclic redundancy check or CRC).
While other implementations of RDMA may include control data for other protocols,
Byte utilization is improved in Remote Direct Memory Access (RDMA) communications by detecting a plurality of concurrent messages on a plurality of application sockets which are destined for the same application, client or computer, intercepting those messages and consolidating their payloads into larger payloads, and then transmitting those consolidated messages to the destination, thereby increasing the payload-to-overhead byte utilization of the RDMA transmissions. At the receiving end, multiplexing information is used to unpack the consolidated messages, and to put the original payloads into a plurality of messages which are then fed into the receiving sockets to the destination application, client or computer, thereby making the consolidation process transparent between the initiator and the target.
The description set forth herein is illustrated by the several drawings.
a shows the general structure of RDMA over TCP according to at least one open standard.
b illustrates the overhead-heavy scenario of RDMA over TCP when the payload contains much less data than the overhead (headers).
The inventors of the present and the related invention have recognized problems not yet recognized by those skilled in the relevant arts regarding the capacity of remote direct memory access (RDMA) to provide high performance data communications. The inventors have realized that because each of the RDMA communications transactions carries an overhead inefficiency, therefore computer applications (“applications”) that exchange small size messages incur limited benefit from use of RDMA. For example, referring to
Furthermore, while Server Virtualization is a compelling technological advance in computing, especially for Data Centers because it allows higher server utilization, flexibility, dynamic infrastructure and enables improved operational economics, but use of RDMA with virtual servers is not currently possible.
IBM's™ Smart Analytics™ applications and other business analytics applications exchange a plurality of concurrent messages with other application servers, such as database servers, to complete a transaction. The application messages can range from very small (ie. 64B) to larger sizes (ie. >64K) but have a request/response characteristic for a specific transaction. Such a system, for example, typically has multiple concurrent transactions in progress to the same destination application server. Concurrency from the application is achieved via the use of multiple sockets. But, the inventors have discovered that when RDMA is used for such applications, considerably system performance overhead is incurred on a per transaction basis due to RDMA's inefficiency of handling short messages (e.g. large overhead to short-payload ratio in the RDMA protocol).
The inventors further recognized that the known solutions to this problem include modifying the application programs to utilize the communications sockets from kernel space, instead of using sockets from the user space of the operating system. The present inventors have identified at least three serious limitations to this approach:
And, bearing in mind that RDMA does not allow application deployment in virtual servers as previously mentioned, then the use of RDMA for certain types of applications such as business analytics applications because problematic, inefficient and undesirable.
In response to recognizing these unaddressed problems in the art, the present inventors are disclosing an invention which solves both problems by making RDMA transactions more efficient even for applications which use a great number of short messages, and by making RDMA-enable applications deployable on Virtual Servers.
The problem at hand is that application server workloads can benefit from circumventing the high CPU costs of small concurrent message transactions to a common destination by a coalescing and aggregating short messages across associated application sockets at the Open Systems Interface (OSI) Session Layer (i.e. Layer 5) inside of the operating system's kernel.
Initially, the RDMA connection establishment of each socket is performed via the traditional process in which each end (initiator device and target device) of the communication session is aware of the connection. Afterward the session is established across multiple sockets (to support concurrent messaging), the OS kernel can discover via an out of band mechanism which specific application sockets are associated with a common source and common destination. The associated sockets are then re-associated with a single kernel-level RDMA connection using a single socket, in which the single connection represents an aggregation group of sockets on top of a kernel RDMA connection. We'll refer to the aggregation mechanism as Application Communication Protocol Module (ACPM) for the purposes of the present disclosure.
Because concurrent socket operations from the application programs will provide an aggregation (multiplexing) of messages to a single kernel RDMA connection, the short messages sent using RDMA are consolidated into larger messages which take benefit from larger RDMA transaction (e.g. percentage of CPU and communication bandwidth consumed by message overhead is lower for the larger messages). This results in several benefits:
After an encapsulation of a message grouping to a common destination has been performed, it is clear that the destination system must be ACPM RDMA aware. On the destination side, the ACPM module unpacks (demultiplexes) and routes each shorter (original) message to the appropriate ingress socket to the destination application.
This new approach allows application deployment in virtual servers. Further, large message sizes enable efficient RDMA usage from Virtual Servers to Virtual I/O Server.
Turning now to
Referring now to
The ACPI modules communicate to each other through an out-of-band process to detect and determine with messages flowing through multiple sockets at a given time represent a session between the two applications, such as the initiator application (214) and the target database (204, 205). Responsive to this being detected, the short-payload messages are intercepted within the kernel and aggregated (multiplexed) on the sending end into larger messages, so as to produce a larger payload-to-overhead byte usage ratio (e.g. more efficient use of the bandwidth). The larger messages can then be transmitted on fewer sockets, such as just over a single socket, to the destination. On the receiving end of the session, the larger messages are unpacked or demultiplexed into shorter messages, and routed into their original user sockets so that the receiving application may receive the shorter messages at their expected sockets normally (e.g. making the multiplexing and demultiplexing transparent to the application).
As one can see from the diagram, the aggregation and unpacking can be performed in either direction of data flow, for either a remote read or a remote write, with similar benefit.
And, the method of multiplexing and demultiplexing can take any number of forms, such as embedding within the larger payload one or more tags, words or flags which separate the original shorter messages and retain information about the original socket on which they were transmitted at the source end of the session. Such information is then used on the receiving end of the session to separate the larger payload into the shorter messages and to route the shorter messages to their appropriate destination sockets.
Referring to
The preceding paragraphs have set forth example logical processes according to the present invention, which, when coupled with processing hardware, embody systems according to the present invention, and which, when coupled with tangible, computer readable memory devices, embody computer program products according to the related invention.
Regarding computers for executing the logical processes set forth herein, it will be readily recognized by those skilled in the art that a variety of computers are suitable and will become suitable as memory, processing, and communications capacities of computers and portable devices increases. In such embodiments, the operative invention includes the combination of the programmable computing platform and the programs together. In other embodiments, some or all of the logical processes may be committed to dedicated or specialized electronic circuitry, such as Application Specific Integrated Circuits or programmable logic devices.
The present invention may be realized for many different processors used in many different computing platforms.
Many such computing platforms, but not all, allow for the addition of or installation of application programs (501) which provide specific logical functionality and which allow the computing platform to be specialized in certain manners to perform certain jobs, thus rendering the computing platform into a specialized machine. In some “closed” architectures, this functionality is provided by the manufacturer and may not be modifiable by the end-user.
The “hardware” portion of a computing platform typically includes one or more processors (504) accompanied by, sometimes, specialized co-processors or accelerators, such as graphics accelerators, and by suitable computer readable memory devices (RAM, ROM, disk drives, removable memory cards, etc.). Depending on the computing platform, one or more network interfaces (505) may be provided, as well as specialty interfaces for specific applications. If the computing platform is intended to interact with human users, it is provided with one or more user interface devices (507), such as display(s), keyboards, pointing devices, speakers, etc. And, each computing platform requires one or more power supplies (battery, AC mains, solar, etc.).
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof, unless specifically stated otherwise.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed.
Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
It should also be recognized by those skilled in the art that certain embodiments utilizing a microprocessor executing a logical process may also be realized through customized electronic circuitry performing the same logical process(es).
It will be readily recognized by those skilled in the art that the foregoing example embodiments do not define the extent or scope of the present invention, but instead are provided as illustrations of how to make and use at least one embodiment of the invention. The following claims define the extent and scope of at least one invention disclosed herein.
