This invention relates to a method and system for communicating data and, more particularly, to a method and system for synchronizing historical data between a first node and second node without the retransmission of data.
In today's data networks, individual blocks of information, called “packets,” are transmitted between computers or other equipment which are interconnected via some physical medium. More specifically, a multi-layered networking model is generally employed which abstracts discrete network-related functions into separate layers in order facilitate independence of functions, and allow for incremental advancement of features and capabilities.
Internet Protocol (IP) based networks, including the Internet, conform to this multi-layered model. For example, a computer communicating over the Internet may utilize copper wire and electronic transceivers as a layer 1 (physical layer) technology, Ethernet for layer 2 (data link layer) technology, and IP for layer 3 (network layer) technology. Additional higher layers are typically employed as well. These can include transmission control protocol (“TCP”), hypertext transfer protocol (“HTTP”), messaging application programming interface (“MAPI”), and common internet file system (“CIFS”), among others.
When communicating, the computer transmits and receives packets of information to and from other computers, usually by way of intermediate networking devices such as switches, routers, modems, and the like. At each device along the path, processing is performed such that the information packet can be relayed to its final destination. This intermediate processing is usually, although not always, limited to layers 1 through 3.
In today's environments, the speed with which communicating computers can exchange information is often limited by the performance of the network infrastructure connecting them. While the processing latency of intermediate networking devices is one factor limiting performance, by far the greatest factor typically limiting network performance is the speed with which packets can be transmitted over the physical layer between devices. This limitation is most prevalent in wide area networks, where last-mile links are often over low-speed copper facilities operating at or below T1 rates (1.5 Mbps).
One approach for increasing the rate of information transfer over wide area networks with low speed links is to perform data compression on the packets being transmitted. Data compression can remove redundant information from the packets, thereby allowing higher effective information transmission rates over a given link. However, because packet compression schemes are typically performed on layer 2 (e.g., Frame Relay) or layer 3 (e.g., IP) packets, they cannot benefit from the reliability afforded by higher layer protocols such as TCP. Therefore, these layer 2 or layer 3 compression schemes must somehow accommodate the fact that transmitted packets may be lost or received out of order between the steps of compression and decompression.
For this reason, known packet-oriented data-compression systems have typically taken one of two approaches for dealing with intervening packet loss and reordering. The first is to simply not maintain any dependency of compression processing between packets. In other words, no historical data or other stateful information is maintained from one packet to the next and each packet is compressed (and thus decompressed) independently. While this removes all need for compressor-to-decompressor synchronization and is free of effects from lost, corrupted, or out-of-order packets, it has the considerable drawback of suffering from poor compression performance.
The second approach taken is to ensure the reliable transmission of compressed packets between the compressor and the decompressor. This approach enables the maintenance of historical data across packets, and thus greatly improves compression performance. However, it can require the implementation of a reliable protocol comprising session initialization and packet accounting and retransmission, effectively duplicating the processing already being done at the higher transport layer.
Accordingly, there is a need for methods and apparatus to synchronize compressor and decompressor processing of packets of data without duplicating the processing itself or compromising compression quality.
Embodiments of the present invention provide an improved approach whereby historical data can be maintained between the compressor and decompressor (thus improving compression performance), but without the complexity and overhead of maintaining a reliable protocol in between. Other embodiments of the invention provide for the consistent maintenance of historical data between paired operators such as, e.g., an encryptor and a decryptor, etc.
In particular, the present invention allows for synchronized updates to historical reference data utilized by two communicating nodes in a network without the need to maintain a reliable transport protocol between the nodes. The technique may be applied to a compressing node which maintains a history buffer containing data from packets which it has compressed, discrete segments or windows of which are available as a reference for compressing new data; and a decompressing node which maintains a history buffer containing data from packets which it has decompressed and which is available as a reference for decompressing new data. Employing this technique, the compressing node receives acknowledgement indications from the decompressing node describing which windows of new data have been successfully decompressed by the decompressing node. In response to these indications, the compressing node makes the corresponding windows in its history buffer available for subsequent compression operations. In this way, the compressing node utilizes for compression only that data which is also available to the decompressing node for subsequent decompression, ensuring consistent use of evolving historical data without the need for a reliable protocol between the nodes.
The invention is amenable to applications beyond compression. For example, it may be applied to packet encryption and decryption, forward error correction, and other operations involving complementary processing at transmitting and receiving nodes.
With reference to
In some embodiments, the method further comprises storing, accessibly to the first node, processing data from the processed packet. In one embodiment, the processing at the first node comprises compression and the processing at the second node comprises decompression. When the process involves compression and decompression, the method may also include storing, accessibly to the second node, decompression data from the decompressed packet. The method may further comprise transmitting an acknowledgment, from the second node to the first node, indicating successful decompression at the second node of the transmitted packet. The acknowledgment may be transmitted form the second node to the first node over a wide area network. Furthermore, the data used for compressing packets may only be associated with compressed packets previously transmitted to the second node and for which acknowledgments were received.
The processing at the first node may instead comprise encryption and the processing at the second node may comprise decryption. Moreover, the processed packet may be transmitted from the first node to the second node over a wide area network. In some embodiments, the method may further comprise transmitting, from the second node to the first node, an acknowledgment indicating successful processing of the transmitted packet at the second node.
In another aspect, a device for transmitting data packets over a computer network comprises a packet buffer; a history buffer for storing processing data from processed packets; a processing module for processing packets from the packet buffer based at least in part on data in the history buffer; and a communication module for transmitting, to destination nodes, packets processed by the processing module.
The processing module may be a compression module, wherein the processing comprises compression, a decompression module, wherein the processing comprises decompression, an encryption module, wherein the processing comprises encryption, or a decryption module, wherein the processing comprises decryption. In some embodiments, the communication module transmits over a wide area network. The communication module may be further configured to receive an acknowledgment from a destination node indicating successful processing of a packet transmitted by the device. The acknowledgment may be received from the destination node through a wide area network. Moreover, the data used by the processing module may be drawn from packets for which acknowledgements have been received.
In some embodiments, the communication module is further configured to receive compressed packets from other nodes, and the device further comprises a decompression history buffer and a decompression module for decompressing packets received from other nodes based at least in part on data in the decompression history buffer and storing decompression data from decompressed packets in the decompression history buffer. In these embodiments, the device may further comprise functionality for transmitting an acknowledgment to another node upon receiving a compressed packet therefrom.
The foregoing and other objects, features, and advantages of the present invention, as well as the invention itself, will be more fully understood from the following description of various embodiments, when read together with the accompanying drawings, in which:
The ensuing discussion focuses on the representative application of compression and decompression, but once again, it should be understood that the invention can be applied to other forms of complementary packet processing.
Referring to
In this embodiment, the history buffer 128 is of finite length, and once it is full, the oldest packet data is removed as new packet data is added. Furthermore, the history buffer 128 is structured to contain N discrete data windows, each of which is of L bytes in length. The compressor 124 also maintains a committed flag C(i) for each window. The flag C(i) indicates whether the packet data residing in history buffer window(i) is available as reference data for a compressing a packet. During such a compression operation, the input packet P 108 is processed using data from available history buffer windows in conjunction with a prescribed compression algorithm to produce a compressed packet CP 120 as output. Contemporaneously, the compressor 124 adds the data from the original packet P 108 to the history buffer 128. The compressed packet CP 120 contains header information indicating where in the history buffer 128 the data from the original packet 108 was added. The compressed packet CP 120 is then transmitted over the WAN 112 to the second networked device 116.
Again referring to
In this embodiment, the history buffer 140 is also structured to contain N discrete data windows, each of which is of L bytes in length. During such decompression operation, the input compressed packet CP 120 is processed using data from history buffer windows in conjunction with a prescribed decompression algorithm to produce the original packet P 108 as output. Contemporaneously, the decompressor 136 adds the data from the output original packet P 108 to the history buffer 140, at the location indicated in the header of the compressed packet CP 120. When the decompressor 136 determines that all original packet data has been added to a particular window (i) of its history buffer 140, it generates an acknowledgement packet ACK 144 indicating that window's data is now available for subsequent decompression operations. The ACK packet 144 is then transmitted over the WAN 112 back to the first networked device 104.
Still referring to
Further Considerations
To provide reliable operation, complementary packet operators (e.g., such as the compressor and decompressor discussed above) should work in concert to handle various manifestations of poor network performance. These include: lost packets, re-ordered packets, corrupted packets, and long transmission latency between the packet operators. To address these issues, embodiments of the present invention include the following features:
Protocol Structures
As discussed above, the compressor 124 processes an input packet P 108 and generates a corresponding output compressed packet CP 120 for transmission over the WAN 112 to the decompressor 136. The compressed packet CP 120 contains header information to control the update and use of data in its history buffer 128 for compressing subsequent packets. In addition, the decompressor 136 generates acknowledgement packets, ACKs 144, for transmission back to the compressor 124 for this same purpose. An exemplary structure for the CP 120 and ACK 144 packets is shown in
Referring to
Again referring to
Basic Procedures
To operate in accord with the present invention, an embodiment providing compression typically implements the following five basic procedures:
By the Compressor:
By the Decompressor:
For further illustration, a pseudo-code representation of each of these steps is given below.
This application claims the benefit of U.S. Provisional Patent Application No. 60/704,856, filed on Aug. 2, 2005, which is hereby incorporated by reference as if set forth herein in its entirety.
Number | Date | Country | |
---|---|---|---|
60704856 | Aug 2005 | US |