1. Field of the Invention
This invention relates to transporting data and more particularly relates to transmitting data packets containing variable-length data strings to a remote device, and reconstructing the data string from received data packets.
2. Description of the Related Art
In the field of data transport, including peer to peer networks, server-client networks, wide area networks (WANs), telephone communication networks, laser communication networks, wireless communication networks, and the more broad networks of the internet (world wide web) which may include some or all of these components, there is a limit to the number of bits that can be transported in one group. Strings of data that need to be transported via such networks are often longer, in terms of bits, than the maximum transmission unit allowed by the network. One type of data transport protocol commonly used to resolve this issue involves fragmenting the data string into shorter, more manageable strings.
Current string fragmentation protocols can become problematic in several ways. To correctly reconstruct the string, the string fragments must be received in the same sequence in which they were sent. In many networks, specifically those that are inherently lossy, such as wireless networks, the string fragments may not only be received out of sequence, they may be lost completely. In such a case, the string cannot be reconstructed correctly and errors may occur, or alternatively, the string must be fragmented again and retransmitted using additional bandwidth. Additionally, the receiving device does not have any information regarding the size of the data string until the last packet is received. Therefore, it cannot accurately allocate space on a storage device sufficient to reconstruct the entire data string.
Consequently, a need exists for a process, apparatus, and system that efficiently transport variable-length data strings by allowing the receiving device to accurately allocate sufficient storage space to reconstruct the data string and correctly reconstruct the data string regardless of the sequence in which the data packets arrive.
The present invention has been developed in response to the present state of the art, and in particular, in response to the problems and needs in the art that have not yet been fully solved by currently available data transport protocols. Accordingly, the present invention has been developed to provide a process, apparatus, and system for concurrent high-performance transport of variable-length messages that overcome many or all of the above-discussed shortcomings in the art.
The apparatus for concurrent high-performance transport of variable-length messages is provided with a logic unit containing a plurality of modules configured to functionally execute the necessary steps of fragmenting a data string into a plurality of data packets, transmitting the data packets, receiving of the data packets by a remote device and reconstructing the data string. These modules in the described embodiments include a string fragmentation module, a data transmitter module and a data receiver module.
The string fragmentation module of the apparatus, in one embodiment, is configured to fragment a data string and package the string fragments into data packets, each configured with the length of the data string. The data packets are preferably configured with additional control information including an identifier linking the data packet to the data string, the data packet sequence number, and an identifier indicating the last data packet in the sequence. The apparatus is further configured, in one embodiment, to calculate the number of data packets required to completely transmit the data string.
In a further embodiment, the apparatus includes a data transmitter module configured to transmit the data packets to a remote device via a connection employing a data transfer protocol. The data transmitter may also be configured to create threads to handle the concurrent transmission of multiple data packets from different clients.
In one embodiment, the apparatus is configured with a data receiver module which, upon receipt of one of the data packets, accurately allocates enough space on a storage device to completely reconstruct the data string. The data receiver module is preferably configured to insert the data packets sequentially into an array of arrays as they are received, and reconstruct the data string upon receipt of the final data packet. In an alternative embodiment, the data packets are inserted into an array of linked lists. The apparatus performs efficiently by eliminating the need to reallocate storage space, and move the data packets in the storage device, if insufficient space was allocated. This embodiment eliminates wasted space created during the space allocation process.
A system of the present invention is also presented for concurrent high-performance transport of variable-length messages. The system may be embodied as a network of devices configured with the data transport apparatus described herein. The system, in one embodiment, includes a transmitter device configured to fragment a data string into a plurality of data packets, each configured with the length of the data string. In one embodiment, the transmitting device packages control information including an identifier linking the data packet to the data string, the sequence number of the data packet, and an identifier indicating the last packet in the sequence with the data string fragment. In one further embodiment, the transmitting device is configured to calculate the number of packets required to completely transmit the data string. Additionally, the transmitting device may be configured to create threads to handle concurrent transmission of multiple data packets, and transmit the data packets via a connection employing a data transfer protocol to a remote device.
The system may further include a receiving device configured, in one embodiment, to receive data packets from a transmitting device, allocate space on a storage medium responsive to the length of the original data string included with the first received data packet, and reconstruct the data string from the data packets. In one embodiment, the receiving device accurately allocates enough storage space to reconstruct the data string upon receipt of one of the list fragments. In this embodiment, the data receiver inserts the incoming list fragments sequentially into an array, the sequence being determined by the sequence number sent with the data packet. The data string is reconstructed upon receipt of all of the data packets.
A process of the present invention is also presented for concurrent high-performance transport of variable-length messages. The process in the disclosed embodiments substantially includes the steps necessary to carry out the functions presented above with respect to the operation of the described apparatus and system. In one embodiment, the process includes fragmenting a data string into data packets, packaging the size of the data string with every data packet, transmitting the multiple data packets to a remote device, allocating space on a storage medium for string reconstruction and reconstructing the data string from the data packets on a remote device. The process also may include specifying a pointer to the location of the data string to be fragmented, and an identifier of the process on a remote device to receive the data string.
In a further embodiment, the process includes calculating the number of data packets required to completely transmit the data string. The string fragment is preferably packaged with control information including an identifier linking the data packet to the data string, the length of the original data string, the fragment sequence number, and an identifier indicating the last data packet in the sequence. In one embodiment, threads are created to handle concurrent transmission of multiple data packets. The data packets may be transmitted to a remote device via a connection employing a data transfer protocol.
In one embodiment of the process, the remote device allocates enough storage space to reconstruct the data string responsive the receipt of the first data packet. In a further embodiment, the data packets are inserted sequentially into an array until the final data packet is received. In this embodiment, the data string is reconstructed upon receipt of the final data packet. The process increases data transport reliability by sending the sequence number with every data packet, and reduces system resource waste by sending the size of the data string with every packet, thus eliminating any issues with allocation of storage space for reconstruction of the data string on the remote device.
Reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the present invention should be or are in any single embodiment of the invention. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussion of the features and advantages, and similar language, throughout this specification may, but do not necessarily, refer to the same embodiment.
Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize that the invention can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the invention.
These features and advantages of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as set forth hereinafter.
In order that the advantages of the invention will be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:
Many of the functional units described in this specification have been labeled as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
Modules may also be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.
Furthermore, the described features, structures, or characteristics of the invention may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
In one embodiment, the connection between the transmitting device 105 and the receiving device 110 is a wire connection. Any wire configuration may be used to connect the transmitting device 105 to the receiving device 110. Alternatively, the connection between the transmitting device 105 and the receiving device 110 may be a conducting strip embedded in an insulating or semi-conducting substrate. In an additional alternative embodiment, the connection between the transmitting device 105 and the receiving device 110 is wireless, the physical connection being radio or light waves. The connection between the transmitting device 105 and the receiving device 110 may also be fiber optic. Although these specific embodiments of the connection have been communicated, they are not intended to limit the embodiments of connections that can be made between the transmitting device 105 and the receiving device 110.
In one embodiment of the system 300 depicted, the connection between the first peer device 305 and the second peer device 310 is a single point to point connection. In an alternative embodiment, additional peer devices are added to the peer to peer network 300. In each of these embodiments, any physical connection may be made between the first peer device 305 and the second peer device 310 as set forth in
In one embodiment, the primary device 405 is a data server or application server. In an alternative embodiment the primary device 405 is a component of a network that is given priority as master device or controller of the network. Additionally, the secondary device 410 may be a work station, a data storage device, a slave device, or user of the network.
Each data packet is preferably configured to include control information including an identifier linking the data packet to the data string, the size of the data string, the sequence number of the data packet and the identifier indicating the last packet in the sequence. In a certain embodiment, a process thread is generated 830 by the thread generator module 625 to handle the transmission of the data packet. The packets are then transmitted 715 concurrently to a remote device via the transmit interface module 630.
In this embodiment, the remote device receives the data packets 840 via the receive interface module 635. The control information is then stripped 845 from the data packet. If the data packet is the first received 850 for a particular data string, space is accurately allocated 720 on a data storage module 620 by the space allocation module 640. The string fragment is then placed 860 sequentially in an array by index number. If the data packet is not the first received 850 for a particular data string, the fragment is placed 860 directly into the array in order by sequence number. In one embodiment, the string fragments are placed 850 in an array of arrays.
In an alternative embodiment, the string fragments are placed 850 in an array of linked lists. In an additional alternative embodiment, the array only contains pointers to the locations elsewhere in the data storage device 620, in which the string fragments have been place 850. Upon receipt 840 of the final packet 865, the data string is reconstructed 725 by the string reconstruction module 645. In one embodiment, the number of elements in the first array is determined by the level of activity within the receiver, or the number of outstanding data strings. In an alternative embodiment, the data string is compiled and stored as a whole entity. The data string is then passed to the target process 875 as specified 810 by the data string identifier module 605. The method 800 may be carried out by computer readable code.
Although for purposes of clarity the method 800 is depicted in a certain sequential order, execution may be conducted in parallel and not necessarily in the depicted order.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.