Patent Application
20170251045
The present disclosure relates to communication data networks and their use in the efficient transmission of large data files from a source to one or more recipients.
In accordance with one embodiment, a method is provided for efficiently routing a file from a file source to be accessed by one or more file recipients connected by a plurality of paths in one or more networks. For each file recipient the method evaluates one or more predetermined criteria to select the most efficient one of the plurality of paths to use for routing the file. The method routes the file to each file recipient using the most efficient one of the plurality of paths. In a preferred embodiment, one of the paths is a direct Peer-to-Peer (P2P) transfer from a file source to a file recipient.
In one implementation, the selected path is between the file source and an origin server and from the origin server to one or more file recipients.
The criteria may include one or more criteria from the group consisting of cost, coverage, speed, security, location of second users and availability of the file recipient.
The routing may comprise delivering and storing the file on a server of a content delivery network, and the selection of the content delivery network is made based on additional predetermined criteria comprising a minimum bandwidth usage, a fixed monthly price and a history of monthly usage.
A status monitor may be used to query and maintain a status of file sources and file recipients.
A file recipient may request the file and the file routing system by initiating a transfer of the file from the most efficient path.
A network performance enhancement protocol may be implemented on at least one of the plurality of paths, with the network performance enhancement protocol being used to further improve the performance of at least one of the plurality of paths and the performance improvement may be taken into consideration in the criteria used to select the most efficient path.
The foregoing and additional aspects and embodiments of the present disclosure will be apparent to those of ordinary skill in the art in view of the detailed description of various embodiments and/or aspects, which is made with reference to the drawings, a brief description of which is provided next.
The foregoing and other advantages of the disclosure will become apparent upon reading the following detailed description and upon reference to the drawings.
While the present disclosure is susceptible to various modifications and alternative forms, specific embodiments or implementations have been shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the disclosure is not intended to be limited to the particular forms disclosed. Rather, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of an invention as defined by the appended claims.
In the absence of CDNs, the file source 130 transmits the file directly 191 to an origin server 190 to create a copy 195. The file recipients 140.1 . . . 140.n can then request the file directly 192 from origin server 190.
Each time the file is updated or modified, the file source 130 re-transmits the file 130 to the origin server.
As the numbers of file sources and file recipients grow, the load on the origin server increases and slows performance. Performance issues are further aggravated by longer distances and larger files when these recipients are spread over a large geographic area.
A CDN 100.1 . . . 100.n is a globally distributed network of proxy servers deployed in multiple data centers. The goal of a CDN is to serve content to end-users with high availability and high performance. The use of CDN generally involves contractual agreements based on the amount of bandwidth and/or storage used. Some contracts may include a fixed monthly fee for a minimum amount of bytes used and a variable fee based on the number of bytes used that exceed the minimum amount. The fixed and variable fee can be quite high depending on the type of CDN. Typically, CDNs are meant to be used when there is a large community of users sharing medium-size files. When the number of users sharing the file is smaller and the files are extremely large (e.g., movies), the use of CDNs may not always be the optimal sharing solution.
Content Delivery Networks (CDNs) offer an alternative by storing cached copies of the files 130 on a plurality of servers spread over a large geographic area. In this case, either the file source 130 sends the file 135 directly 111.1 . . . 111.m to one or more CDNs 100.1 . . . 100.m or the origin server 190 sends its copy of the file 195 directly 193.1 . . . 193.m to the CDNs. Each CDN is then responsible for distributing the file across its own collection of servers.
When a file recipient 140.1 . . . 140.n requests the file 135, the request is redirected to an appropriate CDN (choice may be based on location) which responds directly 112.1 . . . 112.m with a local copy of the file 105.1 . . . 105.m.
Since the CDN uses a large number of servers at varying locations in the world that are sharing the task of serving this file, the response times are significantly improved over the case where the origin server 190 is solely responsible for delivering the file to all recipients. As mentioned above, the use of CDNs involves significant costs associated with both the storage and distribution of files.
The file routing system 200 domain comprises one or more file router server 240, one or more computing platforms acting as file sources 230 equipped with a client software 234 and one or more computing platforms acting as file recipients 235.1 . . . 235.n also equipped with a client software 239.1 . . . to 239.n. In addition, the file routing system 200 maintains a file location list 270 which provides a cross-reference of each file along with the network addresses of all nodes (file sources and file recipients) within the system that have a stored copy of the file. The system uses a status monitor 245 which ascertains whether file recipients are presently active on the network using bidirectional network messaging 246.1 . . . 246.n. A statistic collector module 249 may also be used to monitor the performance of the system and optionally adjust parameters. An administration control module 248 is also provided to configure the system parameters and optimization criteria, such as the utility and cost functions used for the multi-constraint optimization as described herein.
The file routing system 200 is designed to optimize the performance and minimize costs for each file transfer and for each recipient or group of recipients based on a set of criteria comprising, for example, one or more of file size, number of recipients, locations of recipients, availability of recipient (as established by the status monitor 245), speed of transfer, cost and security. The system may vary the cost of using the CDNs with time of month based on the amount of bandwidth used on the CDN. For example, if the CDN contract has a minimum bandwidth usage of 1 TB at a fixed price, then as the month goes by, the system attempts to make sure the 1 TB is all used but not exceeded.
If the recipient is currently available, then the file source 230 may be instructed, if this is the most optimal option based on the criteria, to send the file directly to the recipient using one of several Peer-to-Peer (P2P) protocols known in the art 280.1 . . . 280.n resulting in copies of the files in each recipient 235.1 . . . 235.n.
If deemed more efficient based on the criteria, updated copies of the file in recipients may then be further transmitted using a P2P protocol from one file recipient to another instead of from the file source (paths depicted as 281.1 . . . 281.n).
Another possible option, if the file routing server 240 itself is, for example, within the same Local Area Network (LAN) or Wide Area Network (WAN), then the source 230 may be directed to send 241 the file to the file routing server 240 where it can be more efficiently retrieved by some of the intended recipients.
If the file is sent directly to the recipient 235.1 . . . 235.n, the copy of the file 265 on the origin server 190 may also be updated at a lower priority by either the file source 230 or a file recipient 235.1 . . . 235.n, allowing any other file recipient to retrieve the file 260.
Based on the criteria, it may be established that it would be more optimal to use the origin server 190 and/or one or more of the CDNs 100.1 to 100.n. The file source 230 would then update a copy of its file 260 on any number of these delivery systems. Subsequently, any file recipients can then retrieve the file 260.
The file routing system 200 then updates the file location list 270 to reflect all file transfers.
As there may be a plurality of copies and corresponding paths for a file recipient to retrieve a copy of a required file, the file routing system 200 uses a statistics collector 249 to compile data on, for example, network usage, bandwidth consumed for each CDN and occupancy of the origin server.
Upon the initial request for file 260 (implemented as a network message), the file routing system assesses the criteria to determine the optimal path for the file transfer for each receiver. The process for determining the optimal path is the same as described above.
As described above, it may be using a P2P protocol from the original source 230 via network path 380, from a previous recipient 235.1 . . . 235.n, from the file routing server 240, or the origin server 190. As before, the choice is made according to the criteria.
If a file recipient requests a file that is in the process of being transferred, the file recipient may retrieve the partial file from one location (e.g., origin server) and get the rest of the file using a more efficient path (e.g., P2P).
Optionally, the file routing system 200 may only be used to route files of a size exceeding a pre-determined threshold while the files of smaller size are routed using a fixed path.
The file routing system may prepare plans (criteria, thresholds, etc.) offline using linear programming optimization/simulation based on historical usage data. The plan is periodically loaded into the file routing system to update the criteria and the optimization algorithm.
Network performance enhancement protocols (NPEC) such as TCP acceleration protocols well known in the art, such as the ones described in U.S. Pat. Nos. 8,630,204 and 9,143,454 and other layer 3 acceleration protocols as described in U.S. Pat. Nos. 8,437,370, 9,189,307, 7,742,501, 8,548,003, 9,953,114 and 8,009,696 may be used to improve the performance of one or more of the network paths 111a . . . 111n, 112a . . . 112n, 191, 192, 180a . . . 180n. The file routing system may take into account whether one or more NPEC are enabled on one or more path as one of or more of the criteria to establish the optimal path.
Although the algorithms described above including those with reference to the foregoing flow charts have been described separately, it should be understood that any two or more of the algorithms disclosed herein can be combined in any combination. Any of the methods, algorithms, implementations, or procedures described herein can include machine-readable instructions for execution by: (a) a processor, (b) a controller, and/or (c) any other suitable processing device. Any algorithm, software, or method disclosed herein can be embodied in software stored on a non-transitory tangible medium such as, for example, a flash memory, a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), or other memory devices, but persons of ordinary skill in the art will readily appreciate that the entire algorithm and/or parts thereof could alternatively be executed by a device other than a controller and/or embodied in firmware or dedicated hardware in a well known manner (e.g., it may be implemented by an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable logic device (FPLD), discrete logic, etc.). Also, some or all of the machine-readable instructions represented in any flowchart depicted herein can be implemented manually as opposed to automatically by a controller, processor, or similar computing device or machine. Further, although specific algorithms are described with reference to flowcharts depicted herein, persons of ordinary skill in the art will readily appreciate that many other methods of implementing the example machine readable instructions may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.
It should be noted that the algorithms illustrated and discussed herein as having various modules which perform particular functions and interact with one another. It should be understood that these modules are merely segregated based on their function for the sake of description and represent computer hardware and/or executable software code which is stored on a computer-readable medium for execution on appropriate computing hardware. The various functions of the different modules and units can be combined or segregated as hardware and/or software stored on a non-transitory computer-readable medium as above as modules in any manner, and can be used separately or in combination.
While particular implementations and applications of the present disclosure have been illustrated and described, it is to be understood that the present disclosure is not limited to the precise construction and compositions disclosed herein and that various modifications, changes, and variations can be apparent from the foregoing descriptions without departing from the spirit and scope of an invention as defined in the appended claims.
