Patent Application
20050108373
The present invention pertains generally to computer networks and, more particularly, to peer-to-peer computer networks.
Enabling group communication is a popular application of computer networks. Groups of people use computer networks to share every kind of digital data from simple text and static images to encoded audio and video and more specialized data that enables real-time collaboration and multi-player games.
In general, a node is an application on a peer-to-peer network, typically identified by an Internet Protocol (IP) address, a port, and a name. A graph is an interconnection of such nodes on a peer-to-peer network. A peer-to-peer network is a simple, small type of network in which each node has the ability for equivalent capabilities and responsibilities. Each node can be a server and each can be a client at the same time. This differs from a client-server architecture in which some computers are dedicated to serving others.
In a peer-to-peer network, there are many time-based events. For many events, it is preferred that all nodes connected to the network share a common view of time. Such concepts as data lifetime, membership lifetime, graph partition detection, and graph partition recovery all hinge upon the fact that the nodes in the peer-to-peer network have a common concept of time among themselves.
As one example of a problem that may occur if nodes do not have a common concept of time can be found in data expiration. It is convenient if data across the network expires for each node at approximately the same time. In general, for distributed networks such as a peer-to-peer network, data will be pushed to the graph and will be removed from each node based upon an expiration time. If the expiration times are not approximately the same, then the data may exist on some nodes, and may be expired on others. This phenomenon may cause a number of problems, such as synchronization issues between the nodes.
One method for handling common time between nodes is use of Network Time Protocol (NTP). NTP is a protocol that can synchronize clocks on local computers with radio or atomic clocks on the Internet. While this system works well for its intended purpose, constantly synchronizing all nodes on a graph can be an expensive process. In addition, not all nodes may be connected to the Internet, and thus may not be able to take advantage of NTP, or a node may not be configured to synchronize with an NTP server.
The following presents a simplified summary of some embodiments of the invention in order to provide a basic understanding of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some embodiments of the invention in a simplified form as a prelude to the more detailed description that is presented later.
In accordance with an embodiment, a shared concept of time, called “graph time,” is agreed upon by all nodes within a graph. Each node maintains its own delta representative of a difference in the node's system clock and graph time. Upon connecting to a neighbor node in a graph, a node is sent graph time as perceived by the neighbor node. In accordance with an embodiment, the node assumes that the lag time in receiving the graph time from the neighbor node is one-half of the lag time between the request for the connection and the receipt of the graph time. Graph time as sent by the neighbor node is added to one-half of the total delay and the sum is subtracted from the node's system time to calculate the delta for the node requesting connection.
In accordance with an embodiment, if a node's delta has never been set, it uses the calculated value as its new delta. However, in accordance with an embodiment, if the node's delta has been set in the past, it uses a weighted average of its current calculated delta and the previously calculated deltas.
In accordance with an embodiment, a shift factor is utilized to minimize the delta for all nodes in a graph. In this manner, the average of the delta values for each of the nodes is minimized. In accordance with an embodiment, in order to provide this decrease in delta, during a shifting period, the calculated delta is decreased by a fraction of the time between the present connection and the previous connection, divided by the shift period. If the time between connections is greater than the shift period, then the delta is set to zero.
Other features of the invention will become apparent from the following detailed description when taken in conjunction with the drawings, in which:
Prior to proceeding with a description of the various embodiments of the invention, a description of a computer and networking environment in which the various embodiments of the invention may be practiced is now provided. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, programs include routines, objects, components, data structures and the like that perform particular tasks or implement particular abstract data types. The term “program” as used herein may connote a single program module or multiple program modules acting in concert. The terms “computer” and “computing device” as used herein include any device that electronically executes one or more programs, such as personal computers (PCs), hand-held devices, multi-processor systems, microprocessor-based programmable consumer electronics, network PCs, minicomputers, tablet PCs, laptop computers, consumer appliances having a microprocessor or microcontroller, routers, gateways, hubs and the like. The invention may also be employed in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, programs may be located in both local and remote memory storage devices.
Referring now to the drawings, in which like reference numerals represent like parts throughout the several views, an example of a computer networking environment suitable for incorporating aspects of the invention is described with reference to
Referring to
The computer 102 may also have additional features/functionality. For example, computer 102 may also include additional storage (removable 208 and/or non-removable 210) including, but not limited to, magnetic or optical disks or tape. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, including computer-executable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer 102. Any such computer storage media may be part of computer 102.
The computer 102 preferably also contains communications connections 212 that allow the device to communicate with other devices such as remote computers 214. A communication connection is an example of a communication medium. Communication media typically embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. By way of example, and not limitation, the term “communication media” includes wireless media such as acoustic, RF, infrared and other wireless media. The term “computer-readable medium” as used herein includes both computer storage media and communication media.
The computer 102 may also have input devices 216 such as a keyboard/keypad, mouse, pen, voice input device, touch input device, etc. Output devices 218 such as a display 220, speakers, a printer, etc. may also be included. All these devices are well known in the art and need not be described at length here.
In the description that follows, the invention will be described with reference to acts and symbolic representations of operations that are performed by one or more computing devices, unless indicated otherwise. As such, it will be understood that such acts and operations, which are at times referred to as being computer-executed, include the manipulation by the processing unit of the computer of electrical signals representing data in a structured form. This manipulation transforms the data or maintains it at locations in the memory system of the computer, which reconfigures or otherwise alters the operation of the computer in a manner well understood by those skilled in the art. The data structures where data is maintained are physical locations of the memory that have particular properties defined by the format of the data. However, while the invention is being described in the foregoing context, it is not meant to be limiting as those of skill in the art will appreciate that various of the acts and operation described hereinafter may also be implemented in hardware.
In an embodiment of the invention, joining a peer-to-peer (P2P) communications group includes joining a graph of connected peers or nodes. Each node may be, for example, an application running on a computer (e.g., the computer 102).
A single computer may enable more than one peer, for example, peer 302, peer 304 and peer 306 may interact with the computer network 104 from the same computer 102. A single peer may be enabled by more than one computer, for example, peer 312 may interact with the computer network 104 from a distributed computing environment including several computers 102. Peer-to-peer connections are independent of an underlying data transport mechanism, for example, the connection between peer 314 and peer 308 may be implemented utilizing Transmission Control Protocol (TCP) and Internet Protocol Version 4 (IPv4), the connection between peer 314 and peer 312 may be implemented utilizing TCP and Internet Protocol Version 6 (IPv6), and the connection between peer 302 and peer 304 may be ultimately implemented as a copy of one memory location to another within the same computer 102. Each peer 302, 304, 306, 308, 310, 312, 314 may participate in more than one peer-to-peer graph (not shown in
The node 400 includes a graph time module 402 associated therewith. Alternatively, the graph time module 402 may be a component of an operating system of the computer 102, or may reside as a separate program on the computer 102 or on a separate computer.
The graph time module 402 and/or the node 400 are configured to communicate with a system clock 404 for the computer 102. The system clock 404 may maintain time for the computer 102, may be implemented in hardware or software, and may reside separately from the computer 102 or as part of the node 400.
The system clock 404 for each node 400 in the graph 300 maintains its own time. In accordance with an embodiment of the invention, the system clocks 404 may maintain time in Universal Coordinated Time (UTC). As is known, UTC is the same as Greenwich Mean Time (GMT), and is the mean solar time of the meridian of Greenwich, England, used as the basis for calculating standard time throughout the world. As an example, the system clocks 404 associated with each node or peer 302-314 may maintain time expressed as a 64 bit number, where each unit represents 100 nanoseconds relative to GMT.
Although each system clock 404 utilizes the same system for maintaining time, most likely in any given peer-to-peer network, such as the graph 300, the system clocks 404 for the nodes or peers 302-314 are not synchronized. In the example shown in
In accordance with an embodiment of the present invention, a concept of graph time is defined wherein the nodes or peers 302-314 have a shared concept of time that is agreed upon by all nodes 400 within the graph 300. As further described below, each node 302-314 maintains its own delta representative of a difference in the node's time kept by its system clock 404 and graph time.
In accordance with an embodiment, graph time is initialized by the node 400 which creates the graph 300. Graph time is set according to the system clock 404 of the node 400. As an example, for the graph 300 in
In accordance with an embodiment of the invention, the other nodes 302-308, 312 and 314 share the concept of graph time with the node 310 by maintaining a local delta 406 (
Therefore, each node 400 stores its delta 406 which is used to convert between graph time (“graph_time”) and the time maintained in the system clock 404 of the node (“system_time”), such that:
graph_time=system_time−delta
As can be understood, the delta may be negative or positive, depending on where the system time for the node 400 is relative to graph time.
In accordance with an embodiment of the invention, the local delta 406 for a node 400 is set or adjusted each time a new outgoing connection is formed. The delta 406 may be calculated at other times, such as on an interval, but calculation at connection time works particularly well in that it requires very little additional overhead for the graph 300.
Typically, when forming a connection, a connect message 500 (
In accordance with an embodiment of the invention, the node 400 that is requesting a connection calculates its delta by using the perceived graph time 504 sent by the neighbor node 408 in the connection attempt. However, because the perceived graph time 504 represents the graph time perceived by the neighbor node 408 when the welcome message 502 was queued to be sent by the neighbor node, and there is some lag between that queue time and the actual receipt of the perceived graph time 504 by the node 400, the node 400 is not exactly certain of the perceived graph time 504 relative to its own system time. In other words, the node 400 is not certain of the exact latency of the connection with the neighbor node 408, and cannot determine exactly how long it has been since the neighbor node 408 has created the perceived graph time 504.
In accordance with an embodiment of the invention, to account for the unknown latency, an assumption is made that the latency is one half of the time between the request for the connection and the receipt of the graph time. The node 400 tracks the time in which it queues its connect message 500 (system_CONNECT), and then the local system time at which the welcome message 502 is processed (system_WELCOME). Because the difference in these two times represents a round trip between the node 400 and the neighbor node 408, in accordance with an embodiment, it is assumed that one-half of that period is the amount of time that it took from the queuing of the welcome message to the processing of the welcome message (system_WELCOME time). Using this information, a delta value (“neighbor_delta”) may be calculated using the perceived graph time (“graph_time”) of the neighbor node 408, the system time of the node 400 (“system_time”), and the delay in the round trip between requesting the connection and processing the welcome message 502 as follows:
neighbor_delta=system_time−(graph_time+½ delay)
The calculation above calculates a “neighbor_delta” because the delta is calculated based upon the perception of the graph time of the neighbor node 408.
The neighbor_delta may be calculated at the time of the processing of the welcome message 502 (i.e., at system_WELCOME). As such, system_time at that instance is equal to system_WELCOME, and the formula for calculating neighbor_delta may be rewritten as follows:
neighbor_delta=system_WELCOME−((system_WELCOME−system_CONNECT)/2+graph_time)
At step 604, the graph time module 402 calculates the neighbor_delta as described above. If this connection attempt is a first connection attempt by the node 400, then in accordance with an embodiment of the invention, step 606 branches to step 608 where the neighbor_delta is set as the local delta for the node 400 (i.e., it is stored as the delta 406). If this is not the first connection attempt, then step 606 branches to step 610, where a weighted average of the current local delta 406 and the neighbor_delta is used to calculate the new local delta.
The weighted average of step 610 above may be calculated by many different methods. In one embodiment, calculation of the weighted average takes into account the fact that a local delta already exists for the node 400, and that the neighbor_delta is calculated based upon the perceived graph time of a single neighbor node 408. Thus, the existing local delta is be given a heavier weighting than the neighbor_delta. As one example of how to perform such a weighted average, the following formula may be used:
new local delta=old local delta*((x−1)/x)+neighbor_delta*(1/x))
X is some weighting constant, such as in the range from 4-16. If the weighting constant is too small, the neighbor_delta will be weighted too heavily, and if too large will not have much effect on the new local delta value. Thus, a weighting from 4-16, or even more preferably 4-8 is typically considered to be a good value. The value of 4-8 works particularly well in that most graphs, such as the graph 300, the number of neighbors is typically 3-4, and if the value is 4, then the new local delta will generally include neighbor_delta information from each of its neighbors, and with 8 will take the last 8 connections into account, which increases the likelihood that connections with each of its neighbors will be utilized in averaging. Other values may be used in accordance with a particular graph or a desired result.
Thus, in accordance with the method described above, local deltas 406 are calculated for each of the peers, such as the peers 302-314. A representation of local deltas 406 for the nodes 302-314 is shown in
As an example of how the graph time can be beneficial, creation, modification, and expiration times of shared data in the graph 300 may be set in accordance with graph time. An application may set the expiration time by calculating when the data should expire in terms of its own system time, and then convert it to normalized graph time by utilizing the graph time module 402. To this end, an application programming interface 410 (
When the expiration time of the data is reached, each node 400 having the data stored therewith removes the shared data from its cache or other data store. If the nodes 400 do not have a shared concept of time through the use of graph time as described herein, each node 400 would expire the data at different times, which could cause each node to have a different view of the shared data. As such, problems may occur, such as inaccurate synchronization between the nodes.
As can be seen in
For example, a modified algorithm may utilize the methods shown in
shifted_delta=local_delta*((shift_period−(system_time−last_set))/shift_period)
wherein the shift_period is the time period defined above (e.g., one day), the system_time is the current connection time in which the shifted_delta is being calculated, and the last_set is the previous connection time.
The local_delta in the formula above may be calculated using a number of different factors, but typically is the local delta 406 last calculated for the node 400. In accordance with an embodiment, if the local delta has never been calculated for the node 400 (i.e., in the example above, upon first connection), then the local_delta in the shifting formula above may be calculated using the neighbor_delta calculation above, and the last_set time may be set to system_time. Thus, in such a case, the shifted_delta is equal to the neighbor_delta.
If local delta has been set in the past, then the local delta may be calculated using a weighting average as follows:
local_delta=shifted_delta*((x−1)/x)+neighbor_delta*(1/x))
The shifting factor described above is advantageous in that it minimizes the delta for each of the nodes 400 in a graph (e.g., the graph 300). This feature assures that the graph time remains close to Universal Coordinated Time (UTC), and prevents drifting of the graph time away from UTC. An example of the deltas after shifting has occurred in an ideal graph 300 is shown in
The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate embodiments of the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.
