Patent Application
20090006489
In a collection of computing devices, a data item may be multiply replicated to create a number of copies of the item on the different computing devices and/or possibly within a single device. An item may be any stored data object, such as for example contact or calendar information, stored pictures or music files, software application programs, files or routines, etc. The collection of computing devices may for example be a desktop computer, a remote central server, a personal digital assistant (PDA), a cellular telephone, etc. The group of all such items and replicas where the items are stored may be referred to as a distributed collection.
In many cases, a user would like all of their various data storing devices to have the latest updated information without having to manually input the same changes into each device data store. Replication, or synchronization, of data is one process used to ensure that each data store has the same information. Synchronization protocols are the means by which devices exchange created and updated versions of items in order to bring themselves into a mutually consistent state. The periodicity of the sync may vary greatly. Networked devices may sync with each other frequently, such as once every minute, hour, day, etc. Alternatively, devices may sync infrequently, such as for example where a portable computing device is remote and disconnected from a network for a longer period of time. Whether the synchronization is frequent or infrequent, the distributed collection is said to be weakly-consistent in that, in any given instant, devices may have differing views of the collection of items because items updated at one device may not yet be known to other devices.
As an example, a user may maintain an electronic address book or a set of email messages in a variety of different devices or locations. The user may maintain the address book or email addresses, for example, on a desktop computer, on their laptop computer, on a personal digital assistant (PDA) and/or mobile phone. The user may modify the contact information or send/receive email addresses using applications associated with each location. Regardless of where or how a change is made, one goal of replication is to ensure that a change made on a particular device or in a particular location is ultimately reflected in the data stores of the other devices and in the other locations.
Synchronization between replicas may be described as a sharing of knowledge between replicas. A common knowledge sharing scheme involves tracking, within each replica, changes that have occurred to one or more items subsequent to a previous replication. One such tracking scheme makes use of version vectors, which consist of list of version numbers, one per replica, where each version number is an increasing count of updates made to an item by a replica. During synchronization, one replica sends version vectors for all of its stored items to another replica, which uses these received version vectors to determine which updated items it is missing. Comparing the version vectors of two copies of an item tells whether one copy is more up-to-date (every version number in the up-to-date copy is greater than or equal to the corresponding version number in the other copy) or whether the two copies conflict (the version vectors are incomparable). The replica may then update its copy of the item if required or make efforts to resolve the detected conflict.
Although version vectors enable replicas to synchronize correctly, they introduce overhead. The version vector of each item may take O(N) space in an N replica replication system, thus requiring O(M*N) space across an M item collection. This space requirement could be substantial if the number of items is large and could approach the size of the items themselves if items are small. Similarly, exchanging version vectors during synchronization consumes bandwidth. Even if two replicas have fully consistent data stores, they still need to send a complete list of version vectors whenever they periodically perform synchronization.
Another knowledge sharing scheme, implemented for example in the WinFS data storage and management system from Microsoft Corp., makes use of knowledge vectors. Unlike version vectors, knowledge vectors are associated with the replicas rather than the items. Each replica keeps a count of the updates it generates, and the knowledge vector of a replica consists of the version number of the latest update it learned from every other replica. In addition, items at a replica have a single version number indicating the latest update applied to it. Replicas exchange knowledge vectors during synchronization, determine and exchange the missing updates, and change their knowledge vector to reflect the newly-learned knowledge (each number is set to the maximum of the corresponding numbers in the two knowledge vectors of the synchronizing replicas).
An example of knowledge sharing between a pair of replicas using knowledge vectors is illustrated with respect to prior art
This means that replica A has knowledge including changes up to the 5th change in replica A, the 3rd change in replica B, and the 7th change in replica C.
Each of the changes indicated in the knowledge vector may be represented in the set of replicated items. For example, assume four items in the collection, identified by unique identifiers i, j, l and m. The set of items stored in data store 24 at Replica A may look as follows:
The data store thus indicates, for a given item, which version was produced when that item was last changed (i.e. the item was created, modified or deleted) as far as this replica is aware, and the data showing the actual updated contents (not shown in Table 1). Thus, for example, replica A knows that the 7th change in replica C was to item j, and it includes the data associated with the change to item j.
Similarly, replica B has a data store 24 and a knowledge vector 26, KB. The knowledge vector in replica B represents what knowledge replica B has about changes that have occurred to items in the collection. For example, knowledge vector KB may have the components:
This means that replica B has knowledge including changes up to the 2nd change in replica A, the 5th change in replica B and the 8th change in replica C. Each of these changes is represented in the set of items stored by replica B.
Referring now to prior art
As this is a one-way synchronization, this ends the sync process resulting from replica A's sync request (in a two way sync, the process would be repeated with replica B receiving changes from replica A and learning what knowledge replica A has). Replica A can update its knowledge vector based on the learned knowledge and received changes to include the recently replicated changes as shown in Replica A in
Knowledge vectors impose substantially lower overhead compared to version vectors. The space required per replica to store knowledge vectors is just O(N+M), including the space required for per item version numbers, compared to O(N*M) for version vectors, where the system has N replicas and the replica has M items. Further more, exchanging knowledge vectors only requires O(N) bandwidth compared to O(N*M) for exchanging version vectors.
While knowledge vectors work well for total replication between replicas, it may happen that one or more replicas are only interested in receiving a certain subset of information. This situation is referred to as partial replication. For example, suppose the data store includes email messages in various folders, including an inbox folder and some number of other folders including, perhaps, folders that contain saved email messages. In some cases a user might want to replicate changes to all of the email folders. For example, this might be desirable when the communications bandwidth between replicating devices is large. In other cases—perhaps when the bandwidth is limited, as it might be at some times with a mobile phone or PDA—the user might only want to replicate changes to a particular folder, like their inbox.
It is also conceivable that a user might want to synchronize only part of their entire set of data in all cases. For example, a user might want to maintain all email on a desktop computer or server, but only synchronize their inbox and a selected set of folders to a small device that has limited storage. In this case, some information may never be synchronized with a particular device.
As another example, for a data store that includes digital music files, a user might want to synchronize their entire digital music library-perhaps they have a portable music player or computer with a large hard drive. They may also have a small portable music player with a limited amount of flash memory, on which they only want to store a selected set of music. In one example, this music to be synchronized might include, for example, digital music files the user has rated with “four stars” or “five stars,” as well as music downloaded in the last week.
In order to allow for partial replication in the above situations, as well as a wide variety of others, a replica may contain a filter. A “filter” may be broadly defined as any construct that serves to identify a particular set of items in a data collection. These items are said to fall within the partial replica's “interest set”. When synchronizing in a partial replication scenario, like in the situations introduced above, various additional problems may occur. These problems include the following:
Efficient knowledge sharing: A partial replica is interested in only a certain subset of items and consequently has knowledge that is limited by its interest set. When a partial replica shares its knowledge with a second replica, the second replica must somehow account for this limitation. This is not a problem for a version vector knowledge sharing scheme, which maintains knowledge about each item separately. However, a knowledge vector knowledge sharing scheme maintains its knowledge vector about the replica as a whole rather than about each item separately. This results in a substantial savings in storage and bandwidth as compared with version vectors, but it also makes it a problem to account for a limited interest set.
Partial information: In order for a replica to eventually learn about an item within its interest set, it requires a synchronization path to all other replicas that are interested in the same item. Moreover, each intermediate replica in the synchronization path must also be interested in the item. Otherwise, a replica may not receive complete information about all the items it is interested in. For example, in
Push outs: When a partial replica updates an item, the updated item may no longer fall within the replica's interest set. Although the partial replica would like to discard such an item, it may find itself in the situation of holding the only copy, in which case discarding the updated item would cause the update to evaporate from the collection. In this situation, the partial replica must “push out” the item to another replica before discarding it. A similar situation can arise when a partial replica alters its filter. For example in
Move outs: When a partial replica is the target of a synchronization, the source replica may be aware of an update to an item for which an old version is stored by the partial replica, but the new version does not fall within the partial replica's interest set. The partial replica needs to be made aware that the item it stores has been updated so as to “move out” of its interest set. For example, in
Reincarnation: When a replica deletes an item, the system needs to ensure that all copies of that item are permanently deleted from the system. If not, the deleted copy might get resurrected at a later point of time based on an old version. Resurrection of deleted items is a concern even without considering partial replicas. Partial replicas add the related problem that an item discarded due to a move-out might be “reincarnated” from an old version synced from an out-of-date replica.
Filter Changes: Finally, replicas may change filters at any time causing some items to move out of the interest set as well as disrupt the path of information flow the replica relies on to learn new items. It is desirable to ensure that filter changes do not disrupt information flow and items discarded during filter changes are completely expunged without the risk of resurrections.
Except for the problem of efficient knowledge sharing, a reason for the above problems is that arbitrary synchronization topologies do not provide a guaranteed path of information flow for replicas. A solution to provide guaranteed information paths is to have one or more replicas serve as reference replicas, which replicate all the items in the system, and have replicas synchronize with a reference replica periodically. However, it may not be always possible for all replicas to synchronize with reference replicas. Moreover, reference replicas may not be reachable at a dire time of need.
The present technology, roughly described, relates to a system using item-set knowledge and a hierarchical arrangement of replicas to allow synchronization of partially-replicated collections while keeping synchronization overhead low by using the concept of item-set knowledge. Item-set knowledge consists of one or more knowledge fragments, which associate knowledge vectors with sets of items, called item-sets, instead of the whole replica. An item-set consists of an explicitly represented list of unique item identifiers. In a partial replica, this item-set may be the items known to a replica for which a filter is applied limiting the items known to some subset of the overall items in the collection.
Embodiments of the present system relate to a hierarchical approach to perform synchronization that provides guaranteed paths of information to all replicas in a collection. Each partial replica is assigned a “parent” replica to act as a proxy on its behalf, and with which the partial replica regularly synchronizes as both source and target. The parent is either a reference replica or a partial replica with an interest set greater than or equal to that of the child. Each parent replica thus stores and synchronizes all items that are of interest to its one or more children replicas. Of course, if a parent replica is itself a partial replica, it in turn has its own parent replica. Following such a chain of parent replicas eventually leads to a reference replica.
Reference replicas need not have parents, although they must regularly synchronize with each other in a way that forms a connected topology among all reference replicas. The parent-child relationship between replicas in the collection creates a hierarchical synchronization topology rooted at one or more reference replicas.
The hierarchical topology augments, but does not supplant ad hoc synchronization. In addition to the required sync operations between parent and child replicas, replicas are still free to synchronize with arbitrary peers as they would in a general weakly-connected replication system.
The present system will now be described with reference to
Referring initially to
In the example of
Each replica 100 is shown with a corresponding filter 120 that specifies the interest set of the replica. In the example of
The replicas may communicate with each other in an ad hoc, peer-to-peer network via communication links 112 (represented by dashed lines) between the various replicas. It may be that not all replicas are linked to all other replicas. For example, laptop B is linked to desktop A, laptop C, cellular phone D, PDA E, but not digital camera F. Consequently, laptop B can sync with digital camera F only through one or more intermediate sync steps with replicas A and C through E. The illustrated communication links can be wired and/or wireless links, and may or may not include the Internet, a LAN, a WLAN or any of a variety of other networks.
Referring now to
While the figures and following description indicate a particular order of execution, the operations and/or their order may vary in alternative embodiments. For example, a pair of replicas could sync one-way, exchange roles, and sync the other way, thus performing a two-way synchronization. Furthermore, in some implementations, some or all of the steps may be combined or executed contemporaneously. In the example of
In accordance with the present system, the concept of item-set knowledge, as explained below, may be used to sync partial replicas with low synchronization overhead. Partial replicas are those in which a filter may be specified or provided during a synchronization request. A filter is any construct that serves to identify a particular set of items of local interest to a replica and which get stored in a replica's data store. A filter may select items from the data collection based on their contents or metadata. A filter may be a SQL query over tabular data or an XPath expression over XML representations of items or any other type of content-based predicate.
An item may fall within a filter at one time, but due to a subsequent change in the item, may fall outside the filter at another time. An example would be as follows. Consider partial replica B in
In some embodiments, the filter itself may be transmitted as part of the sync request. In other embodiments, the filter may be stored elsewhere and only some means of identifying the filter may be transmitted as part of the sync request. In yet other embodiments, certain types of sync requests may automatically result in the use of certain filters, in which case the filter itself may not be transmitted with the sync request. For example, a sync request transmitted over a low bandwidth connection might automatically result in the use of a filter that in some way reduces the number or nature of the items or changes returned.
Item-set knowledge associates knowledge vectors with item-sets, instead of with the whole replica. Each replica stores one or more knowledge fragments consisting of an explicitly represented list of items and an associated knowledge vector as well as version numbers for each item similar to the knowledge vector scheme. Item-set knowledge represents an intermediate position between the two extreme cases of per-item version vectors and knowledge vectors in terms of space and bandwidth consumption. In the best case, the item-set knowledge may just require one fragment to cover the knowledge of all the items in the replica, while in the worst case, it may require a separate fragment for each item in the replica.
Each replica's knowledge is a set of knowledge fragments. Each knowledge fragment consists of two parts: an explicit set of items (indicated by their GUIDs) and an associated set of versions represented by a knowledge vector. In addition, the latest version number for each item needs to be maintained separately by the replica. This is similar to the case of knowledge vectors. The semantics are that, for any item in the item-set, the replica is aware of any versions included in the associated knowledge vector. Knowledge fragments are additive, i.e. a replica knows about a specific version of a specific item if any of its knowledge fragments includes the item in the item-set and the version in the associated knowledge vector. A knowledge vector may include versions for items that are not in the associated item-set, in which case nothing can be concluded about these versions.
As a special case, a knowledge fragment may refer to the universal set of all items without needing to list all possible GUIDs. Such a knowledge fragment is called “star knowledge”. Having star knowledge means that the replica is aware of all updates performed by each listed replica up to the corresponding version number in the knowledge vector.
A replica holds knowledge about items that it currently stores. This first type of knowledge is called “class I knowledge”. In addition, a partial replica may be aware of items that it does not store because the current version of the item is outside its scope of interest. This second type of knowledge is called “class II knowledge”. Further details relating to class II knowledge are set forth hereinafter. As an alternative embodiment, a partial replica may store a “place holder” to represent an item that is outside its scope of interest. In this alternative embodiment, knowledge of place holders corresponds to class II knowledge.
A replica initiating synchronization sends all of its knowledge fragments (both class I and class II) to the source replica, which returns, in addition to updated items, one or more knowledge fragments as learned knowledge.
When an item is created with a new version generated by the creating replica, this version is added to the replica's class I knowledge. If the replica has a single class I knowledge fragment, the process is straightforward. The new item's ID is added to the knowledge fragment's item-set and the new version is added to the fragment's knowledge vector. If the replica has multiple class I knowledge fragments, then several options are possible. One option is to create a new knowledge fragment for the new item. This may result in many small knowledge fragments. An alternative is to add the new item and version to all of the knowledge fragments. A still further alternative is to choose one knowledge fragment to which the new item is added. The fragment that is selected may be the one that has the largest item-set or the fragment with the maximal knowledge.
When an item is updated locally, the new version number is simply added to the knowledge vector of the knowledge fragment that includes the item in its item-set. Optionally, it could be added to all knowledge fragments. A partial replica can choose to discard any item that it stores. For example, a partial replica will generally discard items that are updated and no longer match its filter. In such a case, the ID of the discarded item could be simply removed from the item-set of the class I knowledge fragment(s) that contain this item. If the item-set is empty, i.e. it only contained this single item, then the whole knowledge fragment may be discarded. If the version of the removed item does not match the partial replica's filter, it may be retained as class II knowledge.
Replicas may change their filters. If a partial replica modifies its filter, i.e. changes the predicate that selects items of local interest, then in the general case it must discard all of its class II knowledge, because it has no way of knowing whether those items match its new filter or not. However, if the new filter is more restrictive than the old filter, meaning that all items excluded by the old filter are also excluded by the new filter, then the class II knowledge is still valid and need not be discarded.
At the end of a synchronization session, the sending replica transmits as learned knowledge all of its knowledge fragments. However, items that may match the filter predicate provided by the receiving replica but are not stored by the sending replica are removed from the item-sets of the learned knowledge fragments. In practice, this means that class II knowledge will not be returned as learned knowledge unless the sending replica is a full replica or is a partial replica whose filter matches anything that would be selected by the receiving replica's filter. Learned knowledge fragments that are received at the completion of a synchronization session are simply added to the receiving replica's knowledge. Redundant fragments can be discarded as discussed below.
Thus, referring now to
KA={*}: <A5 B3 C7>
representing knowledge about items i, j, l and m having various associated ratings 2 through 5. Furthermore, since this is star knowledge, replica A knows that no other items were created or updated by any of the replicas A, B, and C up to the corresponding version numbers 5, 3, and 7.
In the example of
KB={l,m}: <A2 B5 C8>
representing knowledge about items l and m which have ratings >3.
Upon requesting the sync, replica A sends its knowledge, KA and its filter, FA. Replica B learns that replica A is unaware of version B5 and determines that the item with this version matches replica A's filter. Therefore, replica B returns version B5 and associated data to replica A. As shown in
Lastly, replica B returns the learned knowledge KB. That is, as shown in
KA={*}: <A5 B3 C7>+{l,m}: <A2 B5 C8>.
This process may be repeated for each synchronization between replicas within the collection. In this example, replica B returned its complete knowledge as learned knowledge. However, in general, a replica should only return learned knowledge for items it stores that match the requesting replica's filter or for versions of items that it knows do not match the filter.
Synchronization between replicas may cause a replica's knowledge to partition into multiple knowledge fragments for subsets of items in the original item-set. For example, as seen in
Similarly, synchronization may cause multiple knowledge fragments to be discarded and/or merged into a single fragment with an item-set covering all the items in the original item-sets. For example, if replica B in the previous example synchronizes with replica A and replica A has a knowledge fragment that includes all of replica B's items with superior knowledge, then replica B could just replace its knowledge with the single fragment received from replica A. Table 2 below specifies how a replica may merge or reduce the size of two knowledge fragments, one knowledge fragment with item-set S1 and knowledge vector K1 and a second knowledge fragment with item-set S2 and knowledge vector K2.
Operations on S1 and S2 represent standard set operations and operations on K1 and K2 represent standard knowledge vector operations, except that ≠ is used to mean “incomparable”, that is, neither includes the other. Where K2 properly includes K1 (K2 “dominates” K1), and S2 includes S1, the S1:K1 knowledge fragment may be discarded and the result is S2:K2 (first row, first and second columns of table 2). Vice-versa where K1 dominates K2 and S1 includes S2 (third row, second and third columns). Where K1 equals K2 and S2 dominates S1, the resulting knowledge fragment is S2:K2 (second row, first column). Where K1 equals K2 and S1 includes S2, the resulting knowledge fragment is S1:K1 (second row, second and third columns). The remaining possible additive combinations result in some union or subtraction of either the items-sets or knowledge vectors, except for the case where K1 and K2 are incomparable and S1 and S2 are incomparable. In this case (fourth row, fourth column), there is no discard or merge and the resulting knowledge fragment is S1:K1+S2:K2. A union on two knowledge vectors (such as for example in the fourth row, first column) results in a new knowledge vector with the highest numbered version in the two vectors for each replica. Examples of synchronization and subsequent defragmentation of knowledge fragments is set forth in U.S. patent application Ser. No. 11/751,478, previously incorporated by reference.
As indicated above, replicas with knowledge of all items are said to have “star knowledge.” Conceptually, star knowledge is just an item set knowledge fragment U:KU that covers the universal set U of items in the system; the set of items is implicit in the definition and need not be stored or communicated explicitly. Full replicas may represent their knowledge as a single star knowledge fragment, which avoids the need to explicitly list all of the items in the replicated data collection. Partial replicas can also use star knowledge in some cases. Star knowledge enables replicas to defragment item sets and ensure that the space and bandwidth consumed by item-sets remains low. Star knowledge may include the versions of items a partial replica is interested in keeping in its data store as well as versions of items the replica does not store and knows for sure fall outside its scope of interest. Note that replicas may have star knowledge in addition to other item-set knowledge fragments.
In embodiments, defragmentation involving a replica having star knowledge may take place according to table 3. This table shows how using star knowledge U:KU leads to smaller or fewer item sets by illustrating a merge between item set S:K and U:KU.
The item sets in table 3 merge only when star knowledge is higher than the knowledge fragments in the item sets. Thus in order to continuously defragment split item sets, replicas need to accumulate recent star knowledge.
A method for accumulating star knowledge in a replication system is as follows: each replica speaks for itself in terms of star knowledge, that is, the latest version number issued by a replica represents the star knowledge component for that replica. A replica can accumulate star knowledge components for other replicas by individually synchronizing with every other replica and learning their most recent version numbers. For the above mechanism to work, replicas do not discard items it created or changed. Replicas also need to retain knowledge of discarded items, and not the items themselves, by keeping a place holder for discarded items or by keeping separate item sets to represent learned-knowledge for discarded items (called class-II knowledge). A replica may expunge a place holder or class-II knowledge of a discarded item only after ensuring that every other replica's star knowledge subsumes the current replica's version number in the discarded item's knowledge. More structured forms of synchronization are contemplated in alternative embodiments.
As indicated above, a concern in a system for synchronizing partial replicas is the so-called move-out scenario, where an item moves out of the scope of interest of a replica due to a change in the item performed elsewhere. Such a scenario is illustrated in
According to embodiments of the present system, as shown in
Replica B receives move-out notification and, in embodiments, removes item m from its data as shown in
In general, a source replica will send a move-out notification to a target replica upon a change in an item if: 1) the source replica stores the item, 2) the source replica's version is later than the target replica's version, and 3) the changed item's contents are outside of the interest set defined by the target replica's filter. Alternatively, a source replica can inform a target replica of a move-out if: 1) the source replica does not store the item, but 2) the source replica's filter is less restrictive than the target replica's filter and 3) the source replica's knowledge is greater than the target replica's knowledge.
The above-described system operates effectively to provide move-out notification to all replicas in a weakly consistent distributed collection. However, the above-described methodology does not, by itself, address the issue of item reincarnation, in a partially replicated weakly consistent distributed collection. The problem of item reincarnation is illustrated in
Accordingly, referring now to
Storing class II knowledge prevents the reincarnation scenario shown in
Since no item is outside the scope of interest of a full replica, a full replica has no need for class II knowledge.
According to embodiments of the present system, as shown in
KB={l}: <A2 B5 C4> (class I knowledge)+{m}: <A3 B5 C11> (class II knowledge.
After the sync operation shown in
Source replica A then returns any versions of which it is aware and which replica B is not (there are no such versions in the example of
As shown in
The ad hoc network shown in
Parent replicas have filters which are either the same as, or more inclusive than, their children, and parent replicas thus store and synchronize all objects that are of interest to their one or more children replicas. Replicas avoid cyclic parent relationships by ensuring that they are not assigned one of their descendents as a parent. Reference replicas, however, are not required to pick parent replicas, although they must form a connected synchronization topology between them.
A “descendant” of a replica is any replica that is either (a) a child of that replica or (b) a descendant of a child of that replica. Likewise, an “ancestor” of a replica is a parent or an ancestor of a parent.
Referring to
Referring now to
In the embodiment shown in
In the case in which the parent replica is a reference replica, the parent does not need to bother about adding up the filters of its children. A reference replica is interested in all items, so it is guaranteed to be interested in any item that any of its children might be interested in.
For given ad hoc connections between replicas in a collection, the replicas may automatically establish themselves in embodiments in a hierarchical topology satisfying the above methodology. Alternatively, a user or administrator may designate one or more replicas as proxies and set up part or all of the hierarchical topology. The hierarchical topology augments, but does not replace, ad hoc synchronization. As shown in
At the top of the hierarchical topology are one or more reference replicas, such as replica A in
The hierarchical synchronization according to embodiments of the present system ensures two important properties of a replicated system: all replicas accumulate knowledge of items of interest and replicas are able to disseminate items to other replicas. Through a path of synchronizations between replicas organized in a hierarchy, a picture with rating >4 taken by camera F in
New and updated items are propagated (and knowledge of such items is accumulated) up the hierarchical topology to a reference replica as a result of sync operations occurring between child and parent replicas in the hierarchy. Thus, in the example of
At some later time t2, the reference replica A requests a sync with its child replica B. As a reference replica, replica A is interested in every item. During that sync operation, reference replica A learns of the new item, updates its data store with the new item and updates its knowledge to reflect this new item. The example of
By defining a hierarchical topology of one or more children synching with one or more parents as described above, it is guaranteed that all items created by any replica in the collection will eventually be assimilated into a reference replica. Once the hierarchical relationships are defined, synchronization of the parents to and from their one or more child replicas to pass new or updated items may occur by ad hoc synchronization; that is, by normal synchronization operations of proxies to their children that occur at times but according to no set schedule. However, in an alternative embodiment, synchronization of one or more children to and from their parent replica may be forced periodically according to a set schedule. The periodicity of this schedule may vary in alternative embodiments.
In the embodiment described in
In the simple embodiment of
In order to account for high loads, an embodiment of the present system may operate using split filters in one or more of the replicas. For example, in the embodiment shown in
In embodiments of the hierarchical sync system, it may happen that an item is created by or modified within a partial replica so that it is outside of the interest set of the replica. This is called a “push-out.” Even though the partial replica is not interested in the new or updated item, since it holds the only copy it must keep it until it can guarantee that the item will eventually reach a reference replica. The replica can address this problem by maintaining that item in a “push-out” store that is managed independently from its data store.
The description hereinafter uses the following definitions. A replica “stores” a version of an item if either (a) the version is in the replica's data store, (b) the version is in the replica's push-out store, or (c) the version is superseded by a version that the replica “stores”. A replica “knows” a version of an item if either (a) the replica “stores” the version, (b) the replica is assured that the version falls outside the replica's filter, or (c) the version is superseded by a version that the replica “knows”. Roughly, versions that a replica “stores” are included in its class I knowledge and versions that a replica “knows” are included in its class I or class II knowledge.
In embodiments, a replica may include items in its push-out store in its class I knowledge, since they are items that it “stores”. Alternatively, a replica may include items in its push-out store in its class II knowledge, as items that it knows about but that fall outside its interest set.
Preferably, when a parent requests a sync with its child, the child replica sends push-out notifications of items to the parent replica and in this manner transfers responsibility for the items to its parent. Preferably, push-out notifications are sent for items in the child's push-out store. Alternatively, push-out notifications may also be sent for items in the child's data store. The push-out notification transfers the item regardless of whether or not it falls within the parent's interest set. If such a transferred item is outside of the parent's interest set, the parent likewise maintains the item in its own push-out store until it is able to transfer the item to its parent. Alternatively, push-out notifications could be sent from child to parent when the child initiates a sync or even independently of the ordinary sync protocol. The process of maintaining an item even if outside of the replica's filter continues until the item reaches a reference replica. In this way, updates made that are outside the filter(s) in a hierarchical chain will still reach a reference replica (which has no items outside its filter).
In alternative embodiments, a replica may send push-out notifications to any of the replica's ancestors. Alternatively, a replica may send push-out notifications to any replica that is closer to a reference replica, as indicated by comparing each replica's length of chain of parents, as maintained in a replica's hierarchical information. Note that when a replica receives a push-out notification, that replica receives responsibility for the item and must guarantee that the item eventually reaches a reference replica.
A push-out notification may include the item being transferred. In some embodiments, this is always the case. In alternative embodiments, if the sending replica determines that the receiving replica already “stores” the item, the push-out notification may be abbreviated to include the identifier and version of the item being transferred and need not include the item itself. Preferably, the sending replica determines this by inspecting class I knowledge sent by the receiving replica during the normal synchronization protocol. Alternatively, the information could be sent in a separate protocol.
In further embodiments, a replica R can manage push-outs more efficiently by associating an explicit “responsibility bit” with each item. The responsibility bit of an item is set at replica R when the item is generated by R either for the first time or through an update of an older version of that item. When the item falls outside R's interest set, either as a result of the previous update or as a result of a change in R's filter, the item is transferred to the push out store of R if its responsibility bit is set and simply discarded otherwise. R may send a push-out notification for item i only when the responsibility bit for item is set. When this happens, R clears the responsibility bit and the receiving replica sets it. In addition, R clears the responsibility bit for item i whenever it receives an updated version of item i or a move-out notification for item i. If item i is in a push-out store when its responsibility bit is cleared, item i may be discarded.
New and updated items are disseminated from a reference replica down the hierarchical topology as a result of sync operations occurring between parent replicas and their children in the hierarchy. For example, referring again to
When syncing from reference replica A, replica A may send both class I and class II knowledge as learned knowledge. However, as shown and described above with respect to
Subsequently, replica F may sync from replica C, in a manner described above, and thus, all knowledge that was known to the reference replica and fitting replica F's interest set may be learned by replica F. In general, the children replicas may sync from their parent replicas as described above until all knowledge from the reference replica is received in the bottom tier replicas. In this way, all updates made by any replicas are received in all other replicas having interest in those updates.
By defining a hierarchical topology of one or more children synching with one or more parents as described above, it is guaranteed that all items created by any replica in the collection will eventually be disseminated. Once the hierarchical relationships are defined, synchronization of the children to and from their parents may occur by ad hoc synchronization. However, in an alternative embodiment, synchronization of one or more children to and from their parent replica may be forced periodically according to a set schedule. The periodicity of this schedule may vary in alternative embodiments.
The process of assimilation and dissemination of knowledge may occur during a single synchronization operation between a child and its parent. That is, the synchronization operation may be a two-way sync operation where the parent assimilates the knowledge of the child replica, and the parent disseminates its knowledge to the child replica. However, it is understood that the assimilation and dissemination of knowledge between children replicas and parent replicas may occur in separate, one-way sync operations.
At times, it may be desired to change the parent of a partial replica. In the hierarchical topology of the present system, the filter of the new parent may have to be enlarged in order to guarantee that the new parent has an interest set that contains the interest set of the partial replica. After the old parent has lost the child, it may be possible to reduce the filter of the old parent.
At times, it may be desired to change the filter of a partial replica. If the change is a reduction, then no change to the filter of its parent is required, although subsequently the parent could in turn be permitted to reduce its filter accordingly. Alternatively, the parent replica must change its filter in order to accommodate the new filter of its child. This may require the parent to have its parent change its filter, and so on. Alternatively, the partial replica can be assigned to a new parent.
At times, it may be desirable to add replicas to a collection. In the hierarchical topology of the present system, given a proposed new replica having some proposed filter, the situation is analogous to changing the parent of an existing replica, except in this case there is no old parent. A partial replica could always be assigned a reference replica as a parent.
Hierarchical synchronization enables the present replication system to defragment a replica's knowledge and represent it using fewer knowledge fragments.
Star knowledge reduces the number of item set knowledge fragments maintained by a replica because it subsumes other knowledge fragments. Any item set knowledge fragment in the replica's knowledge that is dominated by star knowledge is subsumed by the star knowledge and need not be maintained. Alternatively, all item set knowledge fragments in the replica's class I knowledge that are dominated by star knowledge can be combined into a single class I item-set knowledge fragment equal to the star knowledge. Thus, a replica can reduce the number of knowledge fragments as its star knowledge increases. In the long run, a replica might even be able to reduce its entire knowledge to a single star knowledge fragment.
An embodiment of the present replication system that uses a non-split hierarchical synchronization topology can increase the star knowledge of a replica through synchronization from its descendant and ancestor replicas according to the method described below. This method works for embodiments in which no replica ever changes its filter so as to become more inclusive.
Referring to
The above method for increasing star knowledge only applies to embodiments of the present system in which no replica ever changes its filter so as to become more inclusive. Alternative embodiments of the present system in which a replica might change its filter, without concern for whether the change increases or decreases the filter, can use the method described below.
An alternative embodiment of the present replication system that uses a non-split hierarchical synchronization topology can increase the star knowledge of a replica through synchronization from its descendant and ancestor replicas according to the method described below. This method works for embodiments in which replicas change their filters without concern for whether the change increases or decreases the filter. However, this method requires that each replica always “stores” every version generated by it or any of its descendants. In an embodiment of the system employing this method a replica can fulfill this requirement by simply adding to its filter all items created or updated by itself. In such an embodiment, note that the hierarchical relationship of replicas ensures that a replica implicitly also “stores” every version generated by any of its descendants.
Recall that when a replica changes its filter and the new filter might be more inclusive than the old filter, the replica needs to discard all its class II knowledge fragments. In addition, when using star knowledge, the replica also needs to change its star knowledge since the star knowledge also implicitly represents class II knowledge. The required change is as follows. The replica must set to zero those components of its star knowledge corresponding to all replicas other than itself and its descendants. Further updates to star knowledge as a result of synchronization happen in the same way as described in the previous method and illustrated in
The inventive system is operational with numerous other general purpose or special purpose computing systems, environments or configurations. Examples of well known computing systems, environments and/or configurations that may be suitable for use with the inventive system include, but are not limited to, personal computers, server computers, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, laptop and palm computers, hand held devices, distributed computing environments that include any of the above systems or devices, and the like.
With reference to
Computer 410 may include a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 410 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, as well as removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), EEPROM, flash memory or other memory technology, CD-ROMs, digital versatile discs (DVDs) or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disc storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 410. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer readable media.
The system memory 430 includes computer storage media in the form of volatile and/or nonvolatile memory such as ROM 431 and RAM 432. A basic input/output system (BIOS) 433, containing the basic routines that help to transfer information between elements within computer 410, such as during start-up, is typically stored in ROM 431. RAM 432 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 420. By way of example, and not limitation,
The computer 410 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, DVDs, digital video tapes, solid state RAM, solid state ROM, and the like. The hard disc drive 441 is typically connected to the system bus 421 through a non-removable memory interface such as interface 440. Magnetic disc drive 451 and optical media reading device 455 are typically connected to the system bus 421 by a removable memory interface, such as interface 450.
The drives and their associated computer storage media discussed above and illustrated in
The computer 410 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 480. The remote computer 480 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 410, although only a memory storage device 481 has been illustrated in
When used in a LAN networking environment, the computer 410 is connected to the LAN 471 through a network interface or adapter 470. When used in a WAN networking environment, the computer 410 typically includes a modem 472 or other means for establishing communication over the WAN 473, such as the Internet. The modem 472, which may be internal or external, may be connected to the system bus 421 via the user input interface 460, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 410, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
The foregoing detailed description of the inventive system has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the inventive system to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the inventive system and its practical application to thereby enable others skilled in the art to best utilize the inventive system in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the inventive system be defined by the claims appended hereto.
The following applications are cross-referenced and incorporated by reference herein in their entirety: U.S. patent application Ser. No. 11/751,478 [MS# 318994.01], entitled “Item-Set Knowledge for Partial Replica Synchronization,” by Ramasubramanian, et al., filed on May 21, 2007. U.S. patent application Ser. No. ______ [MS# 319749.01], entitled “Move-In/Move-Out Notification for Partial Replica Synchronization,” by Ramasubramanian, et al., filed on the same day as the current application.
