This application claims priority to the European application No. 04017035.9, filed Jul. 19, 2004 and which is incorporated by reference herein in its entirety.
The present invention relates to the synchronization of a distributed system and especially to the distribution of data as well as to the access to resources in distributed systems.
In order to handle complex computing tasks and/or to create security in data processing systems through redundancy, systems are frequently used which are a combination of a plurality of individual automation units of computing units or computer systems which however present themselves to the user of the system transparently as a single system. Such systems are referred to as distributed systems in which for example procedures or measures such as memory redundancy or load balancing are preferably designed so as to be transparent, i.e. imperceptible to the user. Distributed systems are distinguished from a networks for example by the fact that in networks the individual computers of the network are presented to the user as separate entities, with memory redundancy or load balancing frequently not being arranged transparently, that is perceptible to the user, and frequently even requiring user interaction.
In distributed systems data must be distributed between the individual computing machines or systems. There are a number of known options for this data distribution. Generally a distinction is to be made here between the relatively slow transaction-based data transmission on the one hand and the faster, relatively more insecure distribution with less effort on the other hand.
When, in the case of transaction-secured, data transmission the transaction security extends to the user interface, a complete data consistency at all times can be ensured in that for example the use only receives an acknowledgment if an entry has been distributed secured in the system. This is however a relatively tedious process with a high communication overhead, especially in cases in which the data has to be distributed over many machines. While long reaction times are generally undesirable there are any number of applications in which a long reaction times must be avoided, for example security-critical applications Furthermore transaction-based systems are complex to implement and expensive.
If on the other hand there is a departure from the principle of full transaction security, for example by an acknowledgment message being sent even if the input can only be forwarded to one other machine or another system, there is the danger of inconsistent states in the system These inconsistent states can lead to a loss of data, in cases, such as where system separations occur as a result of connection problems or the very machines or subsystems fail which have received the input data. As a result a signal could be sent to a user that his input had been processed whereas this input has been lost in the system, something which must specifically not occur in security-critical environments.
A typical example is an emergency call which is then confirmed to the user making the call as a successfully issued emergency call by a display but which in the final analysis is not further processed.
The object of the present invention is thus to specify an alternative transmission method for distributed systems as well as an alternative distributed system where security of a complete transaction security is achieved with low outlay.
This object is achieved by a method for synchronizing components of a distributed system in accordance with which the system status is represented by at least one object that is provided in all components, with a change of status of the object in one of the components being signalled by a status change message to all other components, whereon the local validity of the signalled status change is checked by each of the other components, with, for a locally valid status change, the status of the objects in these components being updated and with, for a locally invalid status change, a component with a valid status of the object being determined which is at least sent to the components with the invalid status, whereon the status of the objects is updated in these components.
The invention further relates to a distributed system of which the components are designed for executing the inventive method.
By contrast with known distributed systems the present invention offers the advantage that for transmission of the status change messages which are needed to maintain the system synchronicity or to restore it, unsecured and thereby faster transmission methods can be used, for example UDP/IP (User Datagram Protocol/Internet Protocol) multicast messages. The requirement here is that these messages also reach their destination in the normal case and that the system remains synchronous through these messages alone. If however faults arise, each component is in a position at the latest by the next status change message received or with longer pauses in the message transmission by the monitoring mechanism used, to establish the where necessary local interruption of the synchronicity and to request this from a component with the correct status. The system status is in this case mapped completely by one or more objects. A number of suitably delimited objects offer the advantage here that the volume of data occurring for a change of status of one of the objects is smaller.
In other words the present invention achieves a loosely-coupled distributed system in which the normal operation without faults runs more quickly than in completely transaction-secured systems, whereas at the same time that the security of a completely transaction-secured system is guaranteed.
In this case it is not necessary in accordance with the invention to maintain information on the system status or on the status of the object in selected central components, e.g. servers or databases which would constitute what is known as the single point of failure. Instead, in the event of an error, a component is determined which has a valid status which can then be used for components with invalid object status. A “single point of failure” always dictated by central components which adversely affects the availability of the system is not required in accordance with the invention.
The present invention equally provides a mechanism for resolving competing accesses to exclusive resources. With such competing accesses it is necessary to set a consistent or synchronous systems state if two or more instances are simultaneously or contemporaneously manipulating an exclusive resource or attempting to do this. Here too an inconsistent status can be prevented or rectified by the present invention.
In many applications a distributed system in accordance with the invention can advantageously replace a separate database as well as a transport mechanism for distributed working (e.g. CORBA). Such a system also offers advantages in environments where from time to time network separations (separate subnetworks) with subsequent recombination occur. Here the system resets itself—imperceptibly for the user as a rule—back to a common status.
The invention is explained in more detail below in exemplary embodiments with reference to a drawing.
It is assumed that all three computers A, B, C have initially stored the same status for the object X, are therefore operating synchronously in relation to the object X. The status is characterized by what is known as the status owner (StOwn) and the identifier of the last status message (StChID). These two parameters are stored in all computers and are the same throughout the system for as long as the system is operating synchronously. Furthermore message identifications (MID) are stored in all other computers, in computer A therefore further message identifications of the computers B and C, in computer B further message identifications of the computers A and C and in computer C further message identifications of the computers A and B.
The computer in which the last status change occurred is designated as the status owner and which as a result of this status change has communicated with other computers by means of the status change message. In the present case: StOwn=B, i.e. Computer B is the status owner of the last status change which carries the identifier StChID=5. The last message sent out by A had the MID 15, the last message sent out by B had the MID 20 and the last message sent out by C had the MID 35, with the initial status shown in
As a result of a status change computer A then transmits to all other components, i.e. to computer B and computer C, a status change message 102, which has the identifier StChID=6 (old StChID plus 1). The status owner is now computer A since the status change came from this computer. The MID for A is also incremented and transmitted with the status change message and now amounts to 16.
The message 102 is correctly received by all other components. The identifiers StChID and MID are compared with the locally stored values and it is established that the previous local status at B and C was valid since the received values for StChID and MID correspond exactly to the local values incremented by 1. It further follows from the fact that A has used the correct StChID and MID, that the status is also valid at computer A. Computers B and C then update the status of the object X, which is then identical again for all computers, with the parameters StChID=6, StOwn=A and MID A:16, B:20, C:35.
Expressed in more general terms, the system behaves in the error-free operation described here as follows:
All objects which represent a part of the system status are basically created on each of the computers involved. This means that all objects affected by a system status are replicated globally. Each change of status is sent by multicast message and thus received by all components or by computers A, B, C. The local object parameter StChID, the parameter StOwn and the local computer or local component parameter MID are inserted into such a message.
The parameter StChID can be used to detect conflicts caused by simultaneous status change or temporary network separation. Temporary network separations can be detected on the basis of the local computer parameter MID. Simultaneously this parameter can be regularly monitored by an active ping mechanism between the computers involved, with the ping mechanism being embodied such that pings can only be exchanged if no other messages of this computer have been received. If none of the monitoring mechanisms notifies an error, this means that the propagation of the status change by the simple sending out of the (multicast) message is at an end.
Again referring to
The message 104 is received correctly by A. Computer A again performs the above-mentioned checks and finally the local status, which is then characterized by the parameters StChID=7, StOwn=B and MID A:16, B:21, C:35.
As a result of a communication fault the message 104 is not received or not correctly received by computer C. Computer C does not take any action and remains in the currently valid status, which—as above—is characterized by the parameters StChID=6, StOwn=A and MID A:16, B:20, C:35.
From this moment the distributed system is no longer synchronous, which however cannot be directly established. The error is detected in the example of
If the error detection is undertaken in the system on the basis of the next status change message, two cases can be distinguished: C transmits a status change message, and A and B detect the problem (shown in
As a result of a status change computer C transmits a status change message 106 to all other components, i.e. to computers A and B. Since the local status of the object X in the computer C does not match the current status, C uses an old status change message—from the point of view of A and B—id. StChID=7. The status of C is recognized as invalid by A and B since an StChID=8 was expected by C.
If an error is detected, a conflict resolution is executed. In this case it is established with reference to the parameters StOwn, StChID or/and specifiable priorities by which component the true status was determined. With the same rights the component can be determined on the basis of a minimum of a generally known comparable characteristic to the actual status owner (e.g. on the basis of the network address). Since all the data needed for the decision is present globally, the decision can be made without additional communication.
In the example of
After this error rectification all components A, B, C again have a uniform status with regard to the object X, characterized by the parameters StChID=7, StOwn=B and MID A:16, B:21, C:36.
After component B in step 104 has transmitted a new status transmission (with StChID=7), there is a change of status at A which is transmitted from A to the other components, i.e. to the computers B and C (step 206). This is done by using the status change message parameters StChID
=8, StOwn=A and MID=17. On arrival of the StChID=8 in the received status change message C establishes that a status change message with StChID=7 was not received by C and thus that the local status at C is invalid. C therefore requests via a broadcast (i.e. to all components including a component D which has not thus far been considered for reasons of simplicity) by means of message 210 a current image of the object and starts a monitoring timer to monitor receipt of the image from A. Beforehand however B had already signalled a further status change by means of a status change message 208 (StChID=9, StOwn=B, MID=9) to all other components.
The status change message 208 is received by C, after which C establishes that a message from B has been lost since the local MID for B amounts to 20, whereas that contained in the received message is 22. If an MID deviates from the expected value no broadcast is sent to all other components but instead a complete list of all status change message identifiers StChID is requested from the partner component for which the MID shows deviations (step 214), which is received (step 218) and on the basis of which it is determined for which objects an updated image is to be requested. For these objects the image is then requested by broadcast, step 220.
In the case of
After this error rectification all components A, B, C, D again have a uniform status as regards the object X, characterized by the parameters StChID=10, StOwn=D and MID A:17, B:22, C:35, D:567.
It should be pointed out that with requests for object images by components which have detected an invalid local status, a timer is started as already mentioned, within which the object image of the status owner of the last status change message (on the basis of which the error was detected) is expected. If this is not sent, the component with the next lowest priority takes over this task. Advantageously this does not require any additional request, if object images are sent by broadcast to all components, since this is how the component with the next lowest priority establishes the absence of the object image from the actual status owner.
Alternatively the ping mechanism can be used to detect the failure of a status owner and to send the image of the object to the component with the invalid local status even before the timer has expired.
The maximum delay, with which the loosely-coupled distributed system in accordance with the present invention executes a synchronization depends on the value of the monitoring timer. This delay only has noticeably disruptive effect however in the event of an error so that transport networks of the prior art which even with insecure transmission exhibit a very low error quota overall enable a faster message transmission by means of simple, unconfirmed multicast messages in conjunction with the invention, with the high security of a completely transaction-secured system being achieved immediately after the expiry of the configurable time of the monitoring timer.
To execute the present invention any multicast mechanisms can be used, provided all components can be reached with multicast-messages. If for example the Internet Protocol IP is used as the preferred transport protocol and the User Datagram Protocol UDP is used as the multicast protocol, it must be ensured for example that all components are addressable, i.e., if the components lie in different IP networks, routers must be used for example and correspondingly reconfigured.
To represent devices which do not of themselves support any multicast mechanisms for status indication or a control of any number of sources, a representative object can be used. This representative object basically behaves like the objects described above. For direct access to the represented device however an object on a selected computer (as representative of the device) is defined which takes over the actual communication with the device. For determining this object the mechanisms described above are used, i.e. according to the minimum of a globally known computer characteristic one of the objects recognizes itself as representative object and sets an attribute with its computer address to notify this to the other objects. Should a number of objects simultaneously declare themselves to be a representative object, this status is resolved again by conflict resolution, as stated above. In order to ensure that there is only ever one representative object for a device, the monitoring function can be expanded such that, if a computer failure is established, all objects reset the now invalid attribute of the representative computer. Thereafter another representative object again takes over responsibility in accordance with the known algorithm. If the representative object receives (status) messages from the device represented It alters its status correspondingly. This is again transmitted automatically to all computers of the network.
Naturally algorithms other than the determination of a minimum given as an example can be used to select a conflict-resolution object or to select a representative object. Thus suitable algorithms can be used to effect a load distribution for this active object and its relevant representation on the computers involved.
To implement the present invention known programming characteristics can be applied to simplify the implementation of a system that can be described in this way. These especially include the use of reflection mechanisms for simple implementation of the information distribution, conflict detection and monitoring of the conflict resolution in basic classes which largely frees the higher layers of the implementation from realizing the specified mechanisms.
The system in accordance with the present invention is especially suitable for applications in which a number of consumers (e.g. operator workstations) with information of a data producer (e.g. sensor system) must be simultaneously provided with input without imposing an unnecessary additional load on the communication system. This is especially important with large numbers of producers and/or for data producers which generate large volumes of data. An important example of this are monitoring systems with a number of operator workstations which are also spatially separated and a multiplicity of different data producers such as video sources, contact sensors, proximity sensors, moisture sensors, smoke detectors etc.
Number | Date | Country | Kind |
---|---|---|---|
04017035.9 | Jul 2004 | EP | regional |