Patent Application
20120185444
Many regulatory authorities and enterprise internal policies require the retention of certain data for a specified period (the “retention period”). As the data required to be retained in this manner is generally intended to provide a reliable record of contemporaneous events (such as stock exchange transactions), the data held in retention is required to be protected against change, at least to some degree.
Much of the data subject to a data retention regime will be in electronic form. Write-once-read-many, WORM, storage systems are well suited for retaining electronic data in immutable form. In a WORM storage system, data to be retained is stored in WORM files and the system provides protection mechanisms preventing changes to the file and at least some of its metadata. Generally, a WORM storage system is not limited to the storage of WORM files and may store non-WORM files as well; as a consequence, the protection provided to WORM files includes protection of the designation of a file as a WORM file, whatever form this designation may take.
In the context of data retention, the “write once” in relation to a WORM file refers to the form of the file data at the point the file is designated a WORM file (it being understood that the file may have undergone many re-writes before this point). From the point of view of resource efficiency, a WORM file created to comply with a particular data retention regime should only be maintained as such for as long as needed to comply with the retention period specified. Therefore, a retention end date (herein “retention date”) is generally stored as metadata along with the WORM file, the retention date having been determined at the time the WORM file is created on the basis of the retention period (or the longest such period) applicable to data in the file.
Upon expiry of the retention period associated with a WORM file (as judged by comparing the retention date held in the file's metadata with the current time, inclusive of date, provided by a reference time source), the WORM storage system is generally arranged to permit the file's WORM designation to be rescinded. Changes can thereafter be made to the file, subject to normal access permissions. This gives rise to a potential way of illicitly changing file data during its retention period; more particularly, if either the stored retention date can be rolled back to the present or the reference time source rolled forward to the stored retention date, the WORM storage system can be tricked into believing that the retention period for a particular WORM file has expired, and allow the WORM designation of the file to be rescinded and data in the file changed. By restoring WORM designation to the changed file and resetting the stored retention date or reference time source (whichever was changed), the fact that the file data has been altered can be hidden. For this reason, the protection of the nietadata storing the retention date, and the trustworthiness of the current time source, are pertinent considerations in any WORM storage system used for implementing a data retention regime.
Embodiments of the invention will now be described, by way of non-limiting example, with reference to the accompanying diagrammatic drawings, in which:
The server 2 shown in
The operating system 14 includes filesystem functionality 13 that implements a filesystem 10, that is, an organization of data, held in the storage subsystem 3, and comprising one or more files 11 and associated metadata 12 (represented in
Application 17 is a server program for handling archival requests from clients over the data access network 28 by making appropriate accesses to the storage subsystem 3 to satisfy those requests (subject to certain protection features described below). Application 18 is an administrator interface program.
In the context of a WORM storage system such as the system 1, the filesystem functionality 13 is arranged to provide certain WORM protection features 15 in respect of files that are designated as WORM files, that is, files that are not to be changed but only read. The ‘WORM status’ of a file, that is, whether or not a file is a WORM file, (and therefore whether or not it is subject to protection by the WORM protection features 15) is designated in the file's metadata (attribute 12S in
The WORM protection features 15 provided by the filesystem functionality 13 will, in fact, mostly already be present in connection with enforcing file access permissions (that is, read, write and execute permissions for various types of user). The WORM protection features 15 include the prevention of deletion of, or any change to, a WORM file and at least certain of its attributes, the prevention of deletion of, or any change to, the path or directory structure for locating the file, and the prevention of any recovery, rollback or restoration function that could affect the file or its metadata, path or directory structure. As will be briefly described below, certain limited exceptions to the application of the WORM protection features will generally be appropriate.
The server 2 of the
In the present example storage system 5 of
As already noted, in the
As well as initially setting the retention date, the retention manager 16 is also responsible for managing its subsequent extension, for periodically checking for expiry of the retention period set for a file, and for having an administrator, or other party, review a file that has exited its retention period.
In
Checking for expiry of a retention period is done by the retention manager 16 using a clock of the server 2 (in the present example, the system clock 19 provided by the operating system 14). As discussed in the introduction, one way in which a WORM file can be illicitly changed is by rolling forward the clock used for retention-period expiry checking (system clock 19 in the case). In order to detect any interference with the system clock 19, the server 2 is provided with a clock monitoring program (clock monitor 20 of
Before describing the clock monitor 20 in detail, a description will first be given of the processes carried out by the retention manager 16 to set and extend a retention date, and to check for retention date expiry.
Considering first the process of setting a retention date for a file, a flow chart of this setting process is depicted in
In a variant of the above-described retention-date setting process, rather than the retention date being calculated as a delta from a base time, a specific retention date may be received as input and used directly (after appropriate checks relative to the retention policy being implemented).
Thus, on completion of the setting process, a retention date has been stored to a file attribute and the WORM attribute set to indicate that the associated file is a WORM file. The WORM protection features 15 subsequently operate to prevent deletion or change of the file, the retention-date attribute, the WORM attribute and, indeed, most of the other attributes of the file concerned.
Handling retention period extension requests is effected by a retention-period extension process (not illustrated) of the retention manager 16, this process operating to validate any extension requests by checking that the new retention date is indeed in advance of that currently stored and, if required by the retention policy, checking that the extension request comes from an appropriately authorised user. Only if these checks are passed is the retention period extended by setting a new retention date in the attribute concerned.
In order to recognize when a WORM file has exited its retention period, the retention manager 16 is arranged to periodically run a retention-period expiry checking process in which it checks for the expiration of the retention period of each file in a predetermined group of files; this group may comprise all WORM files in the filesystem or a subset that, for example, changes at each running of the expiry-checking process such that over a suitably short period of time all the WORM files in the system are checked. A flow chart of the expiry-checking process is depicted in
On completion of the retention-date expiry checking process, the retention manager 16 is arranged to initiate a review of the files in the ‘expired’ list by an administrator or other designated party in order to determine the fate of these files.
Rather than carrying out the active retention-date checking process described above, an alternative approach is use lazy discovery of files that have passed their retention dates. With lazy discovery, the retention manager 16 would only check the retention date of a file when that file is touched for some other reason (file read, a delete attempt, filename rename attempt, etc.). A file that has passed its retention date can either be flagged for immediate review or placed in an expired list for review at a later date.
Turning now to a consideration of the clock monitor 20 monitoring for interference with the system clock 19, it is first noted that this clock provides a measure of time elapsed since some standard reference point in time (epoch') typically many years in the past; the time measured by the clock is thus a measure not only of time of day, but also of the passing of calendar days, months and years. The system clock is only operative when the server 2 is running. However, the always-energised hardware real time clock 9 keeps a track of time even when the server 2 is not running. When the server is booted, the system clock 19 is aligned with the current time indicated by the real time clock 9. Both the system clock time and the real clock time are capable of adjustment through software commands (in the case of the real time clock this involves BIOS routines). The present embodiment of the clock monitor 20 is operative to detect any significant adjustments of the system clock 19 (and, indirectly, any significant adjustments the real time clock 9 that are imported into the system clock 19 when the server 2 is booted).
A flow chart of the monitoring process 40 operated by the clock monitor 20 of server 2 is depicted in
Rather than the current local clock time being obtained as the first step of the monitoring process 40, it can be obtained after receipt back of the non-local current clock times in step 42. Furthermore, in step 45 when an alert is generated, an additional action that can usefully be taken is to prevent files from exiting their retention periods until the clock inconsistencies are resolved. This protects the files during the period of uncertainty.
Running the clock monitoring process 40 on the same machine as the retention manager 16, means that the process 40 is focused on monitoring for interference with the clock used by the retention manager 16 for retention-date expiry checking. The log information provided by the process 40 helps to identify if and when the clock used for expiry checking has been subject to interference. It is, however, possible to obtain a more complete picture by running the same process 40 on the other machines of the storage system 1 from which the clock monitor 20 running on the machine hosting the retention manager 16, has obtained non-local current clock times in step 42. More particularly, to implement this, each of these other machines is arranged to host its own clock monitor 20 to carry out process 40 in respect of its own clock thereby not only to monitor for interference with this clock but also to provide log information useful in understanding interference with clocks in other machines. It will be appreciated that in relation to the execution of the monitoring process 40 on one of these other machines, the terms ‘local’ and ‘non-local’ used above in describing the process 40, are relative to the machine running the process.
It may be noted that although only one server 2 of the clustered storage system 1 is depicted in
Examination of the logs produced by all of the clock monitors 20 each running on a different machine of the storage system allows an administrator of the storage system to identify which system clocks have been interfered with and this, in turn, allows the files that were modified (or at risk of being modified) to be identified. To defeat clock monitoring effected in this way would require the near simultaneous changing of the system clocks of all the machines that run the clock monitoring process 40.
Added security can be provided by spreading the computing machines consulted in step 42 across multiple security domains amongst (i.e. giving them different root passwords), the operational environment being set up such that no single person has sufficient authority to enable them to tamper with time on all the machines consulted.
The above described method and apparatus for monitoring for interference with a clock used in checking for expiry of a file's preset retention date can be applied to any form of clustered storage system. Various forms of clustered storage systems are known and generally differ from one another by the way their servers inter-relate with each other in managing access to the storage subsystem, and whether or not a unified filesystem (single namespace) is implemented. By way of illustration of the applicability of the described clock monitoring method and apparatus to clustered storage systems in general, two examples are given below of how the main elements of the
Each of the servers 51 includes a retention manager 16 for effecting retention management in respect of the filesystem(s) for which it is responsible. In carrying out its retention-date expiry checking function, each retention manager 16 is arranged to use the system clock of the server 51 on which it is hosted. Each server 51 also includes a clock monitor 20 arranged to run the clock monitoring process 40 to monitor the server's clock. Each clock monitor 20, in running the process 40, is arranged to obtain clock times in step 42 from all the servers 51 (other than the one on which it is hosted) over network 52.
In the
Clock monitors 20 are provided in each of the segment servers 61 and in the duster manager 63. Each of these clock monitors 20, in running the clock monitoring process 40 to monitor the clock of the machine on which it is hosted, is arranged to obtain non-local clock times (step 42) from each of the other machines 61/63 provided with a clock monitor.
It will be appreciated that many variations are possible to the above described retention-date expiry reference clock monitoring method and apparatus and the clustered storage systems to which they are applied. For example, at least some of the computing machines of the clustered storage systems can be implemented as virtual machines with multiple such machines being hosted on the same underlying computing platforms (this is particularly suitable for the servers 51 of the
In the foregoing, the clock used by the retention manager 16 to check for retention date expiry was the system clock of the machine hosting the retention manager; accordingly this was the clock which the clock monitor 20 was arranged to monitor for interference. However, the retention manager can be arranged to use a different clock of its hosting machine as the expiry reference clock (for example, the real time clock 9). Generally, whatever clock of the hosting machine is used by the retention manager as the expiry reference clock, the clock monitor is arranged to monitor the same clock. With regards to the non-local clocks from which the clock monitor 20 obtains non-local current times in step 42, these do not necessarily need to be the corresponding clocks of the machines concerned but where these other machines also host retention managers and therefore clock monitors of their own, then the clocks used should be the ones serving as expiry reference clocks on the machines concerned.
In the foregoing, the example retention-date expiry reference clock monitoring method and apparatus have been described in relation to clustered storage systems set up for the archival retention of files and their metadata; it is, however, to be understood that the retention-date expiry reference clock monitoring method and apparatus can be used in relation to clustered storage systems set up for archival retention of any structuring of data (herein a ‘dataset’) capable of being handled as a single logical entity and having associated metadata. Furthermore, the purpose for which datasets are stored for retention (for example, archival and/or compliance purposes) is not relevant to the retention-date expiry reference clock monitoring method and apparatus of the present invention which are applicable independently of such purpose. Similarly, although data retention is almost always done under WORM conditions in order to protect the data during its retention period, it will be apparent that whether or not WORM conditions are applied during the retention period does not affect the operation of the retention-date expiry reference clock monitoring method and apparatus of the present invention which are equally applicable whether or not WORM conditions exist during the retention period of particular data.
