Distributed redundant adaptive cluster

Abstract

Efficiency in the operation of a data archive and media life cycle management system is achieved by utilizing multiple system cores to avoid potential failures and appropriately distribute activities, along with multiple communication channels for efficiently moving information. The two system cores operate in a redundant system fail-over mode, thus insuring continuous operation. The two communication channels include different operating characteristics, one cable of low cost communication, while the other capable of higher cost/higher data rate communication. In the application where large amounts of data are moved between components, efficiency is achieved by having the higher cost/higher data rate network move this large amount of information, while the lower data rate/lower cost communication system allows instructions to easily be communicated, thus coordinating operations of the overall system.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

The inventive data archive and media lifecycle management system will be described below in greater detail by reference to the attached drawings

FIG. 1 is a schematic block diagram depicting the data storage archive and media lifecycle management system architecture typical of prior art systems

FIG. 2 is a schematic block diagram showing the system architecture of a data storage archive and media lifecycle management system in accordance with the present invention

FIG. 3 is a schematic block diagram that depicts the distribution of functions and processes between a system core processor and a data mover in a preferred embodiment.

FIG. 4 is a schematic block diagram which shows the material movement manager in more detail with specific end point managers connecting to data storage devices.

FIG. 5 is a flow chart illustrating the flow of control of a data mover in the inventive data archive and media lifecycle management system.

FIG. 6 is a schematic block diagram which depicts the configuration of the two core systems.

FIG. 7 is a schematic diagram illustrating the fail-over process between redundant system cores within the inventive data archive and media lifecycle management system including Table 1 which describes systems events and their effects on the current state of a system core.

Claims

1. A distributed data archive and media lifecycle management system comprising: a first system core and a second system core managed as an adaptive cluster with the first system core being active and in control of the system at any one time, the first system core and the second system core both containing data movement control capabilities;a plurality of data storage devices connected with said first system core and second system core over a first data communication network;a database containing system configuration data and storage location data for all data stored on the plurality of storage devices connected to said first system core and second system core over said first data communication network;at least one data mover connected to said first system core via said first data communication network and connected to said plurality of data storage devices over a second data communication network;wherein the first system core communicates with said plurality of data storage devices over said first data communication network preparing them for writing or reading data and wherein said first system core also communicates with said database over said first data communication network obtaining data storage address data which is passed to said at least one data mover over the first data communication network enabling said at least one data mover to transfer data to or from the data storage devices over the second data communication network.

2. The system in claim 1 wherein the data bandwidth capacity of the second data communication network is greater than that of the first data communication network.

3. The system in claim 2 where the first data communication network uses a low bandwidth ethernet data channel.

4. The system in claim 2 where the second data communication network uses a high bandwidth fiber optic data channel.

5. The system in claim 1 wherein the first system core and the second system core are able to monitor the functional status of each data mover active in the data archive system.

6. The system in claim 5 wherein the first system core monitors the flow of data within the data archive system and upon the existence of predetermined conditions re-route data transfers from one data mover to another.

7. The system in claim 6 wherein the first core system is capable of adding or delete data movers from the data archive system.

8. The system in claim 1 wherein the plurality of system cores each contain clustering management capability that is active at all times.

9. The system in claim 8 wherein the clustering management capability of the active system core transmits process data on all running processes under its control to a database where it is stored and regularly polled by the active clustering management software of the non-active system cores to determine if there has been a failure in the operation of the active system core.

10. A data archive system for the storage and management of large amounts of data, comprising: a plurality of system cores operating as an adaptive cluster, wherein a first system core is active and in control until another of the plurality of system cores detects that the first system core has an operational issue, at which time a second system core of the plurality of system cores will become active and in control until the operational issue of the first system core is resolved;a plurality of data storage devices;a storage system database containing system configuration data and location data, wherein the location of data indicates the location of data stored on the plurality of data storage devices;at least one data mover for coordinating the transportation of data;a first communication network coordinating communication between the plurality of system cores, the storage system database and the at least one data mover, wherein the first communication network will communicate transfer instructions indicated where data is to be stored and delivered; anda second communication network coordinating communication between the first plurality of cores, the plurality of storage devices and the data mover, wherein the second communication network is configured to transport data to and from the data storage devices.

11. The data archive system of claim 10 wherein the second communication network has a data bandwidth capacity which is higher than the data bandwidth capacity of the first communication network.

12. The data archive system of claim 11 wherein the second communication network is a fiber channel network.

13. The data archive system of claim 11 wherein the first communication network is an Ethernet network.

14. The data archive system of claim 10 wherein the database contains audio-visual data.

15. The data archive system of claim 10 wherein the plurality of system cores include a cluster management system in which the active system core transmits process data reporting all running processes to a cluster management database, the cluster management system regularly polling the cluster management database to determine the potential existence of a system failure.

16. The data archive system of claim 15 wherein the cluster management system is active at all times.

17. The data archive system of claim 16 wherein the cluster management system will transfer control to the second core system upon the detection of a failure by the cluster management system.

18. The data archive system of claim 10 further comprising a second data mover capable of operating in conjunction with the existing data mover.