Remote data mirroring system using local and remote write pending indicators

Information

  • Patent Grant
  • 6647474
  • Patent Number
    6,647,474
  • Date Filed
    Tuesday, August 20, 2002
    22 years ago
  • Date Issued
    Tuesday, November 11, 2003
    21 years ago
Abstract
A primary data storage system is linked to a geographically remote secondary data storage system for automatically maintaining a remote copy of the data in the primary storage. The primary data storage system controller uses write pending indicators to control and coordinate the remote copy process. In response to receipt of data from the host computer, a first write pending indicator is set to write the data into at least one primary data storage device, and a second write pending indicator is set to copy the data to the secondary data storage system controller. The first write pending indicator is reset after the data is written to the primary data storage device, and the second write pending indicator is reset after receiving an acknowledgement back from the secondary data storage system controller.
Description




FIELD OF THE INVENTION




This invention relates to data storage, and more particularly, to a system and method for automatically providing and maintaining a copy or mirror of data stored at a location geographically remote from the main or primary data storage device.




BACKGROUND OF THE INVENTION




Nearly all data processing system users are concerned with maintaining back-up data in order to insure continued data processing operations should their data become lost, damaged, or otherwise unavailable.




Large institutional users of data processing systems which maintain large volumes of data such as banks, insurance companies, and stock market traders must and do take tremendous steps to insure back-up data availability in case of a major disaster. These institutions recently have developed a heightened awareness of the importance of data recovery and back-up in view of the many natural disasters and other world events including the bombing of the World Trade Center in New York City.




Currently, data processing system users often maintain copies of their valuable data on site on either removable storage media, or in a secondary “mirrored” storage device located on or within the same physical confines of the main storage device. Should a disaster such as fire, flood, or inaccessibility to a building occur, however, both the primary as well as the secondary or backed-up data will be unavailable to the user. Accordingly, more data processing system users are requiring the remote storage of back-up data.




One prior art approach at data back-up involves taking the processor out of service while back-up tapes are made. These tapes are then carried off premises for storage purposes. Should access to the backed-up data be required, the proper tape must be located, loaded onto a tape drive, and restored to the host system requiring access to the data. This process is very time consuming and cost intensive, both in maintaining an accurate catalog of the data stored on each individual tape, as well as storing the large number of tapes required to store the large amounts of data required by these institutions. Additionally and most importantly, it often takes twenty-four hours before a back-up tape reaches its storage destination during which time the back-up data is unavailable to the user.




Additionally, today's systems require a significant amount of planning and testing in order to design a data recovery procedure and assign data recovery responsibilities. Typically, a disaster recovery team must travel to the test site carrying a large number of data tapes. The team then loads the data onto disks, makes the required network connections, and then restores the data to the “test” point of failure so processing can begin. Such testing may take days or even weeks and always involves significant human resources in a disaster recovery center or back-up site.




Some providers of prior art data storage systems have proposed a method of data mirroring whereby one host Central Processing Unit (CPU) or processor writes data to both a primary, as well as a secondary, data storage device or system. Such a proposed method, however, overly burdens the host CPU with the task of writing the data to a secondary storage system and thus dramatically impacts and reduces system performance.




Accordingly, what is required is a data processing system which automatically and asynchronously, with respect to a first host system, generates and maintains a backup or “mirrored” copy of a primary storage device at a location physically remote from the primary storage device, without intervention from the host which seriously degrades the performance of the data transfer link between the primary host computer and the primary storage device.




SUMMARY OF THE INVENTION




This invention features a system which automatically, without intervention from a host computer system, controls storing of primary data received from a primary host computer on a primary data storage system, and additionally controls the copying of the primary data to a secondary data storage system controller which forms part of a secondary data storage system, for providing a back-up copy of the primary data on the secondary data storage system which is located in a geographically remote location from the primary data storage system.




Copying or mirroring of data from a primary data storage system to a secondary data storage system is accomplished without intervention of a primary or secondary host computer and thus, without affecting performance of a primary or secondary host computer system. Primary and secondary data storage system controllers are coupled via at least one high speed communication link such as a fiber optic link driven by LED's or laser.




At least one of the primary and secondary data storage system controllers coordinates the copying of primary data to the secondary data storage system and at least one of the primary and secondary data storage system controllers maintains at least a list of primary data which is to be copied to the secondary data storage device.




Additionally, the secondary data storage system controller provides an indication or acknowledgement to the primary data storage system controller that the primary data to be copied to the secondary data storage system in identical form as secondary data has been received or, in another embodiment, has actually been written to a secondary data storage device.




Accordingly, data may be transferred between the primary and secondary data storage system controllers synchronously, when a primary host computer requests writing of data to a primary data storage device, or asynchronously with the primary host computer requesting the writing of data to the primary data storage system, in which case the remote data copying or mirroring is completely independent of and transparent to the host computer system.




At least one of the primary data storage system controller and the secondary data storage system controller maintains a list of primary data which is to be written to the secondary data storage system. Once the primary data has been at least received or optionally stored on the secondary data storage system, the secondary data storage system controller provides an indication or acknowledgement of receipt or completed write operation to the primary data storage system.




At such time, the primary and/or secondary data storage system controller maintaining the list of primary data to be copied updates this list to reflect that the given primary data has been received by and/or copied to the secondary data storage system. The primary or secondary data storage system controllers and/or the primary and secondary data storage devices may also maintain additional lists for use in concluding which individual storage locations, such as tracks on a disk drive, are invalid on any given data storage device, which data storage locations are pending a format operation, which data storage device is ready to receive data, and whether or not any of the primary or secondary data storage devices are disabled for write operations.




Thus, an autonomous, host computer independent, geographically remote data storage system is maintained providing a system which achieves nearly 100 percent data integrity by assuring that all data is copied to a geographically remote site, and in those cases when a back-up copy is not made due to an error of any sort, an indication is stored that the data has not been copied, but instead must be updated at a future time.




Such a system is provided which is generally lower in cost and requires substantially less manpower and facilities to achieve than the prior art devices.




The present invention more particularly concerns the use of write pending indicators by the primary data storage system controller. In response to receipt of data from the host computer, a first write pending indicator is set to write the data into at least one primary data storage device, and a second write pending indicator is set to copy the data to the secondary data storage system controller. The first write pending indicator is reset after the data is written to the primary data storage device, and the second write pending indicator is reset after receiving an acknowledgement back from the secondary data storage system controller.











BRIEF DESCRIPTION OF THE DRAWINGS




These and other features and advantages of the present invention will be better understood when read together with the following drawings wherein:





FIG. 1

is a block diagram illustrating the system with remote data mirroring according to the present invention;





FIG. 2

is a schematic representation of a portion of an index or list maintained by the system of the present invention to determine various features including which primary data has been copied to a secondary disk; and





FIG. 3

is a schematic representation of an additional list or index maintained by the system of the present invention to keep track of additional items including an invalid data storage device track, device ready status and write disable device status.











DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS




The present invention features a system which provides a geographically remote mirrored data storage system which contains generally identical information to that stored on a primary data storage system. Utilizing such a system, data recovery after a disaster is nearly instantaneous and requires little, if any, human intervention. Using the present system, the data is retrieved from a remote device through the host data processing system.




A system in accordance with the present invention is shown generally at


10


,

FIG. 1

, and includes at site A, which is a first geographic location, a host computer system


12


as is well known to those skilled in the art. The host computer system


12


is coupled to a first and primary data storage system


14


. The host


12


writes data to and reads data from the primary data storage system


14


.




The primary data storage system


14


includes a primary data storage system controller


16


which receives data from the host


12


over data signal path


18


. The primary data storage system controller


16


is also coupled to a storage device


20


which may include a plurality of data storage devices


22




a


-


22




c


. The storage devices may include disk drives, optical disks, CD's or other data storage devices. The primary system controller


16


is coupled to the storage device


20


by means of data signal path


24


.




The primary data storage system controller


16


includes at least one channel adapter (C.A.)


26


which is well known to those skilled in the art and interfaces with host processing system


12


. Data received from the host is typically stored in cache


28


before being transferred through disk adapter (D.A.)


30


over data signal path


24


to the primary storage device


20


. The primary data storage controller


16


also includes a data director


32


which executes one or more sets of predetermined micro-code to control data transfer between the host


12


, cache memory


28


, and the storage device


20


. Although the data director


32


is shown as a separate unit, either one of a channel adapter


26


or disk adapter


30


may be operative as a data director, to control the operation of a given data storage system controller. Such a reconfigurable channel adapter and disk adapter is disclosed in Applicant's U.S. Pat. No. 5,335,352 entitled RECONFIGURABLE, MULTI-FUNCTION DATA STORAGE SYSTEM CONTROLLER SELECTIVELY OPERABLE AS AN INPUT CHANNEL ADAPTER AND A DATA STORAGE UNIT ADAPTER, which is fully incorporated herein by reference.




The primary data storage system


14


according to one embodiment of the present invention also includes a service processor


34


coupled to the primary data storage system controller


16


, and which provides additional features such as monitoring, repair, service, or status access to the storage system controller


16


.




The primary data storage system controller


16


of the present invention also features at least a second disk adapter


36


coupled to the internal bus


38


of the primary data storage system controller


16


. The second disk adapter


36


is coupled, via a high speed communication link


40


to a disk adapter


42


on a secondary data storage system controller


44


of a secondary data storage system


46


. Such high speed, point-to-point communication links between the primary and secondary data storage system controllers


16


and


44


include a fiber optic link driven by an LED driver, per IBM ESCON standard; a fiber optic link driven by a laser driver, and optionally T1 and T3 telecommunication links. Utilizing network connections, the primary and secondary data storage system controllers


16


and


44


may be connected to FDDI networks, T1 or T3 based networks and SONET networks.




The secondary data storage system


46


is located at a second site geographically removed from the first site. For this patent application, “geographically removed site” means not within the same building as the primary data storage system. There are presently known data processing systems which provide data mirroring to physically different data storage systems. The systems, however, are generally within the same building. The present invention is directed to providing complete data recovery in case of disaster, such as when a natural disaster such as a flood or a hurricane, or man-made disasters such as fires or bombings destroy one physical location, such as one building.




As in the case of the primary data storage system, the secondary data storage system


46


includes, in addition to the secondary data storage system controller


44


, a secondary data storage device


48


including a plurality of storage devices


50




a


-


50




c


. The plurality of storage devices on the secondary data storage system


46


, as well as the primary data storage system


14


, may have various volumes and usages such as a primary data storage device


50




a


which is primary with respect to the attached storage controller


44


and host


52


in the case of the secondary data storage system


46


, and the primary storage device


22




a


which is primary with respect to the first or primary host


12


in the case of the primary data storage system


14


.




Additionally, each storage device, such as storage device


48


, may include a secondary storage volume


50




b


which serves as the secondary storage for the primary data stored on the primary volume


22




a


of the primary data storage system


14


. Similarly, the primary data storage system


14


may include a secondary storage volume


22




b


which stores primary data received and copied from the secondary site and data storage system


46


and host


52


.




Additionally, each storage device


20


,


48


, may include one or more local volumes or storage devices


22




c


,


50




c


, which are accessed only by their locally connected data processing systems.




The secondary storage system controller


44


also includes at least a first channel adapter


54


which may receive data from an optionally connected secondary host


52


or an optionally connected hotsite host or CPU


56


. Optionally, the primary host


12


may include a data signal path


58


directly into the channel adapter


54


of the secondary data storage system


46


, while the optional secondary host


52


may include an optional data path


60


into the channel adapter


26


of the primary data storage system


14


. Although the secondary host


52


illustrated in

FIG. 1

is not required for remote data mirroring as described in the present patent application, such a host would be required for data retrieval if both the primary host


12


as well as the primary data storage system


14


would be rendered inoperative. Similarly, a hotsite host or CPU


56


may optionally be provided at a third geographically remote site to access the data stored in the secondary data storage system


46


.




The high speed link


40


between the primary and secondary data storage systems


14


and


46


is designed such that multiple links between the primary and secondary storage system may be maintained for enhanced availability of data and increased system performance. The number of links is variable and may be field upgradeable. Additionally, the service processor


34


of the primary data storage system


14


and the service processor


62


of the secondary data storage system


46


may also be coupled to provide for remote system configuration, remote software programming, and to provide a host base point of control of the secondary data storage system.




The secondary data storage system controller


44


also includes cache memory


64


which receives data from channel adapter


54


and disk adapter


42


, as well as disk adapter


66


which controls writing data to and from secondary storage device


48


. Also provided is a data director


68


which controls data transfer over communication bus


70


to which all the elements of the secondary data storage system controller are coupled.




An additional feature of the system


10


of

FIG. 1

is the ability to dynamically reconfigure channel adapters as disk adapters and disk adapters as channel adapters, as described in Applicant's U.S. Pat. No. 5,269,011 entitled DYNAMICALLY RECONFIGURABLE DATA STORAGE SYSTEM WITH STORAGE SYSTEM CONTROLLERS SELECTIVELY OPERABLE AS CHANNEL ADAPTERS OR STORAGE DEVICE ADAPTERS, which is fully incorporated herein by reference.




The primary and secondary data storage systems may optionally be connected by means of currently available, off-the-shelf channel extender equipment using bus and tag or ESCON interfaces.




The data storage system


10


of

FIG. 1

is designed to provide the copying of data from a primary data storage system to a physically remote secondary data storage system transparent to the user, and external from any influence of the primary host which is coupled to the primary data storage system. The system


10


of

FIG. 1

is designed to operate in at least two modes, the first being a real-time or synchronous mode wherein the primary and secondary storage systems must guarantee that the data exists and is stored in two physically separate data storage units before input/output completion; that is, before channel end and device end is returned to the primary host. Alternatively, the system


10


of

FIG. 1

is designed to operate in a point-in-time or asynchronous mode wherein the data is copied to the remote or secondary data storage system asynchronously from the time when the primary or local data processing system returns the input/output completion signal (channel end and device end) to the primary host. This eliminates any performance penalty if the communication link between the primary and secondary data storage systems is too slow, but creates the additional needs to manage the situation where data is not identical or in “sync” between the primary and secondary data storage systems.




Thus, in the real time or synchronous mode, the primary data storage system automatically controls the duplication or copying of data to the secondary data storage system controller transparently to the primary host computer. Only after data is safely stored in both the primary and secondary data storage system, as detected by an acknowledgement from the secondary storage system to the primary storage system, does the primary data storage system acknowledge to the primary host computer that the data is synchronized. Should a disaster or facility outage occur at the primary data storage system site, the user will simply need to initialize the application program in the secondary data storage system utilizing a local host (


52


) or a commercial hotsite CPU or host


56


.




The link between the primary and secondary storage system controllers


14


and


46


may be maintained in a unidirectional mode wherein the primary data storage system controller monitors and controls data copying or mirroring. Alternatively, a bi-directional implementation may be used wherein either controller can duplicate data to the other controller, transparently to the host computer. Should a disaster or facilities outage occur, recovery can be automatic with no human intervention since the operational host computer already has an active path (


40


,


58


,


60


) to the data through its local controller. While offering uninterrupted recovery, performance will be slower than in an unidirectional implementation due to the overhead required to manage intercontroller tasks.




In the second, point-in-time mode of operation, the primary data storage system transparently duplicates data to the secondary data storage system after the primary data storage system acknowledges to the host computer, via channel end and device end, that the data has been written to the storage device and the input/output operation has been completed. This eliminates the performance impact of data mirroring over long distances. Since primary and secondary data are not synchronized, however, the primary data storage system must maintain a log file of pending data which has yet to be written to the secondary data storage device. Such data may be kept on removable, non-volatile media, in the cache memory of the primary or secondary data storage system controller as will be explained below, or in the service processor


34


,


62


.




Accordingly, a feature of the system


10


of

FIG. 1

is the ability of a data storage system to control the transfer or copying of data from a primary data storage system to the secondary data storage system, independent of and without intervention from one or more host computers. Most importantly, in order to achieve optimum data mirroring performance, such data mirroring or copying should be performed asynchronously with input/output requests from a host computer. Accordingly, since data will not be immediately synchronized between the primary and secondary data storage systems, data integrity must be maintained by maintaining an index or list of various criteria including a list of data which has not been mirrored or copied, data storage locations for which a reformat operation is pending, a list of invalid data storage device locations or tracks, whether a given device is ready, or whether a device is write-disabled. Information must also be included as to the time of the last operation so that the data may later be synchronized should an error be detected.




A feature of the system


10


of

FIG. 1

is that both the primary or secondary data storage systems maintain a table of the validity of data in the other storage system. As disclosed in U.S. Pat. No. 5,206,939 entitled SYSTEM AND METHOD FOR DISK MAPPING AND DATA RETRIEVAL, which is fully incorporated herein by reference, the present system maintains a list or index, utilizing one or more flag bits, in a hierarchical structure, on each physical and logical data storage device.




In the system


10


of

FIG. 1

, however, such information is kept on both devices for each individual system as well as the other data storage system. Thus, as illustrated in

FIG. 2

in the partial list or table


100


, each data storage system maintains an indication of write or copy pending


102


of both the primary data (M


1


)


104


, and the secondary data (M


2


)


106


. Similarly, an index is maintained of a pending format change since a disk format change may be accomplished. The format pending bits


108


including a first primary bit


110


and a second secondary bit


112


indicate that a format change has been requested and such change must be made on the disk.




Thus, when a host computer writes data to a primary data storage system, it sets both the primary and secondary bits


104


,


106


of the write pending bits


102


when data is written to cache. For these examples, the M


1


bit will refer to the primary data storage system and the M


2


bit will refer to the secondary data storage system. When the primary data storage system controller's disk adapter writes the data to the primary data storage device, it will reset bit


104


of the write pending indicator bits


102


. Similarly, once the secondary data storage system has written the data to the secondary data storage device, the secondary data storage write pending indicator bit


106


will be reset.




The service processors in one embodiment of the present invention will periodically scan the index table for write pending indicator bits and invoke a copy task which copies the data from the primary data storage device to the secondary. In addition, one or more of the spare index or table bits


114


,


116


may be utilized to store other data such as time stamp, etc.




In addition to the write pending and format pending bits described above, the system


10


also includes several additional general purpose flags to assist in error recovery. As shown in

FIG. 3

, invalid track flags


120


including primary bit


122


and secondary bit


124


are utilized and maintained on each data storage device to indicate that the data storage location such as a track, does not contain valid data. Another background task running on the data storage system such as in the service processor constantly checks invalid track bits on each data storage device, and if a bit is found to be set, the copy task is invoked to copy the data from the known good device to the device with the invalid flag track set. Additional flags may be provided such as the device ready flags


126


including bits


128


and


130


which serve to indicate that the device is ready. Similarly, write disable flags


132


may be provided which indicate that a particular primary device or drive


134


or secondary device or drive


136


can presently not be written to. Data can still be copied to the good or enabled drive and then later copied to the disabled drive. If one drive or device is bad, the system


10


of

FIG. 1

will set all tracks of that drive as not valid to later cause a copy of all the data.




Accordingly, each data storage device keeps data validity information about its mirrored device. If for some reason a device is not accessible, either the primary or the secondary device, every new write command goes to the accessible mirrored device along with information that the not accessible device has a track which is not valid. As soon as the non-accessible device becomes accessible, then automatically, as a background operation, the drives re-synchronize. In the case when a specific track is not shown on both the primary and secondary storage system, an indication of such will be assigned and the user will be alerted. A utility operating on the service processors will give the user a report of all the non-valid (out of sync) tracks. This report can be transferred from one site to another over the link


63


,

FIG. 1

, that connects the two service processors


34


,


62


.




Modifications and substitutions by one of ordinary skill in the art are considered to be within the scope of the present invention, which is not to be limited except by the claims which follow.



Claims
  • 1. A remote copy system for automatically providing a remote copy of data from a host computer, said system comprising:a first data storage system including a first data storage system controller and at least one first data storage device, the first data storage system controller having a first cache memory, the first data storage system controller receiving the data from the host computer and initially storing the data in the first cache memory and subsequently writing the data into said at least one first data storage device; and a second data storage system located remote from the first data storage system, the second data storage system including a second data storage system controller and at least one second data storage device, the second data storage system controller having a second cache memory, the first data storage system controller being coupled to the second data storage system controller for copying the data from the first data storage system controller to the second data storage system controller, the second data storage system controller storing the data in the second cache memory and writing the data into said at least one second data storage device to produce the remote copy of the data from the host computer; the first data storage system controller coordinating and controlling the copying of the data to the second data storage system controller wherein, in response to receipt of the data from the host computer, a first write pending indicator is set to write the data into said at least one first data storage device, and a second write pending indicator is set to copy the data to the second data storage system controller, the first write pending indicator being reset after the data is written to said at least one first data storage device and the second write pending indicator being reset after receiving an acknowledgement back from the second data storage system controller.
  • 2. The remote copy system as claimed in claim 1, wherein the acknowledgement back from the second data storage system controller indicates that the second data storage system controller has received the data.
  • 3. The remote copy system as claimed in claim 1, wherein the acknowledgement back from the second data storage system controller indicates that the data has actually been written to said at least one second data storage device.
  • 4. The remote copy system as claimed in claim 1, wherein the first data storage system returns an i/o completion signal to the host computer, and the data is copied to the second data storage system asynchronously from the time when the first data storage system returns the i/o completion signal to the host computer.
  • 5. The remote copy system as claimed in claim 1, wherein the first data storage system controller returns an i/o completion signal to the host computer after the data has been safely stored in both the first data storage system and in the second data storage system as indicated by the acknowledgement from the second data storage system.
  • 6. The remote copy system as claimed in claim 1, wherein the first data storage system controller includes at least one disk adapter for writing the data from the first cache memory to said at least one first data storage device, and said at least one disk adapter resets the first write pending indicator when it writes the data to said at least one first data storage device.
  • 7. The remote copy system as claimed in claim 1, wherein the second write pending indicator is reset when the second data storage system has written the data to said at least one second data storage device.
  • 8. The remote copy system as claimed in claim 1, wherein the copying of the data from the first data storage system to the second data storage system is controlled independent of and without intervention from any host computer.
  • 9. The remote copy system as claimed in claim 1, wherein the first and second write pending indicators are flags in at least one table of flag bits.
  • 10. The remote copy system as claimed in claim 9, wherein said at least one table of flag bits is maintained by the first data storage system and kept on said at least one first data storage device.
  • 11. The remote copy system as claimed in claim 1, wherein the first and second write pending indicators are flags in a first table of flag bits maintained by the first data storage system and kept on said at least one first data storage device, and the flags are also kept on said at least one second data storage device in a second table of flag bits maintained by the second data storage system.
  • 12. A remote copy system for automatically providing a geographically remote copy of data from a host computer, said system comprising:a first data storage system including a first data storage system controller and at least one first data storage device, the first data storage system controller having a first cache memory, the first data storage system controller receiving the data from the host computer and initially storing the data in the first cache memory and subsequently writing the data into said at least one first data storage device; and a second data storage system geographically removed from the first data storage system, the second data storage system including a second data storage system controller and at least one second data storage device, the second data storage system controller having a second cache memory, the first data storage system controller being coupled to the second data storage system controller for copying the data from the first data storage system controller to the second data storage system controller, the second data storage system controller storing the data in the second cache memory and writing the data into said at least one second data storage device to produce the remote copy of the data from the host computer independent of the host computer; the first data storage system controller coordinating and controlling the copying of the data to the second data storage system controller wherein, in response to receipt of the data from the host computer, when the data is stored into the first cache memory, a first write pending indicator is set to write the data into said at least one first data storage device, and a second write pending indicator is set to copy the data to the second data storage system controller, the first write pending indicator being reset after the data is written to said at least one first data storage device and the second write pending indicator being reset after receiving an acknowledgement back from the second data storage system controller that the data has at least been received by the second data storage system controller; wherein the first data storage system returns an i/o completion signal to the host computer, and the data is copied to the second data storage system asynchronously from the time when the first data storage system returns the i/o completion signal to the host computer; and wherein the copying of the data from the first data storage system to the second data storage system is controlled independent of and without intervention from any host computer.
  • 13. The remote copy system as claimed in claim 12, wherein the acknowledgement back from the second data storage system controller indicates that the data has actually been written to said at least one second data storage device.
  • 14. The remote copy system as claimed in claim 12, wherein the first data storage system controller includes at least one disk adapter for writing the data from the first cache memory to said at least one first data storage device, and said at least one disk adapter resets the first write pending indicator when it writes the data to said at least one first data storage device.
  • 15. The remote copy system as claimed in claim 12, wherein the second write pending indicator is reset when the second data storage system has written the data to said at least one second data storage device.
  • 16. The remote copy system as claimed in claim 12, wherein the first and second write pending indicators are flags in at least one table of flag bits.
  • 17. The remote copy system as claimed in claim 16, wherein said at least one table of flag bits is maintained by the first data storage system and kept on said at least one first data storage device.
  • 18. The remote copy system as claimed in claim 12, wherein the first and second write pending indicators are flags in a first table of flag bits maintained by the first data storage system and kept on said at least one first data storage device, and the flags are also kept on said at least one second data storage device in a second table of flag bits maintained by the second data storage system.
  • 19. A remote copy system for automatically providing a geographically remote copy of data from a host computer, said system comprising:a first data storage system including a first data storage system controller and at least one first data storage device, the first data storage system controller having a first cache memory, the first data storage system controller receiving the data from the host computer and initially storing the data in the first cache memory and subsequently writing the data into said at least one first data storage device; and a second data storage system geographically removed from the first data storage system, the second data storage system including a second data storage system controller and at least one second data storage device, the second data storage system controller having a second cache memory, the first data storage system controller being coupled to the second data storage system controller for copying the data from the first data storage system controller to the second data storage system controller, the second data storage system controller storing the data in the second cache memory and writing the data into said at least one second data storage device to produce the remote copy of the data from the host computer independent of the host computer; the first data storage system controller coordinating and controlling the copying of the data to the second data storage system controller wherein, in response to receipt of the data from the host computer, a first write pending indicator is set to write the data into said at least one first data storage device, and a second write pending indicator is set to copy the data to said second data storage system controller, the first write pending indicator being reset after the data is written to said at least one first data storage device and the second write pending indicator being reset after receiving an acknowledgement back from the second data storage system controller that the data has at least been received by the second data storage system controller; wherein the copying of the data from the first data storage system to the second data storage system is controlled independent of and without intervention from any host computer; and wherein the first and second write pending indicators are flags in a first table of flag bits maintained by the first data storage system and kept on said at least one first data storage device, and the flags are also kept on said at least one second data storage device in a second table of flag bits maintained by the second data storage system.
  • 20. The remote copy system as claimed in claim 19, wherein the first data storage system controller includes at least one disk adapter for writing the data from the first cache memory to said at least one first data storage device, and said at least one disk adapter resets the first write pending indicator when it writes the data to said at least one first data storage device.
RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No. 09/709,814 filed Nov. 10, 2000, now U.S. Pat. No. 6,502,205 which is a divisional of U.S. patent application Ser. No. 09/061,708 filed Apr. 17, 1998 (U.S. Pat. No. 6,173,377 issued Jan. 9, 2001), which is a continuation of U.S. patent application Ser. No. 08/654,511 filed May 28, 1996 (U.S. Pat. No. 5,742,792 issued Apr. 21, 1998), which is a continuation-in-part of U.S. patent application Ser. No. 08/052,039 filed Apr. 23, 1993, entitled REMOTE DATA MIRRORING (U.S. Pat. No. 5,544,347 issued Aug. 6, 1996), which are all incorporated herein by reference.

US Referenced Citations (215)
Number Name Date Kind
3588839 Belady et al. Jun 1971 A
3623014 Doelz et al. Nov 1971 A
3771137 Barner et al. Nov 1973 A
3835260 Prescher et al. Sep 1974 A
3866182 Yamada et al. Feb 1975 A
3936806 Batcher Feb 1976 A
4020466 Cordi et al. Apr 1977 A
4057849 Ying et al. Nov 1977 A
4084231 Capozzi et al. Apr 1978 A
4094000 Brudevold Jun 1978 A
4124843 Bramson et al. Nov 1978 A
4150429 Ying Apr 1979 A
4161777 Behnase et al. Jul 1979 A
4193113 Crkowski Mar 1980 A
4204251 Brudevold May 1980 A
4249802 Muchel et al. Feb 1981 A
4342079 Stewart et al. Jul 1982 A
4361855 Igarashi Nov 1982 A
4368532 Imazeki et al. Jan 1983 A
4384274 Mao May 1983 A
4396984 Videki, II Aug 1983 A
4430727 Moore et al. Feb 1984 A
4453215 Reid Jun 1984 A
4464713 Benhase et al. Aug 1984 A
4523206 Sasscer Jun 1985 A
4523275 Swenson et al. Jun 1985 A
4530055 Hamstra et al. Jul 1985 A
4577272 Ballew et al. Mar 1986 A
4594577 Mao Jun 1986 A
4598357 Swenson et al. Jul 1986 A
4608688 Hanson et al. Aug 1986 A
4618827 Redington et al. Oct 1986 A
4634100 Julich et al. Jan 1987 A
4638424 Beglin et al. Jan 1987 A
4686620 Ng Aug 1987 A
4698808 Ishii Oct 1987 A
4707783 Lee et al. Nov 1987 A
4710870 Blackwell et al. Dec 1987 A
4726024 Guziak et al. Feb 1988 A
4727544 Brunner et al. Feb 1988 A
4736407 Dumas Apr 1988 A
4751578 Reiter et al. Jun 1988 A
4755928 Johnson et al. Jul 1988 A
4769764 Levanon Sep 1988 A
4771375 Beglin et al. Sep 1988 A
4773035 Lee et al. Sep 1988 A
4779189 Legvold et al. Oct 1988 A
4782486 Lipcon et al. Nov 1988 A
4783834 Anderson et al. Nov 1988 A
4785472 Shapiro Nov 1988 A
4797750 Karweit Jan 1989 A
4805106 Pfeifer Feb 1989 A
4807186 Ohnishi et al. Feb 1989 A
4814592 Bradt et al. Mar 1989 A
4823333 Satoh et al. Apr 1989 A
4831558 Shoup et al. May 1989 A
4837680 Crockett et al. Jun 1989 A
4841475 Ishizuka Jun 1989 A
4849978 Dishon et al. Jul 1989 A
4862411 Dishon et al. Aug 1989 A
4866611 Cree et al. Sep 1989 A
4879718 Sanner Nov 1989 A
4916605 Beardsley et al. Apr 1990 A
4949187 Cohen Aug 1990 A
4958270 McLaughlin et al. Sep 1990 A
4959774 Davis Sep 1990 A
4960031 Farrand Oct 1990 A
4974156 Harding et al. Nov 1990 A
4985695 Wilkinson et al. Jan 1991 A
5003382 Omi et al. Mar 1991 A
5007053 Iyer et al. Apr 1991 A
5016876 Loffredo May 1991 A
5029199 Jones et al. Jul 1991 A
5051887 Berger et al. Sep 1991 A
5060142 Menon et al. Oct 1991 A
5060185 Naito et al. Oct 1991 A
5072368 Foreman et al. Dec 1991 A
5083271 Thacher et al. Jan 1992 A
5087968 Omi et al. Feb 1992 A
5088021 McLaughlin et al. Feb 1992 A
5089958 Horton et al. Feb 1992 A
5093787 Simmons Mar 1992 A
5097439 Patriquin et al. Mar 1992 A
5099485 Bruckert et al. Mar 1992 A
5101492 Schultz et al. Mar 1992 A
5109492 Noguchi et al. Apr 1992 A
5123099 Shibata et al. Jun 1992 A
5127048 Press et al. Jun 1992 A
5132787 Omi et al. Jul 1992 A
5134711 Asthana et al. Jul 1992 A
5140592 Idleman et al. Aug 1992 A
5146576 Beardsley et al. Sep 1992 A
5146605 Beukema et al. Sep 1992 A
5155814 Beardsley et al. Oct 1992 A
5155835 Belsan Oct 1992 A
5155845 Beal et al. Oct 1992 A
5157663 Major et al. Oct 1992 A
5157770 Beardsley et al. Oct 1992 A
5159671 Iwami Oct 1992 A
5170471 Bonevento et al. Dec 1992 A
5170480 Mohan et al. Dec 1992 A
5175837 Arnold et al. Dec 1992 A
5175839 Ikeda et al. Dec 1992 A
5185864 Bonevento et al. Feb 1993 A
5185884 Martin et al. Feb 1993 A
5201053 Benhase et al. Apr 1993 A
5202887 Ueno et al. Apr 1993 A
5206939 Yanai et al. Apr 1993 A
5210865 Davis et al. May 1993 A
5212772 Masters May 1993 A
5212784 Sparks May 1993 A
5235601 Stallmo et al. Aug 1993 A
5235687 Bacot et al. Aug 1993 A
5235690 Beardsley et al. Aug 1993 A
5235692 Ayres et al. Aug 1993 A
5239637 Davis et al. Aug 1993 A
5239659 Rudeseal et al. Aug 1993 A
5257144 Curasi et al. Oct 1993 A
5263154 Eastridge et al. Nov 1993 A
5269011 Yanai et al. Dec 1993 A
5274645 Idleman et al. Dec 1993 A
5276867 Kenley et al. Jan 1994 A
5276871 Howarth Jan 1994 A
5285451 Henson et al. Feb 1994 A
5307481 Shimazaki et al. Apr 1994 A
5313664 Sugiyama et al. May 1994 A
5317731 Dias et al. May 1994 A
5321826 Ushiro Jun 1994 A
5335352 Yanai et al. Aug 1994 A
5341493 Yanai et al. Aug 1994 A
5343477 Yamada Aug 1994 A
5357509 Ohizumi Oct 1994 A
5367698 Webber et al. Nov 1994 A
5375232 Legvold et al. Dec 1994 A
5377342 Sakai et al. Dec 1994 A
5379412 Eastridge et al. Jan 1995 A
5381539 Yanai et al. Jan 1995 A
5392390 Crozier Feb 1995 A
5404500 Legvold et al. Apr 1995 A
5404508 Konrad et al. Apr 1995 A
5410667 Belsan et al. Apr 1995 A
5412668 Dewey May 1995 A
5428796 Iskiyan et al. Jun 1995 A
5432922 Polyzois et al. Jul 1995 A
5434994 Shaheen et al. Jul 1995 A
5435004 Cox et al. Jul 1995 A
5437022 Beardsley et al. Jul 1995 A
5440727 Bhide et al. Aug 1995 A
5446872 Ayres et al. Aug 1995 A
5450577 Lai et al. Sep 1995 A
5459857 Ludlam et al. Oct 1995 A
5463752 Benhase et al. Oct 1995 A
5469503 Butensky et al. Nov 1995 A
5473776 Nosaki et al. Dec 1995 A
5479654 Squibb Dec 1995 A
5495601 Narang et al. Feb 1996 A
5495606 Borden et al. Feb 1996 A
5497483 Beardsley et al. Mar 1996 A
5502811 Ripberger Mar 1996 A
5504888 Iwamoto et al. Apr 1996 A
5506986 Healy Apr 1996 A
5522031 Ellis et al. May 1996 A
5526484 Casper et al. Jun 1996 A
5535381 Kopper Jul 1996 A
5537533 Staheli et al. Jul 1996 A
5544345 Carpenter et al. Aug 1996 A
5544347 Yanai et al. Aug 1996 A
5555371 Duyanovich et al. Sep 1996 A
5574950 Hathorn et al. Nov 1996 A
5579318 Reuss et al. Nov 1996 A
5584039 Johnson et al. Dec 1996 A
5592618 Micka et al. Jan 1997 A
5592660 Yokota et al. Jan 1997 A
5596706 Shimazaki et al. Jan 1997 A
5613106 Thurman et al. Mar 1997 A
5613155 Baldiga et al. Mar 1997 A
5615329 Kern et al. Mar 1997 A
5615364 Marks Mar 1997 A
5625818 Zarmer et al. Apr 1997 A
5633999 Clowes et al. May 1997 A
5537585 Blickenstaff et al. Jul 1997 A
5649089 Kilner Jul 1997 A
5649152 Ohran et al. Jul 1997 A
5673382 Cannon et al. Sep 1997 A
5680580 Beardsley et al. Oct 1997 A
5680640 Ofek et al. Oct 1997 A
5682396 Yamamoto et al. Oct 1997 A
5682513 Candelaria et al. Oct 1997 A
5689732 Kondo Nov 1997 A
5692155 Iskiyan et al. Nov 1997 A
5701429 Legvold et al. Dec 1997 A
5721916 Pardikar Feb 1998 A
5724500 Shinmura et al. Mar 1998 A
5734813 Yamamoto et al. Mar 1998 A
5740397 Levy Apr 1998 A
5742792 Yanai et al. Apr 1998 A
5758149 Bierma et al. May 1998 A
5764972 Crouse et al. Jun 1998 A
5819310 Vishlitzky et al. Oct 1998 A
5835954 Duyanovich et al. Nov 1998 A
5845295 Houseman et al. Dec 1998 A
5852715 Raz et al. Dec 1998 A
5889935 Ofek et al. Mar 1999 A
5901327 Ofek May 1999 A
5909483 Weare et al. Jun 1999 A
5978565 Ohran et al. Nov 1999 A
5987566 Vishlitzky et al. Nov 1999 A
6035412 Tamer et al. Mar 2000 A
6052797 Ofek et al. Apr 2000 A
6044444 Ofek May 2000 A
6092066 Ofek Jul 2000 A
6101497 Ofek Aug 2000 A
6108748 Ofek et al. Aug 2000 A
6484187 Kern et al. Nov 2002 B1
6502205 Yanai et al. Dec 2002 B1
Foreign Referenced Citations (40)
Number Date Country
167322 Jan 1986 EP
0239323 Sep 1987 EP
0323123 Dec 1988 EP
593062 Apr 1994 EP
672985 Sep 1995 EP
674263 Sep 1995 EP
2086625 May 1992 GB
56-153460 Nov 1981 JP
57-111900 Jul 1982 JP
62-89150 Nov 1987 JP
62-274448 Nov 1987 JP
63-278155 Nov 1988 JP
1-19437 Jan 1989 JP
1-19438 Jan 1989 JP
1-120652 May 1989 JP
2-32418 Feb 1990 JP
2-32419 Feb 1990 JP
2-32420 Feb 1990 JP
2-35537 Feb 1990 JP
2-37418 Feb 1990 JP
2-91716 Mar 1990 JP
2-91717 Mar 1990 JP
2-93721 Apr 1990 JP
3-40121 Feb 1991 JP
3-92942 Sep 1991 JP
3-93942 Sep 1991 JP
3-250257 Nov 1991 JP
3-256143 Nov 1991 JP
3-256144 Nov 1991 JP
3-256146 Nov 1991 JP
5-158631 Jun 1993 JP
5-233162 Sep 1993 JP
6-175788 Jun 1994 JP
7-73078 Mar 1995 JP
7-191811 Jul 1995 JP
7-239799 Sep 1995 JP
WO9400816 Jan 1994 WO
WO9425919 Nov 1994 WO
WO8502926 Jul 1995 WO
WO 9523376 Aug 1995 WO
Non-Patent Literature Citations (147)
Entry
D.L. Burkes & R.K. Treiber, “Design Approaches for Real-Time Transaction Processing Remote Site Recovery,” Computer Society International Conference (COMPCON), Spring Meeting, Los Alamitos, Feb. 26-Mar. 2, 1990, No., CONF. 35, Feb. 23, 1990, Institute of Electrical and Electronics Engineers, New York, N.Y., pp. 568-572.
Supplementary European Search Report for EP 94 91 4223, Jul. 1, 1997. International Search Report for PCT/US/94/04326, Jul. 20, 1994.
International Search Report for PCT/US/94/04326, Jul. 20, 1994.
PCT Written Opinion for PCT/US94/04326, Feb. 28, 1995.
PCT International Preliminary Examination Report for PCT/US94/04326, Jul. 18, 1995.
SFT Netware 286 Maintenance, #100-313-001, 100/Rev1.00, Novel Incorporated, Provo, Utah, Nov. 1987.
SFT Netware 286 Installation, #100-312-001, 100/Rev1.00, Novel Incorporated, Provo, Utah, Nov. 1987.
SFT Netware 286 Installation Supplement, #100-000225-001, 26/Rev1.02, Novel Incorporated, Provo, Utah, Jun. 1986.
SFT Netware 68 Maintenance, #100-000289-001, 64/Rev1.00, Novel Incorporated, Provo, Utah, Nov. 1986.
SFT Netware 68 Installation, #100-000317-001, 103/Rev1.00, Novel Incorporated, Provo, Utah, Feb. 1988.
SFT Netware 868 Installation Supplement, #100-000286-001, 61/Rev1.00, Novel Incorporated, Provo, Utah, Dec. 1986.
Hank Cote--New Horizons in Accessing Large Amounts of On-Line Data, pp. 71-75, 1982.
Matt Kramer--Fault-Tolerant LANs Guard Against Malfunction, Data Loss, pp. C26-C35, Sep. 1987.
Carol Grossman--Planning for 3990 Extended Functions, pp. 245-258, 1989, 30th Annual Guide Conference.
Bob Buchanan--Administrative Error Causes Most Major System Failures (3 pgs.), Feb. 1988, Government Computer News, V. 17, N. 4, p. 340.
Virginia Dudek--Planning for Network Disasters (4 pgs.), Jun. 12, 1989, MIS Week, vol. 10, N. 24.
Jen Bozman--Escon An Aid To Remote Disks (3 pgs.), V. 25, N. 1, p. 29, Jan. 7, 1991.
Gary A. Boles--A Raid Stack: Micropolis Raidion, Network Computer, Jun. 1992.
Blaine Homer--Raid Level 5 Protects Your Data With Interleaved Parity, LAN Times, May 25, 1992, vol. 9.
Michael Caton--Micropolis Array Secures Data, PC Week, 1992.
RAIDON Fault-Tolerant Disk Arrays Brochure, 1993.
Storagetek 2Q Earnings Down, Iceberg Ready for Testing by Jim Mallory, Newsbytes, Jul. 15, 1993.
Network Backup Evolves, by Michael Peterson et al., PC Magazine, vol. 12, No. 16, Sep. 28, 1993, p. 277(18).
Your Next LAN Storage Could Be a Mainframe, Jeff Moad, Datamination, vol. 39, No. 6, Mar. 15, 1993, p.71.
Heard Any Good Rumors Lately? (Network Management), By Jill Huntington-Lee, LAN Computing, vol. 5, No. 5, Feb. 1994, p. 22.
Special Report: The AT&T Worlwide Intelligent Network--Scope, Nature and Management, Edge, vol. 7, No. 224, Nov. 6, 1992, p. 9.
Software Extends Unix Distributed Computing, by Evan O. Grossman, PC Week, Jun. 5, 1989, p. 35 and p. 40.
IBM Funds Pack Firm, Electronic News, May 22, 1989, p. 12.
IBM Plans U.S. Test of File-Transfer, Network World, Sep. 5, 1988, pp. 15, 17.
IBM Invests In Firm That Offers Unix File-Sharing, Network World, May 22, 1989, p. 4.
IBM Plans Speedy Nationwide File System, Computer Systems News, Sep. 5, 1988, p. 56.
Design Alternatives for Disk Duplexing, by Spencer W. Ng, IBM Research Report, Jan. 30, 1987.
Disk Arm Movement In Anticipation of Future Requests, by Richard P. King, IBM Research Report, Dec. 16, 1987.
Ubik: Replicated Servers Made Easy, by Michael Leon Kazar, pp. 60-67, 1989.
Carnegie-Mellon Forges Standard Look For Multivendor Net, Computerworld, Jan. 30, 1989, pp. SR/8-SR/9.
A Multi-Media Message System for Andrew, USENIX Winter Conference, Feb. 9-12, 1988, pp. 37-42.
Synchronization and Caching Issues in the Andrew File System, USENIX Winter Conference, Feb. 9-12, 1988, pp. 27-36.
Comparative Study of Some UNIX Distributed File Systems, EUUG, Sep. 22-24, 1986, pp. 73-82.
Volumes--The Andrew File System Data Structuring Primitive, EUUG Autumn 1986, Sep. 22-24, 1986, pp. 473-480.
Unix Plays Role in Corporate Networking, MIS Week, vol. 10, No. 28, Jul. 17, 1989, pp. 22-23.
Sun's Open System Made by Marketplace, “Unlike Minis, Unix Machines, Pcs and Even VMS Can Be Glued” by NFS, Software Magazine, vol. 9, No. 12, Oct. 1989, pp. 72-75, Cashin, Jerry.
Protect Your Data! (Database Report), by Michael Liczbanski, Data Based Advisor, vol. 10, No. 5, May 1992, p. 114.
Network Planning to the Rescue; Network Managers Should Design and Test a Network Recovery Plan That Lets Business Continue Uninterrupted Even If Disaster Strikes, by Ramond W. Rudnitskas et al., Networking Management, vol. 9, No. 11, Oct. 1991, p. 46.
If LANS Eye Host Status, Quality Today Falls Short, by Barbara Bochenski, Software Magazine, vol. 11, No. 12, Oct. 1991, p. 66.
Safe and Secure, by Peter Stephenson, LAN Magazine, vol. 6, No. 9, Sep. 1991, p. 34.
Firing Up Fault Tolerance by Brian O'Connell, DEC Professional, vol. 10, No. 6, Jun. 1991, p. 118.
A Which Computer? Guide to Backup Systems, by Chris Bidmead, Which Computer?, Sep. 1989, p. 62.
Optical Disks Compete With Videotape and Magnetic Storage Media, by Henry Urrows et al., Optical Information Systems, vol. 8, No. 3, May-Jun. 1988, p. 101.
Preventing Disasters: Whipping Up an Emergency Response Plan, Network Computing, Feb. 1992, p. 81.
Abbey Banking on FDDI, Communications Week International, Mar. 4, 1991, p. 22.
X.25 Data Networks and IBM: A Troublesome Connection, by Jan Johnson, Computer Decisions, vol. 19, Jan. 12, 1987, p. 16.
DSC Communications Corp. Debuts NEXOS LAN Product Line at Interface, Demos Enhanced Performance, Fault Tolerance and WAN Connectivity, News Release, Mar. 28, 1988.
Analysis of Scanning Policies For Reducing Disk Seek Times, by E. G. Coffman, L. A. Klimko and Barbara Ryan-Siam, Journal on Computing, Sep. 1972, vol. 1, No. 3, pp. 269-280.
Configuring Tandem Disk Subsystems, by Scott Sitler, Tandem Systems Review, Dec. 1986, pp. 80-91.
Technical Report Investigation of Selected Disk Systems, Oct. 1976, Teledyne Brown Engineering, Huntsville, Alabama (18 pages).
Universal Server Under Development, MacWEEK, vol. 3, No. 39, Oct. 31, 1989, p. 5.
Unix Plays Role In Corporate Networking-Vendors Look Unix-Based Network Management Tools as Networking Standards Slowly Evolve, MIS Week, Jul. 17, 1989, pp. 22-23, vol. 10, No. 28.
The 28 Runners Declared for Open Software Foundations, Distributed Computing Stakes, Computergram International, No. 1301, CGI11909912, Nov. 9, 1989.
Some Platforms (Theory of Self-Organizing Systems), Release 1.0, vol. 89, No. 6, Jun. 27, 1989, pp. 12-16.
Self-Organizing Systems (Overview of Detailed Examples of Self-Organizing Systems, Includes related article About the Importance of Self-Organizing Systems), Release 1.0, vol. 89, No. 6, Jun. 27, 1989, pp. 1-9.
IBM-Funded Startup Developing Unix Add-On, PC Week, vol. 6, No. 22, Jun. 5, 1989, p. 40.
Uniting File Systems; Experimental Large Scale, Distributed File Systems Are Now Being Created (included related article on the History of the AFS Project), UNIX Review, vol. 7, No. 3, Mar. 1989, pp. 61-70.
IBM Backs Transarc, Formed to Offer Andrew File System Distributed Unix Database, Computergram International, No. 1180, CGI05190001, May 19, 1989.
A Comparative Analysis of Disk Scheduling Policies, by Toby J. Reorey and Tad B. Pinerton, Communications of the ACM, Mar. 1972, vol. 15, No. 3, pp. 177-184.
Disk Scheduling: FCFS vs. SSTF Revisited, by Micha Hofri, Communications of the ACM, Nov. 1980, vol. 23, No. 11, pp. 645-653.
Using Multiple Replica Classes to Improve Performance in Distributed System, by Peter Triantafillou and David Tayloer, The 11th International Conference on Distributed Computing Systems, May 20-24, 1991, pp. 420-428.
Government Computer News, Oct. 9, 1987, Richard A. Damca, Microcomputing “Patch Fault Tolerance: A Strong Second Team,” vol. 7, Issue 149 P27(2), Dec. 17, 1990.
Computer Reseller, Feb. 19, 1990, Joel Shore, Mirrored Servers' a reality, pp. 51, 52, 62.
Infoworld Articles, Mar. 12, 1990, Vendors Offer Mirroring Solutions P(31).
Wilkinson, Compact and Novell team for fault tolerance: mirrored servers to cut down on costly downtime, Article MIS Week, vol. 11, Issue 7, Feb. 12, 1990.
Article PC Week, vol. 4, Issue 37, Sep. 15, 1987, Matt Kramer, Fault-Tolerant LANS Guard Against Malfunction, Data Loss.
“The DASD Time Bomb,” Datamation, vol. 32, Mar. 1, 1986, P63(4), Alper, Alan.
Symmetrix Remote Data Facility (SRDF) Product Guide, P/N 200-999-554 Rev A, EMC Corporation, Hopkinton, Mass., Sep. 1994, 105 pp. [EMCP 00003686-3791].
“Symmetrix Remote Data Facility Data Sheet, Symmetrix-resident Software Feature to extend mirroring capabilities,” EMC Corporation, Hopkinton, Mass., Apr. 1995, 4 pages [EMCP 00010741-1074].
“When Data is Lost in Minutes, its an Act of God. When Data is Recovered in Minutes, its an Act of EMC.” EMC Corporation, Hopkinton, Mass., 1994, one page [EMCP 00003903].
“EMC Ships Mainframe Disaster Recovery Solution that Restores Data in Minutes,” News from EMC Corporation, EMC Corporation, Hopkinton, Mass., Oct. 3, 1994, two pages [EMCP 00003921-3922].
“Asset Protection: Disk-Based Approach Makes Disaster Recovery More Efficient, Ensures Business Continuity,” EMC Corporate Information, EMC Corporation, Hopkinton, Mass., 5 pages [EMCP 00003916-3920].
“EMC Ships Continuously-Available Disk-Based Disaster Recovery Solution,” News from EMC Corporation, EMC Corporation, Hopkinton, Mass., Oct. 3, 1994, 4 pages [EMCP 00003923-8926].
“Symmetrix Remote Data Facility: Taking the disaster out of disaster recovery,” Brochure, EMC Corporation, Hopkinton, Mass., Sep. 1994, 8 pages [EMCP 00003908-3915].
“Symmetrix Remote Data Facility: EMC redefines disaster recovery,” Brochure, EMC Corporation, Hopkinton, Mass., Sep. 1994, 4 pages [EMCP 00003904-3907].
“EMC Introduces Symmetrix 5500 Continuous Operation Disk Storage for Mission Critical IBM Mainframe Market,” News from EMC Corporation, Hopkinton, Mass., Nov. 10, 1992, 3 pages.
“MCI Deploys New Fast Data Transfer Technology from EMC,” News from EMC Corporation, Hopkinton, Mass., May 22, 1995, 2 pages [EMCP 00007520-7521].
“EMC Agreement with Data Switch to Promote Long Distance Disaster Recovery,” News from EMC Corporation, EMC Corporation, Hopkinton, Mass., Nov. 30, 1994, 2 pages [EMCP 00007516-7517].
“EMC and Comdisco to Benchmark Data Mirroring Capability,” News from EMC Corporation, EMC Corporation, Hopkinton, Mass., Jan. 24, 1995, 2 pages [EMCP 00007518-7519].
“EMC Ships Mainframe Disaster Recovery Solution that Restores Data in Minutes,” New from EMC Corporation, EMC Corporation, Hopkinton, Mass., Oct. 3, 1994, 2 pages [EMCP 00007514-7515].
N. S. Prasad, “IBM Mainframes: Architecture and Design,” McGraw-Hill Book Company, New York, N.Y., 1989, pp. 58-73 and 182-196.
“The IBM 3390 Storage Control Reference,” No. GA32-0099-04, International Business Machines Corporation, Tucson, Arizona, 1991, pp. 1-304.
Patterson et al., “A Case for Redundant Arrays of Inexpensive Disks (RAID),” Report No. UCB/CSD 87/391, Computer Science Division (EECS), University of California, Berkeley, California, Dec. 1987, pp. 1-24.
Patterson et al., “Introduction to Redundant Arrays of Inexpensive Disks (RAID),” COMPCON 89 Proceedings, Feb. 27-Mar. 3, 1989, IEEE Computer Society, pp. 112-117.
Ousterhout et al., “Beating the I/O Bottleneck: A Case for Log-Structured File Systems,” Operating Systems Review, vol. 23, No. 1, ACM Press, Jan., 1989, pp. 11-28.
Douglis et al., “Log Structured File Systems,” Compcon 89 Proceedings, Feb. 27-Mar. 3, 1989, IEEE Computer Society, pp. 124-129.
Rosemblum et al., “The Design and Implementation of a Log-Structured File System,” ACM Transactions on Computer Systems, vol. 1, Feb. 1992, pp. 26-52.
M. Morioka et al., “Design And Evaluation Of The High Performance Multi-Processor Server,” IEEE Transactions, pp. 66-69, 1994.
A. Reinefeld, “The MOL Project: An Open, Extensible Metacomputer,” IEEE Transactions, pp. 17-31, 1997.
Robert D. Sloan, “A Practical Implementation Of The Data Base Machine--Teradata DBC/1012,” IEEE Transactions, pp. 320-327, 1992.
Nancy C. Ramsay, “Integration Of The Optical Storage Processor And The DBC/1012 Data Base Computer,” IEEE Transactions, pp. 94-97, 1990.
Jim Lyon, “Tandem's Remote Data Facility,” Tandem Computers, Inc., pp. 562-567, Jan. 1990.
Network backup -INPI Patent No. 93.12771, dated Oct. 21, 1993 “Dispositif Et Procede De Sauvegarde A Distance De Donnees Numeriques,” 15 pages.
“The Midgets, The Mammoth, And The Mainframes,” Business Week, Jun. 7, 1993, p. 31.
Complaint --Civil Action No. 00-40188-NMG; EMC Corporation v. StorageApps, Inc., In the United States District Court, District of Massachusetts, Central Section, Oct. 20, 2000.
First Amended Complaint --Civil Action No. 00-40188-NMG; EMC Corporation v. StorageApps, Inc., In the United States District Court, District of Massachusetts, Central Section, Nov. 9, 2000.
Answer, Counterclaim and Jury Demand of Defendant StorageApps, Inc. --Civil Action No. 00-41088-NMG; EMC Corporation v. StorageApps, Inc., In the United States District Court, District of Massachusetts, Central Section, Nov. 17, 2000.
Complaint of EMC Corporation Under Section 337 of the Tariff Act of 130, As Amended, United States International Trade Commission, In the Matter of Certain Data Storage Systems and Components Thereof, Investigation No. 337-TA-_, Apr. 11, 2002.
Complaint for Patent Infringement, In the United States District Court for the District of Massachusetts (Central Section), Civil Action No. BBO#546481, EMC Corporation v. Hitachi Data Systems Corporation and Hitachi, Ltd., Apr. 11, 2002.
Notice of Investigation, United States International Trade Commission, Inv. No. 337-TA-471, In the Mater of Certain Data Storage Systems and Components Thereof, May 9, 2002.
Notice of Investigation, United States International Trade Commission, Inv. No. 337-TA-471, In the Mater of Certain Data Storage Systems and Components Thereof, Federal Register, vol. 67, No. 93, Tuesday, May 14, 2002, pp. 34472-34473.
Fault Tolerance for LANs and Microcomputers, Atlantic Microsystems, Inc., Salem, NH, 1989 (7 pages).
IBM 3390 Storage Control Reference, Fifth Edition, Part No. GA32-0099-04, IBM Corporation, Sep. 1991.
Polyzois and Garcia Molina, “Evaluation of Remote Backup Algorithms for Transaction Processing Systems,” 1992 ACM SIGMOD, pp. 246-255.
Polyzois and Garcia Molina, “Evaluation of Remote Backup Algorithms for Transaction Processing Systems,” ACM Transactions on Database Systems, vol. 19, No. 3, Sep. 1994, pp. 423-449.
King and Halim, “Management of a Remote Backup Copy for Disaster Recovery,” ACM Transactions on Database Systems, vol. 16, No. 2, Jun. 1991, pp. 338-368.
Lyon, J., “Tandem's Remote Data Facility,” IEEE Comp 35th Computer Society Meeting, Spring 1990, pp. 562-567, IEEE Computer Society Press, Los Alamitos, CA..
Bhide et al., “An Efficient Scheme for Providing High Availability,” 1992 ACM SIGMOND, 1994, pp. 236-245 (also submitted is a four-page Portal.acm.org citation and abstract).
Gorelik et al., “Sybase Replication Server,” International Conference on Management of Data and Symposium on Principles of Database Management 1994 SIGMOD, p. 469 (also submitted is a two-page portal.acm.org abstract).
Daniels, et al., “Shared Logging Services for Fault-Tolerant Distributed Computing,” Proceedings of the Fourth Workshop on ACM SIGOPS European Workshop, 1990, pp. 1-4.
Ohtsuka, S. et al., “M2522/M2531 Optical Disk Library,” Fujitsu-Scientific and Technical Journal, vol. 31, No. 1, Jun. 1995, pp. 65-75.
Thanhardt, E., and Harano, G., “File Migration in the NCAR Mass Storage System,” Digest of Papers 9th IEEE Symposium on Mass Storage Systems (Editors, Friedman, K., and O'Lear, B.T.), 1988, Storage Systems: Perspectives, Nov. 1988, pp. 114-121 (also submitted is a one-page ieeexplore.ieee.org abstract).
Hou, R.Y., and Patt, Y.N., “Comparing Rebuild Algorithms for Mirrored and RAID5 Disk Arrays,” Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data and Symposium on Principles of Database Systems, 1993, pp. 317-326 (also submitted is a four-page portal.acm.org abstract).
Wilkes, J., et al., “The HP AutoRAID Hierarchical Storage Systems,” ACM Transactions on Computer Systems, vol. 14, No. 1, Feb. 1996, pp. 108-136.
Polyzois, C.A., and Garcia-Molina, H., “Processing of Read-Only Queries at a Remote Backup,” Proceedings of the 13th Symposium on Reliable Distributed Systems, Oct. 25-27, 1994, Dana Point, CA, IEEE Computer Society Press, Los Alamitos, CA, pp. 192-201.
Schatz, W., “When Data is not Enough,” Computerworld Client/Server Journal, Apr. 1, 1995, pp. 25+ (3 pages downloaded from eLibrary.com).
Remote Mirroring Technical White Paper, Copyright 1994-2002 Sun Microsystems, published at least as early as May 17, 2002 at sun.com, 25 pages.
Moad, J., “Will StorageTek's Iceberg Float?”, Datamation, Feb. 1, 1992, pp. 59-60.
Koop, P., “Replication at Work. (four companies use Oracle and Sybase replication servers to solve business problems),” DBMS, vol. 8, No. 3, p. 54(4), Mar. 1995.
IBM 3390 Storage Control Reference, No. GA32-009903, Fourth Edition (Dec. 1989), IBM Corporation.
Double Disk Volume Control Program WDCP/ES, First Edition, Oct., 1997, 6190-3-316-10(E), Hitachi Ltd., Japan.
P/DAS and Enhancements to the IBM 3990-6 and RAMAC Array Family, First Edition (May 1996), No. SG24-4724-00, IBM Corporation, San Jose, CA.
Oracle RDMS Database Administrator's Guide, vols. 1 and 2, Part No. 6693-70-0292 Feb. 1992, Oracle Corp., Redwood City, CA.
Hitachi, Ltd's and Hitachi Data Systems Corporation's Response to the Notice of Investigation and Complaint, Jul. 10, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
Respondent's Hitachi, Ltd. and Hitachi Data Systems Corporation's Unopposed Motion for a Modification to the Protective Order, Jul. 11, 2002, Investigation No. 337-TA-417, United States International Trade Commission, published at dockets.usitc.gov.
United States International Trade Commission, Order No. 5 Modifying Protective Order, Investigation No. 337-TA-471, published at dockets.usitc.gov.
Hitachi, Ltd. And Hitachi Data Systems Corporation, Submission of Representative Invalidity Charts, Jul. 26, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
Hitachi, Ltd's and Hitachi Data Systems Corporation's First Amended Response to the Notice of Investigation and Complaint, Aug. 20, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
Lupo, R.V., letter dated Aug. 21, 2002 and regarding typographical errors and enclosed replacement pages for Hitachi, Ltd's and Hitachi Data Systems Corporation's First Amended Response to the Notice of Investigation and Complaint, Aug. 20, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
Hitachi, Ltd. And Hitachi Data Systems Corporation, Citations of prior art and other references (5 pages), Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
United States International Trade Commission, Order No. 11 Granting Respondents' Motion to Amend Response and Ordering Submission by Private Parties, Sep. 12, 2002, Investigation No. 337-TA-471, public version published at dockets.usitc.gov.
Complainant EMC Corporation's Submission in Response to Specific Questions Raised in Order No. 12, Sep. 27, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
United States International Trade Commission, Order No. 20 Denying Hitachi's Motion No. 471-16 To Compel, Oct. 25, 2002, Investigation No. 337-TA-471, public version published at dockets.usitc.gov.
United States International Trade Commission, Order No. 9 Denying Complainant's Motion No. 471-3 To Modify the Protective Order (Order No. 2), Oct. 31, 2002, Investigation No. 337-TA-471, public version published at dockets.usitc.gov.
United States International Trade Commission, Commission Investigative Staff's Response to Order No. 24, Oct. 31, 2002, Investigation No. 337-TA-471, published at dockets.usitc.gov.
Complainant EMC Corporation's Submission of Responses to Specific Questions Raised in Order No. 24, Oct. 31, 20002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
United States International Trade Commission, Commission Investigative Staff's Response to Order No. 28, Nov. 8, 2002, Investigation No. 337-TA-471, published at dockets.usitc.gov.
Respondents Hitachi, Ltd and Hitachi Data Corporation's Responses to Order No. 28, Nov. 8, 2002, Investigation No. 337-TA-471, United States International Trade Commission, published at dockets.usitc.gov.
United States International Trade Commission, Notice of Commission Decision Not To Review and Initial Determination Granting Complainant EMC's Motion To Terminate the Investigation as to Certain Claims and to Amend the Complaint and Notice of Investigation by Adding Two Claims, Federal Register, vol. 87, No. 226, Nov. 22, 2002.
United States International Trade Commission, Order No. 32 Granting Respondents' Motion No. 471-25 To File A Second Amended Response To The Complaint and Notice of Investigation, Dec. 4, 2002, Investigation No. 337-TA-471, public version published at dockets.usitc.gov.
United States International Trade Commission, Order No. 44 Requiring Submission From Respondents, Jan. 17, 2003, Investigation No. 337-TA-471, published at dockets.usitc.gov.
United States International Trade Commission, Order No. 45 Requiring Submission From Respondents, Jan. 21, 2003, Investigation No. 337-TA-471, published at dockets.usitc.gov.
Continuations (1)
Number Date Country
Parent 08/654511 May 1996 US
Child 09/061708 US
Continuation in Parts (1)
Number Date Country
Parent 08/052039 Apr 1993 US
Child 08/654511 US