The present invention relates generally to the data processing field, and more particularly, relates to a method and controller for implementing enhanced flash backed dynamic random access memory (DRAM) management, and a design structure on which the subject controller circuit resides.
Storage adapters are used to connect a host computer system to peripheral storage I/O devices such as hard disk drives, solid state drives, tape drives, compact disk drives, and the like. Currently various high speed system interconnects are to connect the host computer system to the storage adapter and to connect the storage adapter to the storage I/O devices, such as, Peripheral Component Interconnect Express (PCIe), Serial Attach SCSI (SAS), Fibre Channel, and InfiniBand.
For many years, hard disk drives (HDDs) or spinning drives have been the dominant storage I/O device used for the persistent storage of computer data which requires online access. Recently, solid state drives (SSDs) have become more popular due to their superior performance. Specifically, SSDs are typically capable of performing more I/Os per seconds (IOPS) than HDDs, even if their maximum data rates are not always higher than HDDs.
Storage adapters often contain a write cache to enhance performance. The write cache is typically non-volatile and is used to mask a write penalty introduced by redundant array of inexpensive drives (RAID), such as RAID-5 and RAID-6. A write cache can also improve performance by coalescing multiple host operations (ops) placed in the write cache into a single destage op which is then processed by the RAID layer and disk devices.
Storage adapters also use non-volatile memory to store parity update footprints which track the parity stripes, or portions of the parity stripes, which potentially have the data and parity out of synchronization.
Data and parity are temporarily placed out of synchronization each time new data is written to a single disk in a RAID array. If the adapter fails and loses the parity update footprints then it is possible that data and parity could be left out of synchronization and the system could be corrupted if later the parity is used to recreate data for the system.
The non-volatile memory used for write cache data/directory and parity update footprints has typically taken the following forms:
1. Battery-backed DRAM memory (i.e. a rechargeable battery such as NiCd, NiMh, or Li Ion);
2. Battery-backed SRAM memory (i.e. a non-rechargeable battery such as a Lithium primary cell); and
3. Flash-backed SRAM memory (i.e. using a small capacitor to power the save of SRAM contents of SRAM to Flash, without external involvement).
Only the battery-backed DRAM memory provides for a sufficiently large memory, for example, GBs of DRAM, which is required by a write cache, thus requiring the complexity and maintenance issues of a rechargeable battery. Also, many robust storage adapter designs use a combination of non-volatile memories, such as the battery-backed DRAM memory or the Flash-backed SRAM memory, to provide for greater redundancy and design flexibility. For example, it is desirable for a robust storage adapter design to store parity updated footprints as well as other RAID configuration information in more than a single non-volatile memory.
A new flash-backed DRAM memory technology is available which is capable of replacing the battery-backed DRAM memory. This technology uses a super capacitor to provide enough energy to store the DRAM contents to flash memory when a power-off condition occurs.
However, the flash-backed DRAM memory technology must be managed differently than conventional battery-backed DRAM memory. The battery-backed DRAM memory could save the current contents of DRAM many times over in a short period of time. The DRAM memory could simply be placed into and removed from a self-refresh mode of operation to save the current contents of DRAM.
The flash-backed DRAM memory technology can only be saved when both super capacitors have been sufficiently recharged and the flash memory erased. Thus, prior art storage adapters are not effective for use with the flash-backed DRAM memory technology.
A need exists for a method and controller for implementing enhanced flash backed dynamic random access memory (DRAM) management. New methods and policies for management of this non-volatile memory are required. It is desirable to use a combination of flash-backed DRAM memory and flash-backed SRAM memory. Additional new methods and policies are required in order to be able to mirror data contents between these two different technologies.
Principal aspects of the present invention are to provide a method and controller for implementing enhanced flash backed dynamic random access memory (DRAM) management, and a design structure on which the subject controller circuit resides. Other important aspects of the present invention are to provide such method, controller, and design structure substantially without negative effects and that overcome many of the disadvantages of prior art arrangements.
In brief, a method and controller for implementing enhanced flash backed dynamic random access memory (DRAM) management in a data storage system, and a design structure on which the subject controller circuit resides are provided. The data storage system includes input/output adapter (IOA) including at least one super capacitor, a data store (DS) dynamic random access memory (DRAM), a flash memory, a non-volatile random access memory (NVRAM), and a flash backed DRAM controller. Responsive to an adapter reset, DRAM testing including restoring a DRAM image from Flash to DRAM and testing of DRAM is performed. Mirroring of RAID configuration data and RAID parity update footprints between the NVRAM and DRAM is performed. Save of DRAM contents to the flash memory is controllably enabled when the super capacitor has been sufficiently recharged and the flash memory erased.
In accordance with features of the invention, DRAM testing includes checking for a save or restore currently in progress. Responsive to identifying a save or restore currently in progress, a delay is provided to wait for change. When the DRAM has not previously initialized, checking if a saved flash backed DRAM image exists is performed. After restoring the saved flash backed image to the DRAM when available, and when the DRAM has been previously initialized, non-destructive DRAM testing is performed. After a normal power down of the adapter where no contents of the DRAM need to be saved and thus the save was disabled and a saved flash backed DRAM image does not exist, no restore is needed. Responsive to unsuccessful non-destructive DRAM testing, destructive DRAM testing is performed. The DRAM is tested and zeroed by destructive DRAM testing.
In accordance with features of the invention, mirroring of RAID configuration data and RAID parity update footprints between the NVRAM and DRAM includes merging flash-backed DRAM and flash-backed SRAM contents. The merging process maintains the latest RAID parity update footprints in NVRAM while also maintaining the write cache data/directory contents of the restored DRAM. Mirror synchronization of the DRAM and NVRAM is restored prior to allowing new data to be placed in the write cache.
In accordance with features of the invention, save of DRAM contents to the flash memory includes checking for an existing flash image, and releasing a saved flash image. Checking hardware state including the state of super capacitors is performed before enabling save of data from DRAM to the flash memory on power off.
The present invention together with the above and other objects and advantages may best be understood from the following detailed description of the preferred embodiments of the invention illustrated in the drawings, wherein:
In the following detailed description of embodiments of the invention, reference is made to the accompanying drawings, which illustrate example embodiments by which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In accordance with features of the invention, a method and controller implement enhanced flash backed dynamic random access memory (DRAM) management, and a design structure on which the subject controller circuit resides is provided.
Having reference now to the drawings, in
Storage adapter 100 includes a flash backed DRAM controller 118 for implementing enhanced flash backed dynamic random access memory (DRAM) management in accordance with the preferred embodiment. Semiconductor chip 102 includes a plurality of hardware engines 120, such as, a hardware direct memory access (HDMA) engine 120, an XOR or sum of products (SOP) engine 120, and a Serial Attach SCSI (SAS) engine 120. Semiconductor chip 102 includes a respective Peripheral Component Interconnect Express (PCIe) interface 128 with a PCIe high speed system interconnect between the controller semiconductor chip 102 and the processor complex 104, and a Serial Attach SCSI (SAS) controller 130 with a SAS high speed system interconnect between the controller semiconductor chip 102 and each of a plurality of storage devices 132, such as hard disk drives (HDDs) or spinning drives 132, and solid state drives (SSDs) 132. As shown host system 134 is connected to the controller 100, for example, with a PCIe high speed system interconnect.
Referring to
NVRAM contents 202 include redundant array of inexpensive drives (RAID) configuration data 206, and the flash backed DRAM contents 204 include corresponding RAID configuration data 208. As shown, the RAID configuration data 206 includes RAID device and redundancy group (RG) entries generally designated by the reference character 210 and are additionally stored in the storage devices 132. NVRAM contents 202 include RAID parity update footprints 212, and the flash backed DRAM contents 204 include corresponding RAID parity update footprints 214.
The flash backed DRAM contents 204 include a write cache directory 216 and write cache data 218. The DS DRAM is implemented, for example, with 8 GB of DRAM.
The RAID device and redundancy group (RG) entries 210, stored in RAID configuration data 206 and corresponding RAID configuration data 208, includes device entries generally designated by the reference character 230 and redundancy group entries generally designated by the reference character 240.
The device entries 230 include a flag indicating possible data in cache (PDC) flag 232 and an IOA/Dev correlation data (CD) 234, respectively stored in the storage devices 132. The RG entries 240 include a flag indicating possible parity update (PPU) flag 242 and an IOA/RG correlation data (CD) 244, respectively stored in the storage devices 132. When the PDC flag 232 is on, then there are potentially valid write cache contents in the write cache directory 216 and write cache data 218 on the respective device. Otherwise, if the PDC flag 232 is off, then there are no valid write cache contents for the device. When the PPU flag 242 is on, then there are potentially valid entries in the RAID parity update footprints 212, 214 for the respective redundancy group. Otherwise, if the PPU flag 242 is off, then there are no valid entries for the respective redundancy group.
Referring to
Referring to
In accordance with features of the invention, DRAM testing involves restoring a DRAM image from flash memory 114 to DRAM and testing of DRAM 110. This method addresses not only the cases where no image to restore exists and where a successful restore of an image is done, but also handles when a Save or Restore is already in progress, for example, having been started prior to the adapter being reset, and when the restore of an image is unsuccessful.
Referring to
An existing saved flash backed DRAM image is restored to the DRAM as indicated at a block 410. Checking whether the restore was successful is performed as indicated at a decision block 412. After successfully restoring the saved flash backed image to the DRAM when available, and when the DRAM has been previously initialized, non-destructive DRAM testing is performed as indicated at a block 414. Checking whether the non-destructive DRAM testing was successful is performed as indicated at a decision block 416. Responsive to unsuccessful non-destructive DRAM testing, destructive DRAM testing is performed as indicated at a block 418. The DRAM is tested and zeroed by destructive DRAM testing at block 418.
Checking whether the destructive DRAM testing was successful is performed as indicated at a decision block 420. Responsive to unsuccessful destructive DRAM testing, adapter failure is identified as indicated at a block 422. Responsive to successful non-destructive DRAM testing or successful destructive DRAM testing, indications as to the DRAM being restored or zeroed are saved as indicated at a block 424. The DS DRAM testing operations end as indicated at a block 428.
In accordance with features of the invention, mirroring of NVRAM 112 and DRAM 110 involves merging flash-backed DRAM contents 204 and flash-backed NVRAM or SRAM contents 202. This method addresses scenarios such as the following: Scenarios after a normal power down of the adapter 100 where no contents of the DRAM 110 need to be saved and thus the Save was disabled, that is where no Restore is needed and the DRAM can be tested and zeroed. Scenarios where an abnormal power down of the adapter 100 results in DRAM 110 being saved to flash memory 114, where upon reset of the adapter 100 the restored DRAM 110 has contents in synchronization with that of the NVRAM 112. Also scenarios similar to above, but where upon reset of the adapter the restored DRAM 110 has contents not in synchronization with that of the NVRAM 112 due to a second power down or reset of the adapter 100 prior to the adapter releasing the flash image, which could occur during the extended period where the adapter works to flush the write cache contents within the DRAM while creating many new RAID parity update footprints in the process.
In accordance with features of the invention, the merging process maintains the latest RAID parity update footprints 212 in NVRAM 112 while also maintaining the write cache data 218 and directory contents 216 of the restored DRAM 110. Mirror synchronization of the contents of DRAM 110 and NVRAM 112 is restored prior to allowing new data to be placed in the write cache.
Referring to
Otherwise when the correlation data within RAID configuration data does not match between NVRAM 112 and DRAM 110, and the DRAM 110 was not zeroed, then the RAID configuration data 208 and the RAID parity update footprints 214 are copied from DRAM 110 to NVRAM 112 as indicated at a block 508. Then a flag is set indicating that RAID parity update footprints 214 may be out of date as indicated at a block 510. Next as indicated at a block 512 write cache data may exist in DRAM 110 which has already been fully destaged to devices 132 and RAID parity update footprints 214 may exist in DRAM 110 which are out of date. As indicated at a block 514, mirroring of NVRAM 112 and DRAM 110 ends.
In accordance with features of the invention, enabling of a save of DRAM 110 to flash memory 114 involves releasing any existing DRAM image from flash memory, determining that the hardware is ready to perform another save, for example, that the super capacitor 118 is charged, and enabling a save to occur if a power down should occur. The order of this processing is critical to ensuring that a save of DRAM to flash is both possible and substantially guaranteed to occur should an abnormal power down occur.
Referring to
Design process 704 may include using a variety of inputs; for example, inputs from library elements 708 which may house a set of commonly used elements, circuits, and devices, including models, layouts, and symbolic representations, for a given manufacturing technology, such as different technology nodes, 32 nm, 45 nm, 90 nm, and the like, design specifications 710, characterization data 712, verification data 714, design rules 716, and test data files 718, which may include test patterns and other testing information. Design process 704 may further include, for example, standard circuit design processes such as timing analysis, verification, design rule checking, place and route operations, and the like. One of ordinary skill in the art of integrated circuit design can appreciate the extent of possible electronic design automation tools and applications used in design process 704 without deviating from the scope and spirit of the invention. The design structure of the invention is not limited to any specific design flow.
Design process 704 preferably translates an embodiment of the invention as shown in
While the present invention has been described with reference to the details of the embodiments of the invention shown in the drawing, these details are not intended to limit the scope of the invention as claimed in the appended claims.