Patent Application
20200073759
In computing systems, persistent and/or non-volatile memory resources are memory resources capable of retaining data even after system shutdowns, such as when power is removed due to unexpected power loss, system crash, or a normal shutdown. Non-volatile memory resources may utilize volatile memory components (e.g., DRAM) during normal operation, and transfer (“dump”) the contents of such volatile memory into backup memory, which may be non-volatile memory, in the event of a normal system shutdown, or even in the event of a power failure, such as by using a temporary backup power source.
The contents of volatile memory may be transferred to backup memory in increments of data blocks each consisting of a number of data lines. During the data block transfers, error correction mechanisms may be operable to identify errors in the data. In some cases, the backup memory will reject storage of a data block identified as containing an error; that is, an entire data block may be rejected, even if the error is determined to reside in only one data line within the data block. Subsequent data recovery from a shutdown may therefore be less complete.
For a detailed description of various examples, reference will now be made to the accompanying drawings, in which:
In this description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the examples disclosed herein. It will be apparent, however, to one skilled in the art that the disclosed example implementations may be practiced without these specific details. In other instances, structure and devices are shown in block diagram form in order to avoid obscuring the disclosed examples. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the inventive subject matter, resorting to the claims being necessary to determine such inventive subject matter. Reference in the specification to “one example” or to “an example” means that a particular feature, structure, or characteristic described in connection with the examples is included in at least one implementation.
The term “information technology” (IT) refers herein broadly to the field of computers of all types, computing systems, and computing resources, the software executed by computers, as well the mechanisms, physical and logical by which such technology may be deployed for users.
The terms “computing system” and “computing resource” are generally intended to refer to at least one electronic computing device that includes, but is not limited to including, a single computer, virtual machine, virtual container, host, server, laptop, and/or mobile device, or to a plurality of electronic computing devices working together to perform the function(s) described as being performed on or by the computing system. The term also may be used to refer to a number of such electronic computing devices in electronic communication with one another.
The term “cloud,” as in “cloud computing” or “cloud resource,” refers to a paradigm that enables ubiquitous access to shared pools of configurable computing resources and higher-level services that can be rapidly provisioned with minimal management effort; often cloud resources are accessed via the Internet. An advantage of cloud computing and cloud resources is that a group of networked computing resources providing services need not be individually addressed or managed by users; instead, an entire provider-managed combination or suite of hardware and software can be thought of as an amorphous “cloud.”
The term “non-transitory storage medium” refers to one or more non-transitory physical storage media that together store the contents described as being stored thereon. Examples may include non-volatile secondary storage, read-only memory (ROM), and/or random-access memory (RAM), Such media may be optical or, magnetic.
The terms “application” and “function” refer to one or more computing modules, programs, processes, workloads, threads and/or a set of computing instructions executed by a computing system. Example implementations of applications and functions include software modules, software objects, software instances and/or other types of executable code. Note, the use, of the term “application instance” when used in the context of cloud computing refers to an instance within the cloud infrastructure for executing applications (e.g., for a customer in that customer's isolated instance).
Referring to
Backup storage resource 104 may be implemented in various ways, including storage comprising NVDIMMs, solid-state disk drives, disk drive arrays (e.g., RAIDs), and so on, in any manner known to the art. The connection 106 shown in
In a conventional arrangement, processor 108 operates in part by executing system programming, i.e., code, stored in system read-only memory (system ROM) 114. That is, operation of processor 108 may be controlled by causing processor 108 to execute code stored in system ROM 114. Memory controller 116 coordinates memory storage and retrieval functions between processor 108, system memory 110, mass data storage 112, and backup storage resource 104. Memory controller 116 may include an error correction code (ECC) module 118 operable to detect data errors as memory contents are transferred and stored in various system components, and in particular to detect data errors in data transferred between system memory 110 and backup storage resource 104.
Mass data storage 112 may store, among other things, operating system 120 for controlling overall operation of computing resource 102, as well as drivers 122 for facilitating cooperation of computing resource 102 with external devices and processes (not shown), such as printers, modems, external storage devices, and so on. Mass data storage 112 may also store application software 124 to be executed by processor 108.
Data stored in system memory 110 may be byte-addressable by processor 108; that is, processor 108 can access any selected byte of data within system memory 110 by specifying a byte address to memory controller 116. Data transfers between system memory 110 and backup storage resource 104, on the other hand, are typically performed in increments consisting of data blocks, i.e., multiple bytes spanning a range of byte addresses. Each data block, in turn, may consist of multiple data lines, i.e., multiple bytes spanning a subset of the byte address range of the data block.
For example, for byte-addressable system memory of arbitrary size, a data block may consist of 128k bytes, with each 128 Kb data block consisting of 2000 64-byte data lines. In one example, data transfers between system memory 110 and backup storage 104 occur via a data transfer buffer 126, which may store, for example, one or more 128 Kb data blocks at a time. ECC module 118 may operate to detect data errors in data blocks as they are loaded into data buffer 126 to be transferred to backup storage resource 104. As noted, such errors can occur for various reasons, including permanently or transiently corrupted memory locations in system memory (e.g., DRAM) 110.
When ECC module 118 detects a data block in buffer 126 which contains a data error, the data block is “flagged,” i.e., identified as containing an error. A data block flagged as containing a data error will under most circumstances be rejected by backup storage resource 104 during the process of transferring some or all of the contents of system memory 110 to backup storage resource 104.
Although
In one implementation, a region (e.g., an address range) of system memory 110 may be designated, for example, through execution of operating system 120 or application software 124 by processor 108, as a scalable persistent memory region 128. Such a region is intended to function essentially as a virtual non-volatile dual in-line memory module (virtual NVDIMM), such that data stored in scalable persistent memory region 128 is preserved in case of system shut-downs, including either un-planned power interruptions or intentional shut-downs.
Conventional NVDIMMs, such as those of the NVDIMM-N variety, may include flash (i.e., non-volatile) storage and traditional DRAM on the same physical module, which is interfaced with the memory bus of a computer. A computing system accesses the traditional DRAM of the NVDIMM directly. In the event of a power failure, an NVDIMM module copies data from volatile traditional DRAM to persistent (e.g., flash) storage, and copies it back when power is restored. An NVDIMM may use a backup power source such as a battery or supercapacitor to facilitate data transfer from volatile to backup memory in the event of unplanned power failures.
Whereas conventional NVDIMMs consist of separate self-contained hardware modules interfaced to a computer system's memory bus, a “virtual” NVDIMM as described herein may be implemented by providing a scalable persistent memory region 128 in system memory 110, as herein described. Scalable persistent memory region 128 may be presented to the operating system 120 using an industry-standard defined NVDIMM interface, such that the operating system 120 can access it in the same manner it would a physical NVDIMM device.
To implement a scalable persistent memory region 128 in system memory 110 that is operable as a “virtual” NVDIMM, the contents of that region 128 are transferred to backup storage resource 104 at various times. As described above, such transfers typically occur either prior to planned power outages which would cause the contents of volatile memory to be lost, or unplanned power outages, in which the contents of volatile memory are lost unless backup power is provided to system memory at least temporarily.
In the example of
To address this undesirable outcome, according to one approach a provision is made for identifying a “known good” region 130 of system memory 110. In one implementation, such a known good region 130 may include an address range of one or more data blocks in size.
In one example, a known good region 130 is identified by processor 108, executing instructions stored in system ROM 114 causing processor 108 to perform a “scrubbing” or “scanning” operation on system memory 110. Under control of instructions stored in system ROM 114, processor 108 has byte-addressable access to system memory 110, and can perform such a scanning or scrubbing operation by reading each byte in an allocated data block range of memory addresses. If an ECC error occurs during this scanning/scrubbing process, the read operation will trigger failure from the processor 108, and that failure will be handled by memory error handler routines in system ROM 114.
Once an allocated memory space of a data block (or greater) in size is scanned or scrubbed and found to be without FCC errors, that memory space may be designated as known good region 130 to be used as a recovery transfer buffer as herein described.
During a backup operation in which the contents of scalable persistent memory region 128 are to be transferred to backup storage resource 104, a log of all failed (i.e., rejected) data block transfers may be maintained. Once all data blocks not found to have contained errors have been transferred, the data blocks from failed transfers may be separately scanned or scrubbed by processor 108, on a byte-accessible basis and under control of system ROM 114. For each data line read from a data block that does not result in an FCC error, the data is copied to known good region 130. When an ECC error in a data block is encountered during the CPU scan, the memory error interrupt handler code in system ROM 114 will prevent system crashing by clearing the error status and returning execution back to the CPU to continue the scan operation. Using output parameters from the system ROM 114 error interrupt handler, a scan operation can avoid copying the error to known good region 130, instead storing NULL data at the error location(s) in the buffer.
Once known good region 130 is full, its contents can then be sent to backup storage resource 104 for backup. All errors found during the read scan operation may be recorded in order, for example, to inform operating system 120 to avoid the memory address ranges associated with the detected errors. In this way, the amount of data lost during backup to backup storage resource 104 is advantageously reduced from an entire data block (e.g., 128 Kb) to only one data line (e.g., 64 bytes), or at most multiple data lines comprising less than an entire data block.
Advantageously, the size of scalable persistent memory region 128 can be of any arbitrary memory capacity, in contrast to conventional NVDIMMs, for example, wherein memory capacity is necessarily determined by the physical hardware provided. Particularly when it is considered that a computing resource such as computing resource 102 may in turn be implemented as a combination of physical hardware elements (processing, memory, and so on) and/or possibly combinations of such discrete hardware elements combined via networking, cloud computing, and other combinatorial methods to function as “virtual” hardware elements, a theoretically limitless degree of scalability may be achieved.
During normal operation of computing resource 102, such as under control of operating system 120, an ECC error can cause a system crash. However, if processor 108 performs the scanning operation corresponding to block 202 in the context of a system ROM boot and under control of system ROM 114, an encountered ECC error can be ignored and used as a signal to avoid using the offending memory location or range. The memory error interrupt handler of system ROM 114 can accomplish this by clearing the error status and returning execution back to the ROM boot process.
Once a region of system memory 110 of sufficient size (e.g., one 128 kB data block) is identified as being error-free, that region is tagged as known good region 130, as represented by block 204 in
During operation, software executed by processor 108 may cause a portion of system memory 110 to be designated as a scalable persistent memory region 128, i.e., essentially a virtual NVDIMM. This is represented by block 206 in
When during operation of computing resource 102, it becomes necessary to transfer the contents of scalable persistent memory region 128 to backup storage resource 104, such as for a system shut-down, the transfer operation commenced on a data-block by data-block basis, with successive data blocks being written to data transfer buffer 126 and then transferred to backup storage resource 104 via connection 106. This is represented by block 208 in
During these data block transfers of block 208, any data block determined by ECC module 118 to contain an error will be rejected by backup storage resource 104. The address range of any rejected data block will be recorded, as represented by block 210 in
Next, in block 212, processor 108 under control of system ROM 114, performs a scan operation on each data block rejected in block 210 for containing ECC errors. This scan operation, described in further detail with reference to
Once each line is scanned, and if necessary, scrubbed, it is written to known good region 130, as represented by block 212 in
Turning to
The scan operation involves reading increments, such as 64-byte data lines, of a larger memory unit in system memory, such as a 256 kB data block. This is represented by block 304 in
If a given read does not give rise to an ECC error, then at decision block 310 a determination is made whether sufficient memory increments have been read to create a known good region 130 of the desired size, e.g., one data block. If not, the process continues with a return to block 304 for a further memory increment read.
When sufficient memory has been scanned, at block 312 the memory range (which may or may not involve contiguous memory locations) is designed as known good region 130, and then CPU execution continues, at block 314.
The transfer operation begins at block 402 in
When a data unit is not rejected, next a determination is made whether all of the data units in the scalable persistent memory region 128 have been attempted, in decision block 410. If not, another data block is copied into data transfer buffer 126, in block 402.
Once attempts have been made to transfer the data blocks of the entire scalable persistent memory region 128 to backup storage resource 104, the transfer process continues as shown in
Processor 108 obtains an identification of a rejected memory unit (e.g., a data block) from the log maintained with reference to block 408 in
On the other hand if no ECC error is encountered at decision block 422, the non-offending data increment is written to known good region 130, in block 428. Next, in decision block 430, it is determined whether the entire rejected data unit has been scanned and, to only the extent necessary, scrubbed. If it has not, a next increment is read, at block 420, and the scan process is repeated. If the entire rejected data unit has been read and transferred to known good region 130, at block 432 the contents of known good region are transferred to backup storage resource 104. As noted above, this transfer is not likely to be rejected by backup storage resource 104, since the error causing increments of the previously rejected data units have been scrubbed and stored in known good region 130.
Next, a determination is made whether all rejected data units have been read and scrubbed, in decision block 434. If not, the identification of the next rejected data unit is obtained at block 418 and the scanning process on it is commenced. Once all rejected data units have been scanned and scrubbed, normal processing can resume, as shown at block 436.
Turning to
The recover process begins in block 504 with an identification of a memory range in system memory 110 to be designated as a scalable persistent memory region 128. This memory range (contiguous or otherwise), may be the same as prior to the system shutdown necessitating the backup and restore. On the other hand, because scalable persistent memory region is essentially a virtual resource, the allocation of memory to be used for the persistent memory storage region need not be fixed. As a consequence of this, it may be desirable to allocate the memory in system memory 110 for scalable persistent memory region 128 by taking into account any memory locations that have been previously flagged by ECC circuitry as exhibiting ECC errors. For example. Such information is obtained during a scanning operation such as described above with reference to
In block 506, the recovery process continues with the transfer of data units (e.g., data blocks) from backup storage resource 104 into the scalable persistent memory region 128 allocated in block 504. Thereafter, normal operation can continue, as represented by block 508. Advantageously, with the processing as described with reference to
As also shown in
Computing resource 900 may also include communications interfaces 925, such as a network communication unit that could include a wired communication component and/or a wireless communications component, which may be communicatively coupled to processor 905. The network communication unit may utilize any of a variety of proprietary or standardized network protocols, such as Ethernet, TCP/IP, to name a few of many protocols, to effect communications between devices. Network communication units may also comprise one or more transceiver(s) that utilize the Ethernet, power line communication (PLC), WiFi, cellular, and/or other communication methods.
As illustrated in
Persons of ordinary skill in the art are aware that software programs may be developed, encoded, and compiled in a variety of computing languages for a variety of software platforms and/or operating systems and subsequently loaded and executed by processor 905. In one implementation, the compiling process of the software program may transform program code written in a programming language to another computer language such that the processor 905 is able to execute the programming code. For example, the compiling process of the software program may generate an executable program that provides encoded instructions (e.g., machine code instructions) for processor 905 to accomplish specific, non-generic, particular computing functions.
After the compiling process, the encoded instructions may then be loaded as computer executable instructions or process steps to processor 905 from storage device 920, from memory 910, and/or embedded within processor 905 (e.g., via a cache or on-board ROM). Processor 905 may be configured to execute the stored instructions or process steps in order to perform instructions or process steps to transform the computing device into a non-generic, particular, specially programmed machine or apparatus. Stored data, e.g., data stored by a storage device 920, may be accessed by processor 905 during the execution of computer executable instructions or process steps to instruct one or more components within the computing resource 900.
A user interface (e.g., output devices 915 and input devices 930) can include a display, positional input device (such as a mouse, touchpad, touchscreen, or the like), keyboard, or other forms of user input and output devices. The user interface components may be communicatively coupled to processor 905. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD) or a cathode-ray tube (CRT) or light emitting diode (LED) display, such as an organic light emitting diode (OLED) display. Persons of ordinary skill in the art are aware that the computing resource 900 may comprise other components well known in the art, such as sensors, powers sources, and/or analog-to-digital converters, not explicitly shown in
Certain terms have been used throughout this description and claims to refer to particular system components. As one skilled in the art will appreciate, different parties may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In this disclosure and claims, the terms “including” and “comprising” are used in an open-ended fashion, and thus should be interpreted to mean “including, but not limited to . . . .” Also, the term “couple” or “couples” is intended to mean either an indirect or direct wired or wireless connection. Thus, if a first device couples to a second device, that connection may be through a direct connection or through an indirect connection via other devices and connections. The recitation “based on” is intended to mean “based at least in part on.” Therefore, if X is based on Y, X may be a function of Y and any number of other factors.
The above discussion is meant to be illustrative of the principles and various implementations of the present disclosure. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.
