The reliability of a computer system is only as good as the underlying hardware of the system. Faults in the Random Access Memory (RAM) of a computer system, whether the faults are permanent or transient, often manifest themselves in the form of software instability and crashes. When applications or the Operating System (OS) of the computer system crash, the user may assume the cause is software related and therefore blame the software developer for the instability of their computer system. This not only hurts the reputation of the software developer in the marketplace, but it also requires the company to provide customer service to help users resolve problems that arise from RAM failures.
Aside from the harm to the software developer caused by faults in RAM, there is the possibility that a fault could result in corruption of the user's data or other undesired and unforeseen consequences.
To reduce memory errors, it is known to perform diagnostic tests on RAM at start-up. These tests are performed before the operating system is executing because they alter the contents of the memory, which would interfere with an executing operating system or other software components. For this purpose, some memory chips include circuitry to perform built-in self test (BIST) and can provide information identifying faulty pages in the memory to a memory manager in the operating system. Alternatively, some operating systems have incorporated memory tests such that the operating system itself can identify faulty pages in memory. The memory manager can then maintain stored information about faulty pages in the memory. When an application requests that memory to be allocated to it, pages that have been identified as faulty are not allocated.
It is also possible for memory tests to be implemented as application programs. These implementations generally don't have as much access to the memory and system resources as memory tests that are integrated into the OS—especially the kernel of the OS.
Some computer systems use memory that can correct errors through the use of error correcting coding (ECC). Each ECC has a strength that indicates a number of bit errors in a unit of data read from memory that can be corrected by the code. When more errors than can be corrected occur in a unit of memory, then the errors cannot be corrected. Though, the ECC may nonetheless reveal that an error occurred, such that additional faulty pages may be identified as the operating system is running.
Described herein are techniques for mitigating apparent software errors caused by faulty RAM in a computer system. Errors may be detected through a scan of regions of the memory or may be detected in response to a condition encountered by a software component executing on the computing system. In this way, memory errors may be detected while the OS is executing, without the use of memory equipped with ECC.
The OS may identify and test a region of RAM. If a region identified for testing has been allocated, the OS may move data out of that region such that the region may be subjected to a scan test. If the test determines that the region of RAM is faulty, then it may be added to stored information about faulty regions, which is maintained by the memory manager of the OS. When allocating memory regions to software components, the memory manager may exclude regions of memory that are indicated as faulty according to the stored information.
Dynamic identification of faulty RAM regions, in some embodiments, may be triggered by an event received from a software component, such as an exception or other indicator of an error. The trigger may indicate a specific region of RAM as potentially faulty or it may simply indicate that an error occurred generally. In response to an error of a type likely associated with a memory fault, the operating system may add an identified region to the stored information about faulty regions that are not allocated. If no specific RAM region is indicated by the trigger event, then the operating system may initiate testing of portions of RAM to dynamically identify faulty regions and add them to the stored information.
The foregoing is a non-limiting summary of the invention, which is defined by the attached claims.
The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures is represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. In the drawings:
The inventors have recognized and appreciated that mitigating memory errors in a computer system may increase reliability and stability. The system and method presented herein can mitigate these errors while the operating system (OS) is currently being executed by a processor of the computer system.
The technique also does not require special hardware, such as error-correcting code (ECC) memory. Embodiments of the invention may be implemented in software operating on general purpose hardware. For example, techniques as described herein may be implemented by modifying a portion of the operating system that is sometimes called a memory manager.
In some embodiments, the memory manager interfaces with the RAM of a computer system. The memory manager allocates RAM to software components that request memory resources. To ensure that a software component does not allocate RAM that is faulty, the OS may maintain stored information about RAM regions that have previously been found to cause errors.
In some embodiments, the stored information about faulty RAM regions is created by the memory manager of the OS. Multiple sources of information may be used to create the stored information. One source of information may be from testing. The memory manager may perform a diagnostic test of RAM regions to determine which regions are faulty. Once a region is found to be faulty, it may be added to the stored information about faulty RAM regions that is maintained by the memory manager.
In some embodiments, the memory manager can identify the RAM regions on which to run the diagnostic test. It may identify a particular RAM region to scan, a collection of RAM regions or the entirety of available RAM regions. Regions to test may be identified in any suitable way. In some embodiments, regions may be randomly selected for testing or may be scheduled in accordance with a predetermined pattern.
In some embodiments, the memory manager may receive a trigger event in response to which it performs the diagnostic test. The trigger event may be a user of the computer system manually initiating a scan. Alternatively, the trigger event could be from a software component that encountered an error.
Regardless of the source of the trigger, the memory manager may initiate a memory test to identify or confirm a faulty region. Though, in some scenarios, a software component indicating an error may be able to determine which RAM region caused the error. If so, then the this information may be included in the trigger event sent to the memory manager. The memory manager may selectively run the diagnostic test on the identified RAM region. Though, embodiments are possible in which a region is indicated as faulty, based on such an indication of an error, without any further testing.
Any suitable mechanism may be deployed as the diagnostic test. In some embodiments the diagnostic test comprises scanning the RAM regions to be tested and determining whether the regions are faulty. The scan may comprise writing a pattern of bits to the region being tested and, after the bits have been written, reading the pattern that is stored in the RAM region. The written pattern and the read pattern are then compared. If the pattern has changed, this indicates that there is an error being within the RAM.
The pattern of bits used to scan the RAM region may be any combination of bits, including patterns as are known in the art. For example, it could be all ones or all zeros. Alternatively, the pattern may alternate between one and zero. Also, a random pattern of zeros and ones may be used to scan the RAM region or a pattern in which a single bit of each word is set to a one, and the bit that is set to a one may change in each of multiple successive write cycles.
In some embodiments, the regions that are identified to be tested may be currently allocated to a software component. It is possible for the memory manager to scan this RAM region. The memory manager can swap the contents of the RAM region that needs to be tested with an available region of RAM. This may be accomplished by allocating a free RAM region to the software component, transferring the contents of the RAM region that needs to be tested to the newly allocated RAM region and releasing the region to be tested such that it is no longer allocated to the software component.
Techniques as are known in the art may be used for making this swap. For example, a memory manager may assign virtual addresses for use by software components. The memory manager, or other components of the computer system, may apply a mapping to translate these virtual addresses into physical addresses for RAM chips. To free a first region of physical memory for testing, the memory manager may copy the data from the first region to a second region of physical memory. The memory manager may then alter the mapping so that the virtual addresses used by a component point to the second region of physical memory.
The stored information about faulty RAM regions may be stored in several different ways. In one embodiment, the permanent information about faulty RAM regions may be stored somewhere accessible by the boot manager. When the computer boots, the stored information can be used by the boot manager while initializing the OS such that the faulty RAM indicated as faulty is never made available to the OS, making it appear to the OS that the faulty RAM regions do not even exist.
A different embodiment may store information about faulty RAM in volatile memory, such as the RAM itself. In this embodiment, the stored information will be deleted and reset every time the computer system powers off. Resetting the stored information may be advantageous for storing an identification of RAM regions in a system that may be subject to transient faults. If the memory manager cannot determine whether the RAM region is permanently faulty or cannot determine with a sufficiently high confidence that a RAM region is faulty at all, it may quarantine that RAM region by including that region among the stored information about faulty regions until a reboot occurs and resets the stored information.
Alternatively or additionally, the information about faulty RAM may be stored in a persistent memory structure such that the stored information is persistently maintained through a computer system reboot. This type of faulty RAM region information is useful for recording regions of RAM that have been determined by the memory manager to be permanently faulty and not worth using in the future.
Though, it should be appreciated that other techniques may be used to address the possibility of transient faults and indications of faulty regions with a low confidence that the RAM regions have faults. For example, the stored information about faulty regions may be retained in persistent storage, but may incorporate information indicating a frequency of a fault occurring in a particular region or a confidence level associated with a determination that a region is faulty.
As a specific example, if a software error occurs of the type that could be caused by a faulty memory region, that region may be added to stored information with an indication that such a fault was detected once or that such a fault has a low confidence value. Such an indication may not preclude the memory region from being allocated to another software component. However, if that same fault recurs, the confidence level of the fault may be increased, such that the region is not allocated again. Conversely, if the same fault does not recur over some period, the region may be removed from the stored information entirely.
Other techniques may be used in managing the stored information about faulty regions. Even regions indicated as permanently faulty may be retested periodically and possibly removed from the stored information if they retest without error or retest a sufficient number of times without error. As another example, if changes in a hardware configuration of a computer system are detected, such as may indicate a replacement of the memory chips, the stored information may be reset, regions associated with replaced RAM chips may be removed from the stored information or regions may be retested.
Any embodiment of stored information about faulty RAM may be edited by a user of the computer system. This will allow the user to manually add or remove RAM regions from the stored information. This can be important if the user changes or adds new hardware components to the computer system. For example, if the user removes an old RAM chip and installs a new one, the user should reset the stored information about faulty RAM regions, otherwise the computer system may unintentionally prevent non-faulty RAM regions of the new RAM chip from being allocated to software components.
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The computing environment may execute computer-executable instructions, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
With reference to
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110, although only a memory storage device 181 has been illustrated in
When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
The Operating System (OS) 210 is software that is generally used to control the hardware of the computer system 200. The OS ensures that all the pieces of the computer system work together in a coherent manner. The component of the OS that handles core functionalities of the hardware is referred to as the kernel 212. The OS 210 may be implemented using techniques as are known in the art.
The computer system 200 executes application programs 230, 232 and 234. Application programs interface with the hardware of the computer system 200 through the OS 210. Requests from application programs 230 to utilize hardware, such as RAM 132 or the processing unit 120, are all handled by the OS 210.
The boot manager 240 is another component of computer system 200. When the computer system is initially turned on, there is no OS in RAM 132 for the processing unit 120 to execute. The stored OS 144 must be loaded from non-volatile memory 141 to RAM 132 where the OS 134 can be executed by processing unit 120. This is the job of boot manager 240, which is preferably stored in ROM 131. The boot manager 240 may comprise BIOS 133. The boot manager 240 may be implemented using techniques as are known in the art.
OS 210 includes a memory manager 220, which may be part of the kernel 212, but is shown separately in example computer system 200 of
The memory manager 220 generally breaks the RAM 132 into multiple regions, which are allocated to applications. The specific meaning of a region may depend on the implementation of an operating system and possibly the underlying RAM hardware. Examples of regions are pages, segments or blocks. RAM is not allocated to applications 230 in single bit units. Rather, it is allocated in regions—meaning the smallest unit of memory that may be allocated by the memory manager 220 to an application 230 is a single region. The size of RAM regions varies depending on the specifics of the computer system 200, but an example size of a RAM region may be 4096 bytes.
Example computer system 200 shows the diagnostic test tool 222 and the faulty memory storage tool 224 as part of memory manager 220. It should be understood that the invention is not limited to this particular embodiment. The diagnostic test tool 222 and/or the faulty memory storage tool 224 may be included in some other part of the OS 210. Further, they may not be part of the OS at all, but rather an application program such as application 232, that is installed and executed on computer system 200 independent from the OS 210.
There are several advantages to embodiments where the memory manager 220 is part of the kernel 212, For example, the memory manager 220 may have access to more memory regions, such as regions in use by the kernel itself, allowing for a more comprehensive system. Also, by including the memory manager 220 in the kernel 212 the diagnostic test tool can be run transparently and noninvasively such that a user of the computer system 200 is unaware that a diagnostic test is being performed. Though this embodiment has advantages, the present invention is not limited to this particular embodiment.
As stated in the above, RAM 132 may become faulty. The computer system 200 would be more stable and reliable if faulty regions of RAM 132 were never allocated to software components. Embodiments of the present invention mitigate RAM errors using method 300 detailed in
The technique begins at act 302 by running the OS 210 on processing unit 120. The OS is initiated using the boot manager 240. Running the OS 210 means that the memory manager 220 will also be running and managing the RAM 132.
In some embodiments, the memory manager 220 may receive an indicator of an error from a software component at act 303. The software component may be application program 234 or a component of the OS 210. The error indicator is optional and is not a limitation of all embodiments of the present invention.
The error indicator is received by the memory manager 220 when an error has occurred in the software component. In some embodiments the error indicator is only sent when the error is related to memory failure. If the software component that encountered an error is able to ascertain the RAM region that caused the error then that information may be included in the error indicator sent to the memory manager 220.
At act 304, the memory manager 220 identifies a region of RAM to test. This identification of a RAM region may be triggered in many different ways. In one embodiment, a user of computer system 200 may manually initiate a scan of all RAM regions to test the RAM for faults. Alternatively or additionally, the memory manager 220 receives a trigger event from a software component. Trigger events from software components indicate that an error occurred in the software program while executing. Certain errors are indicative of a memory failure and it is possible, in some instances, for the OS 210 to identify the particular RAM region that caused the error. If the RAM region is identifiable, then the identity of the failed RAM region may be passed to the memory manager 220. At this point the memory manager can choose to test only the failed RAM region, a selected group of RAM regions or all RAM regions.
In a further embodiment, a trigger event may be received based on a set schedule. For example, the OS may arrange for full memory scans at scheduled intervals. Scans may also be scheduled at times when the computer system 200 is not actively being used.
Once at least one RAM region has been identified for testing, the memory manager 220 checks whether the RAM region is currently in use by a software component at act 306. If the RAM is free, then the memory manager 220 will proceed to perform a diagnostic test on the RAM region at act 308. If the RAM region is currently in use, then the memory manager will perform a series of actions (310, 312 and 314) to free the RAM region so that it may be tested. The details of these acts are described in detail below.
At step 308, the diagnostic test tool 222 performs the actual diagnostic test on the identified RAM region. The diagnostic test determines whether the RAM region is faulty. If it is not faulty, then decision block 318 returns to running the OS at act 302. If the RAM region is determined to be faulty, then the decision block 318 continues to act 320 where the faulty RAM region is added to the stored information about faulty RAM regions that is maintained by memory manager 220. The information about faulty RAM regions may be stored in faulty memory storage tool 224. This stored information may be represented in various forms. In one embodiment, this information may be structured as a list of faulty RAM regions. As another example, this information may be represented by using a flag associated with each RAM region, wherein a particular value of the flag indicates that the RAM region is faulty. Many techniques are known in the art for associating descriptive information with objects, and any of these techniques can be readily employed by the faulty memory storage tool 224.
Once act 320 is complete and future memory errors have thus been mitigated, the computer system 200 returns to running the OS at act 302. At this point the entire process may be repeated the next time the memory manager 220 identifies a region of RAM to be tested.
The above discussion described a sequence of events that may occur when it is determined at decision block 306 that the region of RAM to be tested was not in use by a software component. If it is determined that the region of RAM is in use by a software component at decision block 306, then the method continues to act 310. At act 310, RAM allocator 226 allocates a free region of RAM to the software component that is using the RAM region to be tested. The data that is stored in the RAM region to be tested is then transferred to the new free region of RAM at act 312. Once the data transfer is complete, the memory manager 220 releases the RAM region to be tested at act 314. Here, “release” may mean that the data stored in the RAM is no longer used by the software component. It may also mean that the region is no longer allocated to the software component that was previously using it.
Once the RAM region to be tested is no longer in use by any software component of the computer system 200, it continues to act 308. At this point, the method continues in the same manner as described above in the instance that the RAM region was not in use by a software component.
Any suitable mechanism may be used to perform the diagnostic test at act 308 of method 300.
The scan that is performed by the diagnostic test tool 222 at step 404 of the diagnostic test 308 may be performed in any number of ways that will suitably test the RAM regions for faults. An example embodiment of this scan 404 is shown in
Having thus described several aspects of at least one embodiment of this invention, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description and drawings are by way of example only.
The above-described embodiments of the present invention can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in an integrated circuit component. Though, a processor may be implemented using circuitry in any suitable format.
Further, it should be appreciated that a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. Additionally, a computer may be embedded in a device not generally regarded as a computer but with suitable processing capabilities, including a Personal Digital Assistant (PDA), a smart phone or any other suitable portable or fixed electronic device.
Also, a computer may have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound generating devices for audible presentation of output. Examples of input devices that can be used for a user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or in other audible format.
Such computers may be interconnected by one or more networks in any suitable form, including as a local area network or a wide area network, such as an enterprise network or the Internet. Such networks may be based on any suitable technology and may operate according to any suitable protocol and may include wireless networks, wired networks or fiber optic networks.
Also, the various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine.
In this respect, the invention may be embodied as a computer readable medium (or multiple computer readable media) (e.g., a computer memory, one or more floppy discs, compact discs (CD), optical discs, digital video disks (DVD), magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other non-transitory, tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention discussed above. The computer readable medium or media can be transportable, such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present invention as discussed above. As used herein, the term “non-transitory computer-readable storage medium” encompasses only a computer-readable medium that can be considered to be a manufacture (i.e., article of manufacture) or a machine.
The terms “program” or “software” are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects of the present invention as discussed above. Additionally, it should be appreciated that according to one aspect of this embodiment, one or more computer programs that when executed perform methods of the present invention need not reside on a single computer or processor, but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present invention.
Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.
Also, data structures may be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures may be shown to have fields that are related through location in the data structure. Such relationships may likewise be achieved by assigning storage for the fields with locations in a computer-readable medium that conveys relationship between the fields. However, any suitable mechanism may be used to establish a relationship between information in fields of a data structure, including through the use of pointers, tags or other mechanisms that establish relationship between data elements.
Various aspects of the present invention may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and is therefore not limited in its application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.
Also, the invention may be embodied as a method, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.
Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.
This application is a continuation of U.S. patent application Ser. No. 14/148,741, filed Jan. 7, 2014, entitled “SYSTEMATIC MITIGATION OF MEMORY ERRORS” (Atty. Dkt. No. 330885.02), which is a continuation of U.S. patent application Ser. No. 12/956,342, filed Nov. 30, 2010, now U.S. Pat. No. 8,627,176, entitled “SYSTEMATIC MITIGATION OF MEMORY ERRORS” (Atty. Dkt. No. 330885.01). The entirety of each of the afore-mentioned applications is incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 14148741 | Jan 2014 | US |
Child | 14698270 | US | |
Parent | 12956342 | Nov 2010 | US |
Child | 14148741 | US |