Non-volatile storage devices, such as solid-state drives (SSD) and the like, may include a memory device for storing data and a controller for managing the internal operations of the storage device. The controller may process foreground operations including, for example, executing instructions such as data read and/or write instructions received from a host. The controller may also process background operations including, for example, executing internal operations to manage the resources on the storage device. In managing the resources of the storage device, the controller may execute relocation functions including compaction, read scrubbing, wear leveling, garbage collection, and the like, to move data from one location to another on the memory device, optimize how space on the memory device is used, and improve efficiency. The host may be unaware of the background operations being carried out on the storage device.
In hyperscale storage devices, the controller may perform frequent foreground operations and the host may expect a high quality of service from the storage device.
For example, the host may expect that the storage device responds to certain operations within a predetermined period. If the foreground operations are competing for the same resources on the storage device as the background operations, quality of service for the foreground operations may suffer. For example, the foreground and background operations may compete for the Random Access Memory (RAM), input/output bandwidth, processing time, and the like, on the storage device. This may also be true for a boot drive where frequent metadata updates may be needed for the entire platform where the storage device is being used.
To address this issue, the controller on some storage devices may interleave foreground operations with background operations. For example, within a five-second period, the controller may perform foreground operations for a certain period (for example, four seconds) and background operations for a certain period (for example, 1 second). In another example, the controller may allocate a percentage (for example, 80%) of the storage device bandwidth for host instructions and may allocate the remaining bandwidth for background operations. Interleaving foreground operations with background operations may lead to inefficient use of the storage device resources because the allocated time and/or bandwidth may be unused if it is not needed when allocated for a given operation, even if it is needed for other operations.
In some instances, the controller may perform command queue skipping and re-arrangement of foreground instructions to set and/or reset priority given to foreground operations. This may cause some host instructions to be carried out more quickly than others and may lead to inconsistency in the performance of the host.
There are instances where data may be stored in most of the space on the memory device and foreground instructions may not be carried out without relocating data on the memory device to free up space. To ensure that the controller can perform background operations in these instances, the controller may leave sufficient space on the memory device to perform garbage collection and other relocation functions when a certain percent of the memory device has data. The controller may also reduce the host access priority and increase the priority given to background operations, thereby degrading the quality of service expected by the host as the controller is trying to free up space on the memory device.
In some implementations, a storage device is communicatively coupled to a host that defines a quality of service level for responses transmitted from the storage device to the host. The storage device may include a memory device to store data. The storage device may also include a controller to perform background operations to manage resources on the memory device while performing foreground operations according to the quality of service level set by the host. The controller may generate a free block file including information on free blocks in the memory device and transmit the free block file to the host. The host may use the free block file to determine when the memory device is at or near a critical level of block availability and may transmit an indication to the controller. The controller adjusts the priority of the background operations in response to receipt of the indication to maintain the quality of service level.
In some implementations, a method is disclosed for maintaining the quality of service in a storage device including a memory device to store data and a controller.
The storage device is communicatively coupled to a host. The method includes performing background operations, by the controller, to manage resources on the memory device while performing foreground operations according to a predefined quality of service level set by the host. The method also includes generating, by the controller, a free block file including information on free blocks in the memory device and transmitting the free block file to the host. The method further includes receiving, by the controller, an indication from the host when the memory device is at or near a critical level of block availability and adjusting a priority of the background operations in response to receipt of the indication to maintain the predefined quality of service level.
In some implementations, the host includes a first memory to store instructions and data. The host also includes a processor to define the quality of service level for responses received from the storage device. The host may transmit foreground instructions to the storage device to access information in a second memory in the storage device. The host may receive a free block file including information on free blocks in the second memory and may use information in the free block file to determine that the second memory device is at the critical level of block availability at a given time. The host may adjust transmissions of the foreground instructions to the storage device to maintain the quality of service level for responses received from the storage device.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of implementations of the present disclosure.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing those specific details that are pertinent to understanding the implementations of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art.
The following detailed description of example implementations refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements.
Host 102 and storage device 104 may be in the same physical location as components on a single computing device or on different computing devices that are communicatively coupled. Storage device 104, in various embodiments, may be disposed in one or more different locations relative to the host 102. System 100 may include additional components (not shown in this figure for the sake of simplicity).
Storage device 104 may include a controller 108 and one or more non-volatile memory devices 110a-110n (referred to herein as memory device(s) 110). Storage device 104 may be, for example, a solid-state drive (SSD), and the like. Controller 108 may interface with host 102 and process foreground operations including instructions transmitted from host 102. For example, controller 108 may read data from and/or write to memory device 110 based on instructions received from host 102.
Memory device 110 may be flash based, including, for example, NAND flash memory. Memory device 110 may be included in storage device 104, as shown in
Cells in memory device 110 may degrade over time and data on memory device 110 may become obsolete. Therefore, controller 108 may also execute background operations to manage resources on memory device 110. For example, controller 108 may monitor memory device 110 and may execute garbage collection and other relocation functions per internal relocation algorithms to refresh and/or relocate the data on memory device 110. While controller 108 is executing the relocation functions, host 102 may continue to transmit instructions to controller 108 to fill up the free blocks on memory device 110.
Data traffic between host 102 and storage device 104 may be executed, for example, per the PCIe with NVMe standard using a doorbell type of command and acknowledgment communication protocol. According to this protocol, host 102 may store commands/instructions to be carried out by storage device 104 in a command queue and controller 108 may transmit an acknowledgment to host 102 upon receiving the command from the command queue and/or executing a command. Host 102 may use this protocol to write data to and/or read data from memory device 110, wherein host 102 may predefine the performance and quality of service level it expects from controller 108 when controller 108 executes instructions received from host 102.
Controller 108 may maintain metadata about storage device 104 including, for example, a free block file 112 with real-time information on the free blocks in memory device 110. Free block file 112 may include a free block count that may identify the percentage and/or amount of the remaining free space on memory device 110. Free block file 112 may include a background command queue count that may identify the size of data to be moved during pending relocation functions in storage device 104. In some instances, the background command queue count may also identify the percentage of the block space in memory device 110 that is needed to execute the pending relocation functions.
In some implementations, controller 108 may maintain free block file 112 as part of its Self-Monitoring, Analysis, and Reporting Technology (SMART) functions. Current SMART monitoring may detect and report various reliability indicators of storage device 104 with the intent of anticipating imminent hardware failures so that preventive action can be taken by host 102 to prevent data loss. In addition to transmitting indications of imminent hardware failure in a SMART file, controller 108 may transmit information in free block file 112 to host 102 for host 102 to determine how much free space is available on memory device 110 at any given time. The free space may be blocks or pages in memory device 110 with no data.
Using the free block file 112, host 102 may determine when memory device 110 is at or near a critical level of block availability. For example, host 102 may determine that memory device 110 is at or near a critical level of block availability when due to insufficient free space on memory device 110, controller 108 may be unable to execute foreground instructions at the predefined quality of service level expected by host 102. In another example, host 102 may use the information in free block file 112 to determine if there is sufficient space on memory device 110 such that controller 108 can execute foreground instructions prior to performing pending relocation functions. Host 102 may thus determine if the quality of service expected from storage device 104 may be degraded because of insufficient space on memory device 110.
In some implementations, host 102 may perform periodic status checks of the availability of blocks on memory device 110 by reading free block file 112. Consider an example where host 102 is transmitting a queue of write instructions to controller 108. Host 102 may monitor free block file 112 to determine the free blocks available on memory device 110 and to determine if controller 108 can execute all or some of the write instructions while meeting predefined quality of service requirements. If based on the information in free block file 112 host 102 determines that memory device 110 is at or near a critical level of block availability and that controller 108 cannot execute foreground operations at the predefined quality of service level, host 102 may adjust the transmissions of foreground instructions. This may enable host 102 to react gradually when free blocks on memory device 110 are limited, without sacrificing the quality of service host 102 expects from controller 108. For example, based on the free block count and the amount of space on memory device 110 needed for pending relocation, host 102 may determine that pending relocation functions may lead to degraded quality of service and host 102 may throttle its instructions to controller 108 and/or divert its instructions to another storage device in the host platform.
In some instances, when memory device 110 is at or close to the critical level of block availability, host 102 may instruct controller 108 to execute instructions in a faster operation mode, wherein the programming speed for data relocation write operations may be increased at a temporary expense of a tolerable Bit Error Rate (BER). This may ensure that quality of service is not degraded when memory device 110 is at or close to the critical level of block availability while controller 108 is executing foreground operations. As resources become available, host 102 and/or controller 108 may refine the data in memory device 110 by re-writing data at a normal program speed and normal BER. Host 102 may allow this increased programming speed for data relocation write operations when the temperature of storage device 104 is lower than a predefined level.
Storage device 104 may perform these processes based on a processor, for example, controller 108 executing software instructions stored by a non-transitory computer-readable medium. As used herein, the term “computer-readable medium” refers to a non-transitory memory device. Software instructions may be read into the memory device from another computer-readable medium or from another device. When executed, software instructions stored in the memory device may cause controller 108 to perform one or more processes described herein. Additionally, or alternatively, hardware circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.
The number and arrangement of components shown in
If, for example, during the second time host 102 has no pending write requests when host 102 determines that memory device 110 is at or close to the critical level of block availability that may affect quality of service, host 102 may transmit an instruction with a time-idle-for-relocation period to controller 108 prior to transmitting a write instruction. If, for example, host 102 determines that the pending relocation functions can be carried out in 100 milli-seconds, taking into account the size of the data to be relocated and the processing speed of controller 108, host 102 may provide a time-idle-for-relocation period of 100 milli-second to controller 108 and may pause other foreground instruction transmissions to controller 108 during the time-idle-for-relocation period. Host 102 may also instruct controller 108 to prioritize background operations and/or pause executing foreground instructions during the time-idle-for-relocation period. Controller 108 may use the time-idle-for-relocation period to complete the pending relocation functions.
If, for example, during the second time host 102 has pending write requests from the e-commerce platform when host 102 determines that memory device 110 is at or close to the critical level of block availability that may affect quality of service, rather than pausing foreground instructions to controller 108, host 102 may slow down the transmission of foreground instructions to storage device 104 by inserting extra idle time between foreground commands. Controller 108 may execute background instructions during the idle time inserted between the foreground commands.
Host 102 may also pause foreground instructions by reducing the response time during which storage device 104 may respond to a foreground command. For example, instead of predefining the quality of service wherein host 102 expects a response to a foreground command within one second, host 102 may instruct controller 108 to add, for example, a half second to the response time. With the adjusted response time, controller 108 may execute background instructions during the extra time allowed by host 102.
If in another example during the second time, host 102 determines that controller 108 may relocate more data than the data listed in the pending relocation command queue, during periods where host 102 may not need to access data on memory device 110, host 102 may instruct controller 108 to relocate more data than the data in the pending command queue. While host 102 is allowing controller 108 to perform relocation functions, host 102 may periodically examine free block file 112 to determine if the free block count and pending relocation workload are low enough to resume normal foreground instructions to storage device 104.
In an example where host 102 is unable to pause foreground instructions to storage device 104 during the second time, host 102 may divert the workload to another storage device in the host platform. Host 102 may record the quality of service impact statistics to determine which storage device 104 in its platform has impacted quality of service and/or which data is more difficult or easy to access on a storage device in its platform.
Host 102 may also instruct controller 108 to reconfigure certain frequently accessed data when memory device 110 is at or close to the critical level of block availability and thereby reduce future relocation of the reconfigured data. Consider an example where host 102 is accessing prices for items on the e-commerce platform. If host 102 frequently accesses the price of a first item and the price of the first item is infrequently updated, host 102 may issue multiple read instructions to access the same data. If host 102 determines, based on the quality of service impact statistics, that storage device 104 quality of service may be reduced, host 102 may instruct controller 108 to reconfigure where the price for the first item is stored on memory device 110 and reset the internal relocation parameters for relocating the price for the first item. For example, host 102 may instruct controller 108 to store the price for the first item in the SLC format while storing the prices for other items that are accessed less frequently and/or updated more frequently in the TLC or QLC format. This will enable controller 108 to execute read instructions for the price of the first item more quickly and maintain the quality of service. As indicated above
At 330, controller 108 may maintain metadata including free block file 112 with real-time information on the status of available space on memory device 110. At 340, controller 108 may transmit information in free block file 112 to host 102 for host 102 to determine how much free space is available on memory device 110 at any given time. At 350, host 102 may use the information in free block file 112 to determine when memory device 110 is at or near a critical level of block availability such that controller 108 may be unable to execute foreground instructions at the predefined quality of service level expected by host 102. At 360, if host 102 determines that storage device 104 is at or near the critical level of block availability and that controller 108 cannot execute foreground operations at the predefined quality of service level, host 102 may adjust the transmissions of foreground instructions. At 370, host 102 may throttle its instructions to controller 108 and/or divert its instructions to another storage device.
At 430, controller 108 may maintain metadata including free block file 112 with real-time information on the status of available space on memory device 110. At 440, controller 108 may transmit information in free block file 112 for host 102 to determine how much free space is available on memory device 110 at any given time. At 450, host 102 may use the information in free block file 112 to determine when memory device 110 is at or near a critical level of block availability such that controller 108 may be unable to execute foreground instructions at the predefined quality of service level expected by host 102.
At 460, if host 102 determines that storage device 104 is at or near the critical level of block availability and that controller 108 cannot execute foreground operations at a predefined quality of service level, host 102 may instruct controller 108 to execute background operations at an increased programming speed. At 470, as resources become available, controller 108 may refine the data in memory device 110 by re-writing data at a normal program speed and normal bit error ratio.
At 4130, controller 108 may maintain metadata including free block file 112 with real-time information on the status of available space on memory device 110. At 4140, controller 108 may transmit information in free block file 112 for host 102 to determine how much free space is available on memory device 110 at any given time. At 4150, host 102 may use the information in free block file 112 to determine when memory device 110 is at or near a critical level of block availability such that controller 108 may be unable to execute foreground instructions at the predefined quality of service level expected by host 102.
At 4160, if host 102 determines that storage device 104 is at or near the critical level of block availability and that controller 108 cannot execute foreground operations at a predefined quality of service level, host 102 may instruct controller 108 to reconfigure certain frequently accessed data and reduce relocation of the reconfigured data. At 4170, controller 108 may execute read instructions for the reconfigured data at a faster rate than for data that is not reconfigured.
Storage device 104 may include a controller 108 to manage the resources on storage device 104. Controller 108 may generate free block file 112 and host 102 may access free block file 112 to determine the space available on memory devices 110 associated with a storage device 104. Host 102 may also use information stored in free block file 112 to determine if controller 108 can execute foreground instructions without compromising quality of service because of the available space on memory device 110 associated with storage device 104. Host 102 may transfer foreground instructions from a first storage device, for example, storage device 104a, to a second storage device, for example, storage device 104c, if host 102 determines that based on the available space on memory devices 110 associated with storage device 104a, storage device 104a may not execute foreground instructions at a predefined quality of service level. Hosts 102 and storage devices 104 may communicate via a PCIe with NVMe interface, a SATA interface, a PATA interface, a SCSI interface, a SAS interface, a FireWire interface, a Fibre Channel connection interface, or the like.
Environment 500 may enable improvements in application performance and/or infrastructure efficiency through the integration of free block file 112 in directing instructions to storage devices 104. Environment 500 may also enable parallel computation and/or alleviate constraints on existing processing, memory, storage, and input/output.
Devices of environment 500 may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections. For example, the network of
The number and arrangement of devices and networks shown in
Input component 610 may include components that permit device 600 to receive information via user input (e.g., keypad, a keyboard, a mouse, a pointing device, a microphone, and/or a display screen), and/or components that permit device 600 to determine the location or other sensor information (e.g., an accelerometer, a gyroscope, an actuator, another type of positional or environmental sensor). Output component 615 may include components that provide output information from device 600 (e.g., a speaker, display screen, and/or the like). Input component 610 and output component 615 may also be coupled to be in communication with processor 620.
Processor 620 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or another type of processing component. In some implementations, processor 620 may include one or more processors capable of being programmed to perform a function. Processor 620 may be implemented in hardware, firmware, and/or a combination of hardware and software.
Storage component 625 may include one or more memory devices, read-only memory (ROM), and/or another type of dynamic or static storage device (e.g., a flash memory, a magnetic memory, and/or optical memory) that stores information and/or instructions for use by processor 620. A memory device may include memory space within a single physical storage device or memory space spread across multiple physical storage devices. Storage component 625 may also store information and/or software related to the operation and use of device 600. For example, storage component 625 may include a hard disk (e.g., a magnetic disk, an optical disk, and/or a magneto-optic disk), a solid-state drive (SSD), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cartridge, a magnetic tape, and/or another type of non-transitory computer-readable medium, along with a corresponding drive.
Communications component 605 may include a transceiver-like component that enables device 600 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communications component 605 may permit device 600 to receive information from another device and/or provide information to another device. For example, communications component 605 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, and/or a cellular network interface that may be configurable to communicate with network components, and other user equipment within its communication range. Communications component 605 may also include one or more broadband and/or narrowband transceivers and/or other similar types of wireless transceiver configurable to communicate via a wireless network for infrastructure communications. Communications component 605 may also include one or more local area network or personal area network transceivers, such as a Wi-Fi transceiver or a Bluetooth transceiver.
Device 600 may perform one or more processes described herein. For example, device 600 may perform these processes based on processor 620 executing software instructions stored by a non-transitory computer-readable medium, such as storage component 625. As used herein, the term “computer-readable medium” refers to a non-transitory memory device. Software instructions may be read into storage component 625 from another computer-readable medium or from another device via communications component 605. When executed, software instructions stored in storage component 625 may cause processor 620 to perform one or more processes described herein. Additionally, or alternatively, hardware circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.
The number and arrangement of components shown in
The foregoing disclosure provides illustrative and descriptive implementations but is not intended to be exhaustive or to limit the implementations to the precise form disclosed herein. One of ordinary skill in the art will appreciate that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings.
As used herein, the term “component” is intended to be broadly construed as hardware, firmware, and/or a combination of hardware and software. It will be apparent that systems and/or methods described herein may be implemented in different forms of hardware, firmware, and/or a combination of hardware and software.
Even though particular combinations of features are recited in the claims and/or disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and/or disclosed in the specification Although each dependent claim listed below may directly depend on only one claim, the disclosure of various implementations includes each dependent claim in combination with every other claim in the claim set.
No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related items, unrelated items, and/or the like), and may be used interchangeably with “one or more” The term “only one” or similar language is used where only one item is intended. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise
Moreover, in this document, relational terms such as first and second, top and bottom, and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, or “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting implementation, the term is defined to be within 10%, in another implementation within 5%, in another implementation within 1% and in another implementation within 0.5%. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way but may also be configured in ways that are not listed.
The present application claims the benefit of U.S. Provisional Application Ser. No. 63/451,338 titled “PROTOCOL FOR SOLID STATE DRIVE WITH HIGH QUALITY OF SERVICE,” filed Mar. 10, 2023, which is incorporated by reference herein in their entirety.
Number | Date | Country | |
---|---|---|---|
63451338 | Mar 2023 | US |