Patent Grant
10831403
The patent application is related to commonly owned and co-pending U.S. patent application Ser. No. 14/471,981 filed Aug. 28, 2014, the entire contents of which are hereby incorporated by reference.
In computing, queueing may be used to store commands for subsequent execution. For example, various methods may be employed for selecting and servicing an Input/Output (I/O) command from a queue. Some examples of these methods include first-in-first-out (FIFO), last-in-first-out (LIFO), shortest processing time, and shortest access time. In some instances, the selection and servicing of a command is non-uniform in that it favors some aspect of the command over another. That is, one command may be selected for servicing over another based on some sort of prioritization. This may cause unselected commands to linger in the queue, generally referred to as “starvation”. To prevent commands from starving, deadlines for executing the commands may be imposed. However, the disruption of executing a command that has met its deadline may reduce the overall performance of the system by causing a command timeout that could ultimately lead to a computing system undesirably servicing commands as FIFO.
Embodiments described herein provide for servicing commands. For example, systems and methods herein generally enforce maximum command completion times in a way that substantially reduces disruption to performance, while minimizing the value of the maximum command completion time limits (e.g., by tightening the distribution of command completion times). In one embodiment, a method is operable in a computing system and includes receiving first and second commands (e.g., I/O commands) to the computing system. The method also includes determining that the first command has a higher priority than the second I/O command, and queueing the second command for servicing at a later time. The method also includes servicing the first command, and servicing the second command after a timeout period based on a selectable performance degradation value that overrides a timeout of the second command and increases a probability of executing the second I/O command.
The various embodiments disclosed herein may be implemented in a variety of ways as a matter of design choice. For example, some embodiments herein are implemented in hardware whereas other embodiments may include processes that are operable to implement and/or operate the hardware. Other exemplary embodiments, including processor implementing instructions found in software and firmware, are described below.
Some embodiments are now described, by way of example only, and with reference to the accompanying drawings. The same reference number represents the same element or the same type of element on all drawings.
The figures and the following description illustrate specific exemplary embodiments. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the embodiments. Furthermore, any examples described herein are intended to aid in understanding the principles of the embodiments and are to be construed as being without limitation to such specifically recited examples and conditions. As a result, this disclosure is not limited to the specific embodiments or examples described below.
The computing system 10 may be a single device, such as a desktop/laptop computer, a server, a smart phone, a digital music player, or the like. Alternatively, the components of the computing system 10 may be configured separately. For example, the host system 115 may be implemented as a computer that interfaces with the storage system 110. In this regard, the storage system 110 may be a network storage system or a cloud storage system to which the host system 115 interfaces to store and retrieve data. In one embodiment, the controller 112 is a Redundant Array of Independent Disks (RAID) storage controller and the storage system 110 is thus a RAID storage system.
Whatever the configuration, the computing system 10 is operable to prioritize commands, allowing both higher and lower priority commands to be executed by enforcing maximum command completion times in a way that allows for minimal disruption in performance, while minimizing the value of the maximum command completion time limits (e.g., by tightening the distribution of command completion times). For example, the host system 115 may issue I/O commands to the storage system 110 for the storage and retrieval of data. And, the controller 112 processes the I/O commands to access the individual storage devices 100-1-100-N of the storage element 102.
The I/O commands themselves may have assigned priorities which designate how and when the controller 112 processes the I/O commands. In one basic configuration, the I/O commands may have first and second priorities assigned thereto, with the first priority being higher than the second priority. In this instance, the first priority I/O command would have a maximum command completion time that is less than the second lower priority I/O command. For example, both higher and lower priority I/O commands are often queued for servicing at a later time. The higher priority I/O command would thus have a lower maximum time for execution from its time of arrival at the storage system 110 than the lower priority command.
Although the term “priority” as used herein refers to a maximum command completion time that specifies a maximum allowable time limit for a command to be executed from its time of arrival, other prioritization schemes may be employed. Additionally, the embodiments are not intended to be limited to I/O commands as the computing system 10 may employ other types of commands and/or tasks.
In some previous prioritization command servicing methods, a single command completion period would often be specified for all workloads. For example, in processing I/O commands at an HDD (e.g., the storage device 100-1), the command completion period needed to be relatively large (e.g., 600 ms to 3 seconds) to prevent a cascade of I/O command timeouts as such would lead to a FIFO behavior for the HDD. However, the larger command completion periods were problematic when shorter response times were desired. And, the larger command completion values lead to poor performance on average for the workloads.
The embodiments herein calculate a maximum command timeout period by means of a user-defined/tunable amount of performance degrade. For example, the controller 112 may have an established command completion time period based on the current workload presented to storage element 102. Workload parameters may include, for example, queue depth, transfer size, I/O rate, etc. The controller 112 may then allow for a maximum performance loss (e.g., due to stale timeouts) and use that performance loss to establish a probabilistic stale timeout period which would generally assure that older queued I/Os are serviced before reaching their maximum command completion time limits (e.g., before a timeout occurs).
For the sake of simplicity, this embodiment describes commands having first and second priorities. And, time limits are established for servicing both priorities of commands. For example, the first/higher priority commands may have an established maximum execution time (i.e., the process element 252) that is shorter than the established maximum execution time of the second/lower priority commands (i.e., the process element 253).
A performance degradation limit may also be established, in the process element 254. Generally, this includes establishing a maximum allowable performance loss due to stale timeouts of the commands, and using this performance loss to establish a stale command limit. This part of the process is described in greater detail below. For example, I/O processing performance for the storage device 100-1 may be intentionally degraded by some amount to allow older queued I/Os to be serviced before reaching their maximum command completion time limits.
With the performance degradation limit established, the controller 112 can begin processing the commands, in the process element 255. Thus, in the process element 256, the controller 112 determines the priorities of the incoming commands. If an incoming command is a first priority command, the controller 112 may service the command before the second priority command, in the process element 257. If the command is a second priority command, the controller 112 may queue the second priority command in the memory device 108, in the process element 258, and then monitor the command's progress towards being serviced, in the process element 259. For example, the established time for the second priority command may cause the command to timeout if it is not serviced within that time. Thus, the controller 112 may monitor the timeout status of the queued second priority command. If no timeout has occurred, the controller 112 may monitor the queued second priority command until it is serviced, in the process element 257. Otherwise, if a timeout has occurred (or is about to occur), then the controller 112 may degrade the command processing performance, in the process element 260, based on the established performance degradation limit of the process element 254. This increases the probability that the command will be serviced, in the process element 257, while the performance of the computing system 10 is acceptably and temporarily degraded.
Generally, the controller 112 may queue the first priority command as well, albeit for a shorter period of time than the second priority command due to its shorter command completion time. However, in some instances, higher priority commands (e.g., commands of the first priority) can be serviced after lower priority commands (e.g., commands of the second priority). For example, when a lower priority command is approaching its command completion time, the lower priority command may be selected for execution even though a higher priority command also exists in the queue.
It should be noted, however, that the embodiments herein are not intended to be limited to any number of priorities for commands. For example, in one embodiment, the I/O commands to the storage system 110 may have a single priority and, thus, a single command completion time limit for executing the I/O command. In such a case, the embodiments herein allow queued I/O commands to execute before timing out by increasing their chances of being selected from the queue. To illustrate, the controller 112 may establish an execution time limit that defines a period of time in which all queued I/O commands to the storage system 110 are to execute. The controller 112, in this regard, may also establish a performance degradation value that temporarily decreases I/O processing performance of the storage system 110, overrides the execution time limit of an I/O command in the queue, and increases a probability of executing the queued I/O command as it approaches its command completion time limit. Thus, the embodiments herein showing two priorities are merely illustrative in nature and intend to aid the reader in understanding the embodiments disclosed herein.
Establishing the performance degradation limit will now be exemplarily discussed with respect to I/Os as these commands are common in terms of queueing. More specifically, the discussion will focus on I/Os to an HDD (e.g., storage device 100-1). For example, in an HDD embodiment, read and write I/Os may be sent by the host system 115 to the storage device 100-1, where they may be queued (e.g., in the memory device 108) while they await execution. The data associated with an I/O can have locational attributes, such as its rotational offset and track position (e.g., radius). In selecting the next I/O to execute after the currently executing I/O is completed, the storage device 100-1 may calculate an access time for the I/Os (or some portion of the I/Os) in the queue from the end of the currently executing I/O, and select the I/O with the shortest access time from the ending position of the current I/O.
Again, the performance degradation limit is established based on stale timeouts of the I/Os and is used to establish a stale I/O limit (e.g., a buffer period to execute I/Os whose timeout periods have expired). The first step in this process is to multiply the total I/O rate by a user defined/acceptable amount of performance loss expressed as a percentage (e.g., 2%). The total I/O rate is that which is presented to the storage element 102, the storage system 110, or a storage device 100 in terms of “wedges”.
Generally, a wedge refers to a fraction of a revolution of a magnetic disk. For example, the storage device 100-1 may have 500 wedges or “spokes” in a full rotation of a magnetic disk of the storage device 100-1 (e.g., a persistent storage medium). Accordingly, requiring 500 wedges to perform unqueued I/Os may indicate that, on average, the magnetic disk spins 500 wedges to complete the I/O operation, while the storage device 100-1 may only need to spin the magnetic disk 300 wedges to execute an average I/O when running at a queue depth of 128 I/Os.
Wedges may also be thought of as a measure of time, wherein an operation that takes 500 wedges to complete takes longer than an operation requiring 300 wedges. For example, if the storage device 100-1 is operating at a 128 I/O queue depth and performs an average of 300 wedges per I/O operation at 300 I/O operations per second, the storage device 100-1 may perform an average of 90,000 wedges per second (i.e., the number of wedges per I/O operation*number of I/O operations per second). Assuming that the storage device 100-1 has a maximum rotation speed, performing unqueued operations at a higher wedge count of 500 wedges per I/O would accordingly reduce the number of I/O operations that could be completed in a given time frame.
Returning to the discussion of establishing the performance degradation limit, the amount of performance loss is an upper bound and is calculated in terms of performance loss wedges (PLW). For example, the number of performance loss wedges is equal to the quantity of the average I/O rate in wedges (AIORIW) plus the average transfer time in wedges (AXTIW) times the performance loss target (PLT), as shown in Equation (Eq.) 1, as follows:
PLW=(AIORIW+AXTIW)PLT. Eq. 1.
The PLW result generally needs to be equal to the probability of timeouts (POT) multiplied by a “cost” of a timeout (“delta wedges”, or DW). These two equations are set equal to each other to solve for the unknown POT, as follows:
PLW=POT·DW; and Eq. 2.
POT=PLW/DW. Eq. 3.
Once the POT is found, the POT is used to find an index of entry into a table of probabilities of command aging. An example of such is shown and described below as well as in commonly owned and co-pending U.S. patent application Ser. No. 14/471,981. Then, that probability is inverted to yield a number of “trips remaining” (TR) in a queue (e.g., the stale I/O command limit in trips, “trips” being generally referred to as I/Os in the queue). The queue depth is corrected by dividing the TR by a queue depth scale (QDS). This value is then converted to milliseconds such that the stale I/O command limit can be computed, as follows:
where the I/O rate is the number of I/Os per second that the storage system 110, the storage element 102, or the individual storage devices 101 are operable to process.
To summarize, the controller 112, in one embodiment, would generally assure that all “high” priority I/Os are forced into execution once they reach a 70 ms limit. “Low” priority I/Os would be forced into execution once they exceed 1000 ms. With the new stale I/O command limit, the controller 112 calculates a probabilistic “soft limit” for low priority I/Os to get executed in the event of a timeout or near timeout. The soft limit generally means that I/Os are not automatically activated even if the timeout period is breached. This cuts down on performance loss due to low priority timeouts. The controller 112, in setting the low priority soft limit, reduces the 99% command completion times for low priority I/Os, without impacting the overall I/O rate. The table below exemplarily illustrates how low priority limits change with queue depth and the selected allowable percentage of performance degradation.
A probabilistic command aging algorithm may track an age for each command in the queue 300. For example, the queue 300 may store the actual age value, a number of clock ticks since receipt, a timestamp of when the command was received from which the age can be calculated as needed, or the age may be monitored by other methods. In the queue 300, the age is shown for each command in terms of seconds since the command was received. Although the commands in the queue 300 are shown as tightly packed and ordered, they may be sparsely arranged or, more generally, may be organized in any logical fashion suitable for the circuitry of the storage device 100-1.
Commands may have a designated timeout period (e.g., 1.5 seconds, although longer or shorter timeout periods may be used). If a command reaches the timeout period without being executed, the command may expire, or it may be executed immediately regardless of its position in the queue 300. Assuming commands are queued such that commands with a lower command number are executed earliest, the queue 300 depicts an example in which the last command 9 in the queue 300 may be approaching the 1.5 second timeout period with an age of 1.35 seconds.
The controller 112 may calculate a probability of a command timing out, and factor that probability into an algorithm for organizing pending operations. For example, a probability of a command timing out may be weighed against a performance loss (e.g., based on an I/O rate of executing queued vs. unqueued commands) to determine when to execute the command.
The controller 112 may also calculate an effect on performance, using a number of calculations and approximations. In some embodiments, performance calculations or estimates may be performed externally to the computing system 10 by a manufacturer or other party. For example, some values and estimates may be pre-loaded on a device, or updated in device firmware with a software update. This may reduce processing of the controller 112 and improve performance.
An exemplary process of calculating a probability of timing out is now provided. With an example queue depth of 64 commands, the odds of a command not being picked for activation is approximately 63 out of 64, since one command may be activated at a time. The odds of this happening two times in a row is generally (63/64)2. The odds of this happening “X” times in a row are generally (63/64)x. The more general form of this equation may be represented as:
Probability of not being picked for activation=((queue depth−1)/(queue depth))x, Eq. 5.
where “X” is the number of times the command is not picked for activation.
By knowing the I/O rate for a given workload and the amount of time remaining before a command will timeout, it is possible to calculate the probability that a command will timeout in the future. For example, with a queue depth of 64, suppose the storage device 100-1 can perform at an approximate I/O rate of 300 I/Os per second. An approximate I/O rate for a given queue depth for the storage device 100-1 can be determined during a manufacturing process, or may be determined based on performance measurements. For example, the approximate I/O rate for a selected number of queue depths may be stored (e.g., in the memory device 108) for the storage device 100-1, and the controller 112 may consult a table for a nearest I/O rate approximation based on the current queue depth. In some embodiments, an I/O rate approximation for a single selected queue depth may be used to simplify the probability calculation. In some embodiments, a larger queue depth means that more I/Os per second can be performed by a given drive, but higher average command completion times may occur and, thus the possibility of timeouts may increase, as command completion times are generally proportional to the queue depth.
If a command in the 64 depth queue has one second remaining before it times out, and the storage device 100-1 performs at approximately 300 I/O operations per second at 64 queue depth, this can leave approximately 300 opportunities for activation of the command: (I/Os per second*(time remaining until timeout)). The number of opportunities remaining can be inserted into Eq. 5 as the value of “X” to determine a probability of the command timing out. Then, using Eq. 5, the probability of this command timing out may be:
POT=(63/64)300=0.0089. Eq. 6.
The controller 112 may calculate an access time for a selected command to determine approximately how long it may take for the command to begin execution after the end of the current command being executed completes. This may be considered the “base” access time. For example, access time to the storage device 100-1 may be approximately calculated as a seek time (e.g., the time it takes to move a transducer head from a starting position to a target data track) plus additional rotational latency (e.g., the time it takes for the disk to rotate so that the head is positioned over the target data sectors on the track). The access time may be calculated from a given starting disk and head position, for example, based on a calculated head and disk location after executing a previous operation.
After the base access time has been calculated for a selected command, an adjustment to this value may be added based on the command's probability of timing out. Now, considering that I/O rates (or operation times) may also be expressed in terms of wedges, assume that the storage device 100-1 can perform at an average rate of 500 wedges per I/O when performing unqueued I/Os, and at an average rate of 300 wedges per I/O when operating at a queue depth of 128. Both I/O rates may have a distribution about a mean that is normal. Also assume that for a selected command, the probability of it timing out (i.e., the POT) is 50%, based on its age and the I/O rate of this workload. Then the future expected access time (FEAT) for the selected command may be calculated as follows:
FEAT=Q128I/O rate(IOR)+POT*(Q1IOR−Q128IOR). Eq. 7.
Or, more specifically, FEAT=300+0.5*(500−300)=400 wedges to execute the selected command.
In some embodiments, the FEAT of Eq. 7 may be the weighted average of the queue 128 value (Q128) and the queue 1 value (Q1), and it may converge on the Q1 value when the probability of timeout reaches 100%. The FEAT value can be used to represent the average expected access time this command will have if left in the queue 300, instead of being activated as the next command, or being repositioned to a selected place in the queue 300.
Now, suppose an initial or “base” access time for a command in a queue of depth 128 is calculated to be 200 wedges. For example, this initial or base access time, as described above, may be calculated as seek time+rotational latency. The base access time may be calculated for a current head and disk position, or for an anticipated head and disk position after executing another given command. The base access time of 200 wedges may be a very fast access time relative to the Q128 distribution, and may be well above the FEAT calculated above. Relative benefit or detriment of activating the selected command may be calculated, for example, as:
FEAT−base access time. Eq. 8
In this case, the relative benefit or detriment may be calculated as 400−200=200. So in general, activating a command “A” at the calculated position would represent a relative benefit of 200 wedges compared to leaving the command at its current queue position. For example, in systems that calculate which command to execute next from all commands in the queue 300, the controller 112 may choose to execute command “A” next rather than leaving it in the queue 300 for execution later, depending on calculated values of other queued commands. In some embodiments, a device may compare the FEAT to the base access time to determine whether to activate the command now or to place it in a corresponding command queue slot. If the comparison is positive, it may indicate that the command should be placed into the slot scheduled for activation. If the comparison is negative, it may indicate that the command should not be relocated to the slot scheduled for activation.
Once again, the FEAT, as calculated in the above example, is 400 wedges. An initial access time for a selected command “B”, in a queue of 128, may be calculated to be 450 wedges. 450 wedges may be a relatively slow access time compared to the Q128 distribution, and is slower than the FEAT of 400 calculated above. In general, activating command “B” now would represent a relative negative over re-queuing of −50 wedges. For example, FEAT−base access time, in this case, is 400−450=−50 wedges. From this it can be seen that command timeouts may complicate the reordering process. So, the probability of a command timeout may be incorporated into access time calculations, as exemplarily described below.
AverageAccessTimeForWorkload(ATWL)=Estimated; Eq. 9.
Q1 Access Time(Q1AT)=Estimated; and Eq. 10.
IOR=Estimated. Eq. 11.
ATWL generally refers to an average I/O rate at a given queue depth in terms of wedges per I/O operation. For example, this may be a representation of an average number of wedges per I/O operation when operating at a queue depth of 128. In some embodiments, the memory device 108 may store, for the storage device 100-1, a number of ATWL values for different workload sizes, it may store a single representative value (e.g., average workload), it may store a “worst case” value for a full queue workload, it may store other values, or any combination thereof. Q1AT generally refers to an average I/O rate in terms of wedges per operation for unqueued operations. And, IOR generally refers to an average number of operations per second at a given queue depth. Again, a number of I/O rates, one or more representative values, or any combination thereof may be stored in the memory device 108.
For example, ATWL, Q1AT, and IOR may be determined by a manufacturer of the storage device 100-1 based on the device's components and performance attributes, and stored to the device (or memory 108) for accessing. In some embodiments, these values may be determined or measured during operation of the storage device 100-1.
The following values may be calculated for each command:
Access Time(AT)=Seek Time+Rotational Latency; Eq. 12.
Time Remaining Before Timeout(TIME)=Timeout Limit−Age of Command; Eq. 13.
Opportunities Remaining In Queue(X)=TIME*IOR; Eq. 14.
POT=(Queue Depth−1)/(Queue Depth)x; Eq. 15.
FEAT=ATWL+PROB*(Q1AT−ATWL); and Eq. 16.
Timeout Adjusted AT(TAAT)=AT−(FEAT−AT). Eq. 17.
The TAAT for a command, therefore, may be the command's AT (or “base” access time unmodified by a timeout probability), modified by the comparison of the probability of timeout—the modified FEAT and the AT. In some embodiments, the controller 112 may compare the TAAT of commands within the queue 300 to determine an order to execute the commands. For example, the TAAT for each command may be calculated, and the command with the lowest TAAT may be selected for execution next. TAAT may be simplified, as follows:
TAAT=2AT−ATWL−PROB*(Q1AT−ATWL). Eq.18.
In some embodiments, ATWL may be a predetermined value (e.g., calculated by a manufacturer), and therefore may be considered a constant for any given workload. Adding a constant to all commands in the queue 300 may not change their order, so it may be removed from the TAAT calculation. Also, since dividing all access times by two may not change a command order, TAAT may be further simplified as:
TAAT=AT−PROB*(Q1AT−ATWL)/2. Eq. 19.
In some embodiments, a lookup table may be used in place of performing some calculations. For example, using a lookup table may avoid the need to perform exponent calculations when computing the PROB value. In some embodiments, rather than having a lookup table for all possible queue depths, a number of lookup tables for selected queue depths could be used (e.g., 16, 32, 64, etc.), and a closest approximate queue-depth table could be used for calculations. In some embodiments, lookup tables for one or more queue depths could be stored, and index scaling could be employed to account for variations in the actual queue depth.
An example of employing index scaling on a lookup table may include:
For example, a lookup table for a selected Qbase may be populated with values based on the equation:
Probability(Qbase)=((Qbase−1)/Qbase)x, Eq. 20.
for some input value ‘X,’ with X indicating a number of I/O opportunities left before a command times out.
Similarly, a lookup table for an actual queue depth ‘Q’ may be populated with values based on the equation:
Probability(Q)=((Q−1)/Q)x. Eq.21.
It can be assumed that the Probability (Q) differs from the Probability (Qbase) by a scaling constant ‘C’ on the exponent ‘X’, as follows:
Probability(Q)=((Q−1)/Q)x=((Qbase−1)/Qbase)cx. Eq.22.
By taking the logarithm of both sides, and solving for ‘C,’ it can be seen that:
C=log((Q−1)/Q)/log((Qbase−1)/Qbase)). Eq. 23.
The equation may be further simplified by using the linear approximation of: “Log(R)˜R−1” for values of R close to 1. This can be a fairly good assumption given that ratio of (Q−1)/Q may be very close to 1 for the probability calculations. This approximation can lead to a simplified equation for ‘C’, as follows:
C=Qbase/Q. Eq.24.
In some examples, using the probability table architecture derived above, the new process steps may include calculating, for each command:
AT=Seek Time+Rotational Latency; Eq. 25.
TIME=Timeout Limit−Age of Command; Eq. 26.
Opportunities Remaining In Queue(X)=TIME*IOR; Eq. 27.
C*X=X*Qbase/Q; Eq. 28.
PROB=Probability term of Table[C*X]; and Eq. 29.
TAAT=AT−PROB*(Q1AT−ATWL)/2, Eq. 30.
assuming the same estimated or calculated values for ATWL, Q1AT, and IOR as described above.
The algorithms and computations provided above are just examples. Different equations, estimates, variables, and values can be used to calculate a probability of a command timing out and adjust command executions based on the probability. For example, the TAAT equation provided above may be simplified further by approximating variables. For example, rather than employing multiple probability tables, or using a scaling constant ‘C,’ a single probability lookup table could be implemented using a “worst-case” full queue depth and disregarding the actual current queue depth. In some embodiments, ATWL, Q1AT, or both could be constants stored for the storage device 101-1 in the memory device 108.
Using Eq. 30 above as an example, if the values for Q1AT and ATWL were constants, and a single probability lookup table was employed, the probability lookup table could be populated with values taking into account the Q1AT and ATWL values. The calculations for “PROB*(Q1AT−ATWL)/2” could then be performed prior to command execution and stored in the lookup table by a manufacturer of the storage device 101-1. For example, the lookup table could store values representing an access time adjustment as a factor of a number of execution opportunities remaining. When simplified as described, the calculations performed for a command to derive an access time modified based on a probability of timeout may then be reduced to:
AT=SeekTime+Rotational Latency; Eq. 31.
TIME=Timeout Limit−Age of Command; Eq. 32.
Opportunities Remaining In Queue(X)=TIME*IOR; Eq. 33.
PROB=Probability term of Table[X]; and Eq. 34.
TAAT=AT−Probability term of Table[X]. Eq. 35.
Using estimated constants and simplified probabilistic aging algorithms may provide much of the performance benefit while reducing a run-time computational overhead.
Calculating TAAT is based on a statistical algorithm, so scenarios may arise where an I/O operation is selected for a queue position even with a poorly calculated AT. The storage device 100-1 may be configured to perform additional calculations or logic checks to limit such instances. For example, the storage device 100-1 may be configured to exclude reordering of an operation if the calculated AT is greater than the Q1AT, meaning that the command will take longer to execute than an average command executed at a queue depth of 1. In some embodiments, a command may require an AT that is better than the Q1AT by some selected threshold. Other embodiments are also possible.
The HDD 100 can be implemented with a desktop computer, a laptop computer, a server, a personal digital assistant (PDA), a telephone, a music player, or any other device requiring the storage and retrieval of digital data. Additionally, certain components within the HDD 100 may be implemented as hardware, software, firmware, or various combinations thereof.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4224667 | Lewis | Sep 1980 | A |
| 4237533 | Mills | Dec 1980 | A |
| 4271468 | Christensen et al. | Jun 1981 | A |
| 4423480 | Bauer et al. | Dec 1983 | A |
| 4458316 | Fry et al. | Jul 1984 | A |
| 5185737 | Nassehi et al. | Feb 1993 | A |
| 5265252 | Rawson et al. | Nov 1993 | A |
| 5339405 | Elko et al. | Aug 1994 | A |
| 5392397 | Elko et al. | Feb 1995 | A |
| 5418971 | Carlson | May 1995 | A |
| 5436892 | Tago et al. | Jul 1995 | A |
| 5457793 | Elko et al. | Oct 1995 | A |
| 5459839 | Swarts et al. | Oct 1995 | A |
| 5509134 | Fandrich et al. | Apr 1996 | A |
| 5530948 | Islam | Jun 1996 | A |
| 5544304 | Carlson et al. | Aug 1996 | A |
| 5559988 | Durante et al. | Sep 1996 | A |
| 5646918 | Dimitri et al. | Jul 1997 | A |
| 5664143 | Olbrich | Sep 1997 | A |
| 5692138 | Fandrich et al. | Nov 1997 | A |
| 5802343 | Fandrich et al. | Sep 1998 | A |
| 5809541 | Fandrich et al. | Sep 1998 | A |
| 5875290 | Bartfai et al. | Feb 1999 | A |
| 5887191 | Adiga et al. | Mar 1999 | A |
| 5930252 | Aaker et al. | Jul 1999 | A |
| 5940866 | Chisholm et al. | Aug 1999 | A |
| 5946466 | Adiga et al. | Aug 1999 | A |
| 5956742 | Fandrich et al. | Sep 1999 | A |
| 5983292 | Nordstrom et al. | Nov 1999 | A |
| 5990913 | Harriman et al. | Nov 1999 | A |
| 5991825 | Ng | Nov 1999 | A |
| 6012150 | Bartfai et al. | Jan 2000 | A |
| 6047334 | Langendorf et al. | Apr 2000 | A |
| 6065088 | Bronson et al. | May 2000 | A |
| 6092158 | Harriman et al. | Jul 2000 | A |
| 6092215 | Hodges et al. | Jul 2000 | A |
| 6112265 | Harriman et al. | Aug 2000 | A |
| 6145052 | Howe et al. | Nov 2000 | A |
| 6170022 | Linville et al. | Jan 2001 | B1 |
| 6182177 | Harriman | Jan 2001 | B1 |
| 6212611 | Nizar et al. | Apr 2001 | B1 |
| 6219750 | Kanamaru et al. | Apr 2001 | B1 |
| 6226695 | Kaiser et al. | May 2001 | B1 |
| 6272565 | Lamberts | Aug 2001 | B1 |
| 6279064 | Bronson et al. | Aug 2001 | B1 |
| 6286079 | Basham et al. | Sep 2001 | B1 |
| 6339801 | Hefferon et al. | Jan 2002 | B1 |
| 6339802 | Hefferon et al. | Jan 2002 | B1 |
| 6345254 | Lewis et al. | Feb 2002 | B1 |
| 6345324 | Baskey et al. | Feb 2002 | B1 |
| 6401145 | Baskey et al. | Jun 2002 | B1 |
| 6427196 | Adiletta et al. | Jul 2002 | B1 |
| 6442634 | Bronson et al. | Aug 2002 | B2 |
| 6457095 | Volk | Sep 2002 | B1 |
| 6467012 | Alvarez et al. | Oct 2002 | B1 |
| 6490644 | Hyde et al. | Dec 2002 | B1 |
| 6496877 | Greenberg et al. | Dec 2002 | B1 |
| 6499077 | Abramson et al. | Dec 2002 | B1 |
| 6516379 | Deshpande et al. | Feb 2003 | B1 |
| 6567886 | Saitoh et al. | May 2003 | B1 |
| 6571298 | Megiddo | May 2003 | B1 |
| 6574676 | Megiddo | Jun 2003 | B1 |
| 6591348 | Deshpande | Jul 2003 | B1 |
| 6604178 | Hall | Aug 2003 | B1 |
| 6614709 | Spiegel et al. | Sep 2003 | B2 |
| 6665756 | Abramson et al. | Dec 2003 | B2 |
| 6681289 | Espeseth et al. | Jan 2004 | B2 |
| 6684311 | Fanning | Jan 2004 | B2 |
| 6694390 | Bogin et al. | Feb 2004 | B1 |
| 6704835 | Garner | Mar 2004 | B1 |
| 6725348 | Marier et al. | Apr 2004 | B1 |
| 6728845 | Adiletta et al. | Apr 2004 | B2 |
| 6763404 | Berning et al. | Jul 2004 | B2 |
| 6774927 | Cohen et al. | Aug 2004 | B1 |
| 6779036 | Deshpande | Aug 2004 | B1 |
| 6785793 | Aboulenein et al. | Aug 2004 | B2 |
| 6848020 | Hall | Jan 2005 | B2 |
| 6892250 | Hoskins | May 2005 | B2 |
| 6895454 | Barrick | May 2005 | B2 |
| 6901461 | Bennett | May 2005 | B2 |
| 6944721 | Arimilli et al. | Sep 2005 | B2 |
| 6963882 | Elko et al. | Nov 2005 | B1 |
| 6965965 | Espeseth et al. | Nov 2005 | B2 |
| 6973550 | Rosenbluth et al. | Dec 2005 | B2 |
| 7010654 | Blackmon et al. | Mar 2006 | B2 |
| 7013336 | King | Mar 2006 | B1 |
| 7061714 | Yu | Jun 2006 | B1 |
| 7080174 | Thorsbakken | Jul 2006 | B1 |
| 7082480 | Bogin et al. | Jul 2006 | B2 |
| 7093111 | Frommer et al. | Aug 2006 | B2 |
| 7107413 | Rosenbluth et al. | Sep 2006 | B2 |
| 7127574 | Rotithor et al. | Oct 2006 | B2 |
| 7136938 | Clark et al. | Nov 2006 | B2 |
| 7143226 | Fields et al. | Nov 2006 | B2 |
| 7143246 | Johns | Nov 2006 | B2 |
| 7149226 | Wolrich et al. | Dec 2006 | B2 |
| 7177982 | Barrick | Feb 2007 | B2 |
| 7203811 | King et al. | Apr 2007 | B2 |
| 7219273 | Fisher et al. | May 2007 | B2 |
| 7225326 | Bennett et al. | May 2007 | B2 |
| 7266650 | Ganfield et al. | Sep 2007 | B2 |
| 7277982 | Calvignac et al. | Oct 2007 | B2 |
| 7289992 | Walker | Oct 2007 | B2 |
| 7296108 | Beukema et al. | Nov 2007 | B2 |
| 7296273 | Kline | Nov 2007 | B2 |
| 7305500 | Adiletta et al. | Dec 2007 | B2 |
| 7313638 | Ainm et al. | Dec 2007 | B2 |
| 7321369 | Wyatt et al. | Jan 2008 | B2 |
| 7328317 | Hillier et al. | Feb 2008 | B2 |
| 7359824 | Blouin et al. | Apr 2008 | B2 |
| 7366800 | Flynn | Apr 2008 | B2 |
| 7383464 | Gilbert et al. | Jun 2008 | B2 |
| 7392367 | Clark et al. | Jun 2008 | B2 |
| 7418540 | Rohit et al. | Aug 2008 | B2 |
| 7444435 | King et al. | Oct 2008 | B2 |
| 7467256 | Jain et al. | Dec 2008 | B2 |
| 7475202 | Hillier et al. | Jan 2009 | B2 |
| 7493456 | Brittain et al. | Feb 2009 | B2 |
| 7506084 | Moertl et al. | Mar 2009 | B2 |
| 7523228 | Biran et al. | Apr 2009 | B2 |
| 7535918 | Vasudevan et al. | May 2009 | B2 |
| 7546393 | Day et al. | Jun 2009 | B2 |
| 7546604 | Xu et al. | Jun 2009 | B2 |
| 7613841 | Asano et al. | Nov 2009 | B2 |
| 7631154 | Bellows | Dec 2009 | B2 |
| 7644198 | King et al. | Jan 2010 | B2 |
| 7660919 | Flynn | Feb 2010 | B2 |
| 7673111 | Chen et al. | Mar 2010 | B2 |
| 7694026 | Huffman | Apr 2010 | B2 |
| 7747788 | Chang et al. | Jun 2010 | B2 |
| 7765081 | Blouin et al. | Jul 2010 | B2 |
| 7805543 | Chang et al. | Sep 2010 | B2 |
| 7827449 | Gilbert et al. | Nov 2010 | B2 |
| 7831812 | Ng et al. | Nov 2010 | B2 |
| 7870111 | Walker | Jan 2011 | B2 |
| 7870334 | Zohar et al. | Jan 2011 | B2 |
| 7895239 | Wolrich et al. | Feb 2011 | B2 |
| 7908403 | Bendyk et al. | Mar 2011 | B2 |
| 7908443 | Hillier et al. | Mar 2011 | B2 |
| 7913034 | Calvignac et al. | Mar 2011 | B2 |
| 7925824 | Brittain et al. | Apr 2011 | B2 |
| 7925826 | Brittain et al. | Apr 2011 | B2 |
| 7930469 | Brittain et al. | Apr 2011 | B2 |
| 7985825 | Brittain et al. | Apr 2011 | B2 |
| 7962921 | James et al. | Jun 2011 | B2 |
| 7996572 | Blankenship et al. | Aug 2011 | B2 |
| 8056080 | Alexander et al. | Nov 2011 | B2 |
| 8081646 | Bishop et al. | Dec 2011 | B1 |
| 8082396 | Blackmon et al. | Dec 2011 | B2 |
| 8092288 | Theis | Jan 2012 | B2 |
| 8131921 | Ooi | Mar 2012 | B2 |
| 8140781 | Teh et al. | Mar 2012 | B2 |
| 8161234 | Ooi | Apr 2012 | B2 |
| 8230450 | Acedo et al. | Jul 2012 | B2 |
| 8255592 | Wang et al. | Aug 2012 | B2 |
| 8316179 | Ooi | Nov 2012 | B2 |
| 8316219 | Bellows et al. | Nov 2012 | B2 |
| 8341237 | Benhase et al. | Dec 2012 | B2 |
| 8345549 | Stephens | Jan 2013 | B2 |
| 8352946 | Srivatsa et al. | Jan 2013 | B2 |
| 8364863 | McGowan | Jan 2013 | B2 |
| 8380923 | Wolrich et al. | Feb 2013 | B2 |
| 8392636 | Fritz et al. | Mar 2013 | B2 |
| 8423970 | Pett | Apr 2013 | B2 |
| 8447905 | Ambroladze et al. | May 2013 | B2 |
| 8457777 | Goodman et al. | Jun 2013 | B2 |
| 8478968 | Bellows et al. | Jul 2013 | B2 |
| 8488960 | Decusatis et al. | Jul 2013 | B2 |
| 8495604 | Bellows et al. | Jul 2013 | B2 |
| 8516461 | Bellows et al. | Aug 2013 | B2 |
| 8560803 | Orf et al. | Oct 2013 | B2 |
| 8566532 | Berger et al. | Oct 2013 | B2 |
| 8572622 | Alexander et al. | Oct 2013 | B2 |
| 8606992 | Ooi | Dec 2013 | B2 |
| 8607003 | Bland et al. | Dec 2013 | B2 |
| 8635384 | Wang et al. | Jan 2014 | B2 |
| 8677031 | Tamir et al. | Mar 2014 | B2 |
| 8706970 | Orf et al. | Apr 2014 | B2 |
| 8719843 | Tamir | May 2014 | B2 |
| 20030182499 | Espeseth | Sep 2003 | A1 |
| 20070168610 | Kobayshi et al. | Jul 2007 | A1 |
| 20090077233 | Kurebayashi | Mar 2009 | A1 |
| 20100011149 | Molaro et al. | Jan 2010 | A1 |
| 20110314182 | Muppirala et al. | Dec 2011 | A1 |
| 20120297155 | Yamaguchi | Nov 2012 | A1 |
| 20130262678 | Tung | Oct 2013 | A1 |
| 20140195699 | Sokol, Jr. | Jul 2014 | A1 |
| 20150100617 | Diederich | Apr 2015 | A1 |
| 20150220278 | Sarcone et al. | Aug 2015 | A1 |
| 20160004479 | Hayes et al. | Jan 2016 | A1 |
| 20160098228 | Coronado et al. | Apr 2016 | A1 |
| 20180260152 | Bar | Sep 2018 | A1 |
| Entry |
|---|
| USPTO-issued prosecution history for U.S. Appl. No. 14/471,981, filed Aug. 28, 2014, including: Notice of Allowance and Fees Due (PTOL-85) dated Mar. 20, 2019, 7 pages; Decision on Reconsideration Granted—Test 2 issued Feb. 15, 2019, 12 pages; Patent Board Decision—Examiner Affirmed issued Nov. 6, 2018, 12 pages; Appeal Docketing Notice issued Feb. 28, 2018, 2 pages; Examiner's Answer to Appeal Brief issued Dec. 19, 2017, 9 pages; (continued in row 2 below). |
| Pre-Appeal Brief Conference decision issued Sep. 26, 2017, 2 pages; Final Rejection dated Jun. 2, 2017, 7 pages; Non-Final Rejection dated Jan. 12, 2017, 5 pages; Advisory Action dated Sep. 26, 2016, 4 pages; Final Rejection dated Jul. 1, 2016, 12 pages; Non-Final Rejection dated Feb. 2, 2016, 11 pages; 83 pages total. |
| Number | Date | Country | |
|---|---|---|---|
| 20180335976 A1 | Nov 2018 | US |
