Patent Application
20120173840
Serial attached SCSI (SAS) is a computer bus that is used to move data to and from computer storage devices, such as hard drives and tape drives. SAS depends on point to point serial protocol. Typical serial attached SCSI systems include an initiator, which is a device that originates service and requests for processing by a target device and receives responses for these requests from the target devices. An initiator may comprise a controller or host device. Targets are devices containing logical units and target ports that receive device service and task management requests for processing, and send responses for the same requests to initiator devices. Target devices may comprise a hard disk or disk array. Expanders are devices that facilitate communication between the SAS devices and facilitate the connection of multiple SAS end devices.
An embodiment of the present invention may therefore comprise a method of reducing the number of entries in a lookup table of an expander that is used in a serial attached SCSI storage system to provide routing addresses in response to an open address frame request for data comprising: providing routing information in the open address request comprising: source expander identification; source physical address information; destination expander identification; and, destination physical address information; using the routing information in the lookup table to route data between an initiator and a target.
An embodiment of the present invention may further comprise a method of reducing the number of entries in a lookup table of an expander that is used in a serial attached SCSI storage system to provide routing addresses in response to an open address frame request for data comprising: providing routing information in the open address frame request comprising, source expander address information; and, destination expander address information; using the routing information in the lookup table to route data between an initiator and a target.
An embodiment of the present invention may further comprise a process of reducing the number of entries in a lookup table of an expander that is used in a serial attached SCSI storage system that has a given topology to provide routing addresses in response to an open address frame request for data comprising: generating SAS address ranges for physical addresses of devices that exist in the topology of the storage system; storing the SAS address ranges in a lookup table; examining destination SAS addresses of the open address frame request to determine address range information of the addresses of the open address frame request; using the address range information of the addresses of the open address frame request to retrieve a physical layer index for routing to the devices.
An embodiment of the present invention may further comprise a method of reducing the number of entries in a lookup table of an expander that is used in a serial attached SCSI storage system that has a given topology to provide routing addresses in response to an open address frame request for data comprising: running a discovery routine to obtain the routing addresses for the topology of the storage system; storing the routing addresses in additional RAM; populating a lookup table with the routing addresses that are most recently used; reading the open address frame request; comparing address information contained in the open address frame request with the routing addresses stored in the lookup table; providing the routing addresses for routing the open address frame request when the routing addresses stored in the lookup table match the address information contained in the open address frame request; accessing the additional RAM when the routing addresses stored in the lookup table do not match the address information contained in the open address frame request; comparing the address information contained in the open address frame request with the routing addresses stored in the additional RAM; providing the routing addresses for routing the open address frame request when the routing addresses stored in the additional RAM match the address information contained in the open address frame request.
Rather than having an entry that includes a destination SAS address that specifies the port identifier of the SAS port to which a connection is being requested and a source SAS address field that specifies the port identifier of the SAS port that originated the open address frame, such as initiator 102, the OAF “Destination SAS Address” fields can be modified. For example, the OAF request from initiator 102 can include the routing information, such as the source expander ID, source physical address ID, the destination expander ID and the destination port ID. For example, the OAF request may be stated as:
In this case, the field of SrcExpID contains the value of the index assigned to the expander on which the source port is directly attached. In this case, the source is initiator 102, and the expander is expander 104. The field of SrcPhyID has a value of the physical address on which the source port is directly attached, i.e., one of the ports 128. Field DestExpID contains a value of the index assigned to the expander on which the destination port is directly attached. For example, if the destination is target 114 (target32), the expander is expander 112. The field DestPhyID has a value of the port on which the destination is attached, i.e., port 142.
Since there are 64 bits that are allocated for the destination addressing in the existing OAF, eight bytes having 8 bits for use in the OAF for addressing. These bytes are indicated above as bytes 11-4, as indicated above.
An alternative method of reducing table entries in table 126 is illustrated in
Accordingly, list of addresses in the table entries are addresses of the expanders, rather than the targets. For example, rather than having a target SAS address, the table will include the physical address of the last expander to which the target is directly connected, as the destination expander SAS address, and the address of the expander to which the initiator is connected. For example, if the target is target 114 and the initiator is initiator 102, as shown in
Another alternative method for reducing the entries in table 126 is to use the data fields of the OAF “MORECOMPATIBLEFEATURE,” which comprises bytes 24-27, to include address information in the OAF. A physical layer that receives an OAF ignores the “MORECOMPATIBLEFEATURE” field (“FEATURE field”). The more compatible feature field specifies additional features that are incompatible with previous versions of the standard. Since, in most topologies, there are very few initiators and expanders, and most of the devices are target drives, to take advantage of this fact, the FEATURE field of the OAF can be used to provide addition information. When the FEATURE field is set to 0, the OAF is normal, as defined in the SAS-2 specification. When the FEATURE field is set to 1, the MORECOMPATIBLEFEATURE [7:0] field value indicates the Phy number, or port number of the destination expander to which the connection should be routed. If the FEATURE field is set to 1 and the destination SAS address in OAF matches the SAS address of the expander, such as expander 112, then expander 112 knows to route the connection directly to the Phy number indicated in the MORECOMPATIBLEFEATURE [7:0]field, which is a direct connection of the target to the expander 112. If there are intermediate expanders present, entries in the table 126 will be routed to the connection of the specified destination address, which is SAS address of last expander.
The entries in the routing table 126, in accordance with this method, will include initiator SAS addresses and expander SAS addresses for the entire topology. Using this method, the number of entries in the routing table 126 will be greatly reduced, since the number of initiators and number of expanders is much less than the number of targets or end devices in most topologies, by at least an order of magnitude.
In accordance with the second and third disclosed method, the expander SAS addresses and the destination Phy number, or port number, can be used to route connections through a given topology instead of using destination target SAS addresses. One method has been shown for achieving that result, but any process that provides the expander SAS address, along with the destination port number or destination SAS address in the OAF request, is certainly within the scope of this invention.
In accordance with the first method, an expander identifier, along with a destination port number , can be used to route a connection in any given topology instead of using the destination target SAS address. Two methods have been shown, but any method for providing an expander identifier, along with a destination port number, is within the scope of this invention.
As indicated above, existing table based routing mechanisms for SAS expanders have one routing entry per each SAS address. Whenever an open address frame (OAF) request is received on an incoming port having a specified address, such as one of the ports 128 of expander 104, expanders perform a content addressable memory lookup in table 126, or some similar process, to find the outgoing physical address to which the OAF should be routed. Since the tables have one entry per SAS address, the number of table entries in table 126 that is needed to cover the topology of a storage access system increases linearly with the size of the topology.
Another method of reducing the number of entries in the content addressable memory table 126 is to specify a range of SAS addresses that are routed to a particular phy. For example, if Phy 0 is configured to reach SAS address range
Implementation of range searches can be accomplished in several different ways. One manner of performing searches for ranges of addresses is to use a ternary content addressable memory 602, which provides a mechanism for performing a lookup based on a reduced number of bits in the SAS address. The reduced number of bits automatically provides a range of addresses.
Another approach is to use a standard content addressable memory and expander hardware logic that performs an SAS address lookup. In other words, logic can be added to the processor chip, or a state machine could be used to perform address range lookups in the content addressable memory. Further, manufacturers of SAS devices can provide facilities to the drives and other devices that are able to make efficient use of SAS address ranges based upon routing mechanisms in the topology of the storage access system. For example, when purchased in bulk quantities, SAS devices from some manufacturers come with consecutive SAS addresses. This process is equally applicable to SAS expanders, as well as SAS initiators with multiple physical addresses.
As explained above, SAS expanders typically employ content addressable memories to store SAS addresses of end devices in the SAS topology. The content addressable memory is used by the expander hardware to lookup the SAS address of the destination in the incoming open address frame (OAF) and the physical address on which the OAF needs to be routed. The content addressable memory (Table 126 of
In accordance with one embodiment of the invention, software assisted SAS address lookup features are implemented that can also be automated in hardware. As such, it is possible to keep the frequently accessed SAS address entries in the content addressable memory. Software can then be used to provide additional lookup data when the content addressable memory lookup for a SAS address fails. In this manner, the expander can cater to the frequently used addresses and cater to a larger number of SAS addresses than can be supported by the content addressable memory by using a software lookup routine. Accordingly, the number of end devices that can be connected in the topology of a storage system is not limited by the content addressable memory size. Further, it is possible to implement higher level topology validation and recovery algorithms in software using this technique. At least a portion of these techniques can be automated in hardware to speed the lookup process. In addition, it is possible to profile the input/output functions flowing through the expander using this feature. Also, virtual targets and proxy targets can be established using the various embodiments of the invention.
In accordance with one embodiment, the expander software, as part of the discovery process, populates the routing table or content addressable memory 126 of the expander 104. The discovery routine in the expander software, in accordance with one embodiment, maintains the entire topology discovery information in RAM 127 since the routing table 126 is not able to hold all of the addresses for the end devices. In that case, a portion of the addresses are stored in the routing table (CAM) 126. When the expander hardware detects that there is not an address match for a destination SAS address of the open address frame (OAF), the expander hardware may generate an interrupt or other control function indicating the condition to the software. Status information provided to the software may include the destination SAS address that failed in the content addressable memory lookup. Alternatively, the entire contents of the open address frame can be provided for advanced analysis by the software.
The software, upon receiving an interrupt or other control signal that indicates the address match failure, attempts to resolve the failure in the address match by performing a lookup or comparison of the addresses in the discovery information that is stored remotely in RAM 127 from the processing chip 125 on expander 104. If a match is found using the software lookup techniques, one of two different approaches can be used to provide the match information to the expander hardware. In the first approach, the software can provide the physical address information indicating the target to which the OAF needs to be routed. Upon acknowledging the interrupt, the hardware can then utilize the physical address route provided by the software to route the OAF. In accordance with the second approach, the software can use a least recently used process to replace existing content addressable memory entries in table 126 with the new addresses provided by the software. The hardware then updates the least recently used tag in the content addressable memory 126. Upon acknowledging the interrupt, the hardware can re-initiate the content addressable memory lookup and a match will be found as a result of the address information newly added to the content addressable memory by the software. The hardware can then route the OAF on the indicated outbound physical address.
It is also possible to implement a combination of the two different approaches described above. In such a combination, the software can include logic to determine when to replace the existing content addressable memory entry to increase the performance of the routing of subsequent OAFs for the same destination end device.
Since the majority of the input/output requests in a larger topology tend to center around specific initiators and targets, at any given point in time, a software lookup in the software stored address data (content addressable memory miss-hit) will result in the new SAS address being cached in the content addressable memory. The performance penalty for performing the software lookup, as a result, is not likely to therefore be repetitive in nature. To further optimize the process described above, when a software address lookup occurs, the software may also provide the source SAS address. If the source SAS address belongs to a serial synchronous port (SSP) initiator and the SAS address is not stored in the content addressable memory, the source and destination SAS address can both be stored in the content addressable memory 126 to ensure that when the target attempts to open a connection back to the initiator, the destination SAS address is already present in the content addressable memory, to avoid an additional software lookup. Of course, these processes can also be implemented in hardware in a manner similar to a cache controller.
Caching of the content addressable memory information provided by the discovery routine in a hardware implementation would improve the performance of the system over a software lookup routine, as described above. In accordance with this alternative embodiment, the software discovery routine performs a topology discovery analysis and creates a complete routing table that is stored in RAM. Of course, a limiting factor in a hardware implementation is the size of the RAM required to store all of the routing table information, which may be on the order of a magnitude bigger than the content addressable memory on the processing chip of the expander. In the hardware implementation, the processes described above can be performed in the same manner as a software lookup. Additionally, with a software lookup, it is possible to proxy the end device in the expander by routing the OAF to the expander's internal SSP target virtual physical address. The software can then provision the bandwidth for certain initiators and targets to ensure certain quality of service.
The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and other modifications and variations may be possible in light of the above teachings. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the appended claims be construed to include other alternative embodiments of the invention except insofar as limited by the prior art.
