1. Field
Embodiments of the invention relate to the field of retry mechanisms in serialized protocols. More particularly, embodiments of the invention relate to an automated Serial (Small Computer System Interface (SCSI)) Protocol (SSP) initiator port transport layer retry mechanism.
2. Description of Related Art
Serial Attached SCSI (SAS) is a protocol evolution of the parallel SCSI protocol. SAS provides a point-to-point serial peripheral interface in which device controllers may be directly linked to one another. SAS integrates two established technologies—SCSI and Serial Advanced Technology Attachment (SATA) technologies, combining the utility and reliability of the SCSI protocol with the performance advantages of SATA's serial architecture.
SAS is a performance improvement over traditional SCSI because SAS enables multiple devices of different sizes and types to be connected simultaneously in a full-duplex mode. In addition, SAS devices can be hot-plugged.
Computer devices, storage devices, and various electronic devices are being designed to comply with faster protocols that operate in a serial fashion, such as SAS protocol, to deliver the speed and performance required by today's applications.
In the SAS specification [e.g. Serial Attached SCSI-1.1 (SAS-1.1), American National Standard for Information Technology (ANSI), T10 committee, Revision 09d, status: T10 Approval, Project: 1601-D, May 30, 2005] [hereinafter the SAS standard] defines an SSP initiator port transport layer retry (TLR) requirements for SSP initiator ports.
According to the SAS standard, the SSP initiator port should, upon receipt of a new transfer ready (XFER_RDY) frame with a RETRANSMIT bit set to one, while processing a previous XFER_RDY frame, stop processing the previous XFER_RDY frame and start servicing the new XFER_RDY frame. The initiator port should not send any write data frames for the previous XFER_RDY frame after sending a write data frame for the new XFER_RDY frame.
The SSP initiator port should process link layer errors that occur while transmitting write data frames transmitted in response to an XFER_RDY frame that has it's RETRY DATA FRAMES bit set to one as described as follows.
If a SSP initiator port transmits a write data frame and does not receive an acknowledgement (ACK/NAK timeout) or receives a negative acknowledgement (NAK), the SSP initiator retransmits all the write data frames for the previous XFER_RDY frame. For the ACK/NAK timeout case, the SSP initiator port should close the connection and open a new connection to retransmit the write data frames. In this case, the CHANGING DATA POINTER bit is set to one in the first retransmitted write data frame and to zero in subsequent write data frames. The maximum number of times the SSP initiator port retransmits each write data sequence is typically vender-specific.
On the other hand, if the SSP initiator port receives a new XFER_RDY frame or a RESPONSE frame for a command while retransmitting or preparing to retransmit the write data frame, the SSP initiator port processes the new XFER_RDY frame or RESPONSE frame and stops sending the retransmitted write data frames. In this case, the SSP initiator port does not send a write data frame for the previous XFER_RDY frame after sending a write data frame in response to the new XFER_RDY frame.
These fairly well defined rules set forth in the SAS specification for the SSP initiator port to handle transport layer retries are presently handled in firmware. Firmware implementation introduces a great deal of firmware overhead due to the large amount of required handshaking between firmware and hardware, and a great deal of processor compute cycle time.
In the following description, the various embodiments of the invention will be described in detail. However, such details are included to facilitate understanding of the invention and to describe exemplary embodiments for employing the invention. Such details should not be used to limit the invention to the particular embodiments described because other variations and embodiments are possible while staying within the scope of the invention. Furthermore, although numerous details are set forth in order to provide a thorough understanding of the embodiments of the invention, it will be apparent to one skilled in the art that these specific details are not required in order to practice the embodiments of the invention. In other instances details such as, well-known methods, types of data, protocols, procedures, components, electrical structures and circuits, are not described in detail, or are shown in block diagram form, in order not to obscure the invention.
Embodiments of the invention relate to an automated Serial (Small Computer System Interface (SCSI)) Protocol (SSP) initiator port transport layer retry mechanism. Particularly, embodiments relate to a hardware automated SSP initiator port that employs a transport layer retry (TLR) mechanism in both a wide and narrow port configuration, as opposed to utilizing firmware, to thereby improve frame processing latency, reduce protocol overhead, and to improve overall system input/output (I/O) performance. For example, the SSP initiator port may be implemented as a circuit, such as, an integrated circuit.
Turning to
Device 102 may include a processor 107 to control operations in the device 102 and SAS controller 104 to control serial communication with device 110 in accordance with the SAS standard. Further, device 102 may include memory 109 coupled to processor 107 as well as a plurality of different input/output (I/O) devices (not shown).
Similarly, device 110 may likewise include processor 117 to control operations in device 110 and SAS controller 113 to control serial communication with the other device 102 in accordance with the SAS protocol. Further, device 110 may include memory 119 coupled to processor 117 as well as a plurality of different input/output (I/O) devices (not shown).
Each device may include a SAS controller 104 and 113, respectively. Further, SAS controller 104 may include an SSP initiator port 103 and an SSP target port 106 whereas SAS controller 113 may include an SSP target port 114 and an SSP initiator port 116. In accordance with this example, device 102 through SSP initiator port 103 may communicate a task over a link to SSP target port 114 of SAS controller 113 of device 110.
It should be appreciated that device 102 and device 110 may be any type of device such as a personal computer, laptop computer, network computer, server, router, expander, set-top box, mainframe, storage device, hard disk drive, flash memory, floppy drive, compact disk read-only memory (CD-ROM), digital video disk (DVD), flash memory, hand-held personal device, cell phone, etc., or any sort of device having a processor and/or memory.
Embodiments of the invention relate to a device 102 having an SAS controller 104 that includes an SSP initiator port 103 that communicates a task across a link to another device 110 and the structures and functions by which SSP initiator port 103 implements a transport layer retry (TLR) mechanism, as will be described in detail hereinafter. To aid in this description, a task nexus 120 may be defined as a nexus between SSP initiator port 103, SSP target port 114, a logical unit (comprising the devices, links, and nodes through which a task is transmitted), and the task itself (termed an ITLQ nexus).
Looking briefly at
In one embodiment, a plurality of I/O contexts are defined for each task or ITLQ nexus, previously discussed. With reference to
For example, the I/O context for an ITLQ nexus may include a retransmit bit 320 and a target port transfer TAG 330.
The I/O context for an ITLQ nexus may further include dynamic fields 360, such as the current scatter gather list pointer (SGL_PTR) which may be a pointer to a local or host memory buffer; the current address length pair (A/L); the current I/O read/write data transfer count (I/O_XC); and the current I/O read/write data relative offset (I/O_RO).
Further, as well as the dynamic fields 360, the I/O context for the ITLQ nexus may further include snapshot fields 370, such as: snapshot SGL_PTR; snapshot A/L; snapshot I/O_XC; and snapshot I/O_RO. The snapshot fields are analogous to the dynamic fields except that they are previously saved fields for use in the SSP initiator port transport layer retry mechanism, as will be described.
As will be described, the transmit transport layer of the SSP initiator port updates the dynamic fields 360 when it transmits a write data frame from the transmit buffer to the link and receives an acknowledgement (ACK). Further, the receive transport layer updates the dynamic fields 360 when the DMA processor transmits a read data frame from the receive buffer to the host or local memory.
With reference now to
It should be noted that it is assumed that an SSP initiator port 103 assigns a unique TAG for each ITLQ nexus. The TAG field is used by the SSP initiator port 103 to associate an I/O context to a particular ITLQ nexus. If the TAG is not unique across different remote nodes the SSP initiator port 103 concatenates the remote node index with the TAG to form a unique I/O context ID to associate I/O context for a particular ITLQ nexus. Note that, each remote node is assigned a unique remote node index by the device.
With reference now to
In one embodiment, the initiator port may be hardware based. The initiator port 103 may be a circuit. For example the circuit may be an integrated circuit, a processor, a microprocessor, a signal processor, an application specific integrated circuit (ASIC), or any type of suitable logic or circuit to implement the functionality described herein.
Particularly, the initiator port 103 includes a transmit transport layer 508 and receive transport layer 504 both of which are coupled to a link 502. A transmit protocol processor 512 of the transmit transport layer 508 controls a TLR mechanism in a serialized protocol. A receive protocol processor 532 of the receive transport layer 504 is coupled to the transmit transport layer and likewise controls the TLR mechanism in the serialized protocol.
Particularly, both receive and transmit transport layers 504 and 508 are coupled to link and physical layers 502. Further, both the receive transport layer (RxTL) 504 and transmit transport layer (TxTL) 508 both utilize a direct memory access (DMA) processor 520 and Context Memory.
Looking more particularly at receive transport layer 504, receive transport layer 504 includes a receive frame parser 536 for parsing frames which received from link and physical layer 502, a receive buffer 534 for storing receive frame data, an SAS receive protocol processor 532, and common I/O context storage 530 to store I/O contexts for the ITLQ nexuses as previously discussed with
Looking at the transmit transport layer 508, the transmit transport layer 508 includes common I/O context storage 530 to store the I/O contexts for the ITLQ nexuses (as discussed with reference to
The SAS transmit protocol processor 512 and the SAS receive protocol processor 532 are utilized in implementing SAS standard protocols as well as in implementing aspects of the transport layer retry (TLR) mechanism as will be described. The SAS transmit and receive processors may be any type of suitable processor or logic to accomplish these TLR functions. Additionally, each of the previously discussed components of the SSP initiator port 103 and their respective functionality in implementing aspects of the transport layer retry mechanism will now be discussed in detail with reference to
With reference now to
Looking particularly at
As illustrated at point 610 of
As shown in
If one of the transmit transport layers 508 is processing the previous XFER_RDY frame for that ITLQ nexus (e.g. transport layer 5080), the transmit transport layer 5080 of the SSP initiator port 103 finds that the broadcasted TAG matches its current executing I/O write command TAG. This is the case here as is indicated at 620. Further, the SSP initiator port checks that the new XFER_RDY frame is valid based on the SAS standard before it starts the SSP TLR process. Otherwise, if none of the transmit transport layers 508 is processing the previous XFER_RDY frame, the receive transport layer can start processing the retransmit XFER_RDY frame by setting the retransmit bit in the I/O context for that ITLQ nexus.
First, based upon the SAS standard, the new XFER_RDY frame header includes a different value in its TARGET PORT TRANFSER TAG FIELD. Thus, the transmit transport layer 5080 of the SSP initiator port 103 updates the I/O context TARGET PORT TRANSFER TAG FIELD for that particular ITLQ nexus. Thus, all the subsequent write data frames use the new TARGET PORT TRANSFER TAG value in their frame header.
It should be noted that if the transmit transport layer 5080 has a write data frame already on the fly in the link layer it must finish it first. Then, based on the SAS standard, there are two possible ways that the SSP initiator port 103 may stop processing the previous XFER_RDY frame before servicing the new XFER_RDY frame. For example, the transmit transport layer 5080 of the SSP initiator port 103 may: a) finish transmitting all the write data frames in the transmit buffer 514 and complete all the DMA descriptors already passed to the DMA processor 520; or b) flush all the remaining write data frames in the transmit buffer 514 and terminate all the DMA descriptors already passed to the DMA processor 520. This operation is indicated at point 630 in
Lastly, in order for the transmit transport layer 5080 to process the new XFER_RDY frame, the transport layer 5080 and the receive transport layer 5040 cooperate to maintain the dynamic and snapshot fields in the I/O context for that ITLQ nexus. Particularly, when any of the receive transport layers 504 in a wide SSP initiator port 103 receives a valid XFER_RDY frame with the RETRANSMIT bit set to zero for any outstanding I/O write commands, it takes a snapshot of the dynamic fields and copies them to the snapshot fields of the I/O context for that ITLQ nexus, as previously discussed with reference to
Thus, if later the SSP initiator port 103 receives a new XFER_RDY frame with the RETRANSMIT bit set to one for that particular ITLQ nexus, transmit transport layer 5080 can roll back the dynamic fields in the I/O context to the beginning of the previous XFER_RDY frame by copying the snapshot fields back to the dynamic fields. This enables the transmit transport layer to service the new XFER_RDY frame from the beginning of the new XFER_RDY frame, as is required by the SAS standard. Particularly, as seen at point 650 of
Turning now to
As particularly shown in
For the ACK/NAK or NAK case, the transmit transport layer requests the link layer to close the connection. In order to handle closing the connection due to the ACK/NAK timeout before the transmit transport layer 508 begins the retry sequence, the transmit transport layer (as shown at point 740 of
The maximum number of times a transmit transport layer attempts to retry a write data frame may be programmable by a specific configuration space or mode page, or chip initialization parameter.
Turning now to
As shown in
For example, the transmit transport layer 5080 may: a) finish transmitting all the write data frames in the transmit buffer 514 and complete all the DMA descriptors already passed on to the DMA processor 520; or b) flush all the remaining write data frames in the transmit buffer 514 and terminate all the DMA descriptors already passed through the DMA processor 520. After the transmit transport layer 5080 stops retransmit the retry write data frames, at point 840, the receive transport layer 504N can process the RESPONSE frame.
On the other hand, if none of the transmit transport layer in the same SSP initiator port is retransmitting or preparing to retransmit write data frames for that ITLQ nexus, the receive transport layer can start processing the RESPONSE frame.
Lastly, with reference to
When an SSP target port retransmits read data frames for an ITLQ nexus due to a ACK/NAK timeout or receives a NAK, it is required to retransmit all the read data frames from the last ACK/NAK balance point (e.g. as shown at point 910 in
As shown in
If the read data frames data offset field is less than the I/O context dynamic's I/O read/write data relative offset, the SSP initiator port 103 jumps to discard mode (for this particular ITLQ nexus) and discards all the read data bytes received for that ITLQ nexus until the saved dynamic's read/write data relative offset has been reached; then it switches back to the normal receive mode to save all future data bytes for this particular ITLQ nexus. On the other hand, if the read data frame's data offset field is equal to the I/O context dynamic's input/output read/write data offset field, it just enters the normal receive mode to save all data bytes for this particular ITLQ nexus.
Looking at the particular example shown in
Then, at point 940 in
According to embodiments of the invention, a complete hardware automated mechanism to handle SSP initiator port transport layer retries, requiring virtually no assistance from firmware at all, is disclosed. In this way, firmware overheads are significantly reduced and there is a significant reduction in CPU compute cycle time and handshaking between firmware and hardware. This translates into improved overall system performance and improved SAS protocol control performance, especially, in multiple protocol applications. Moreover, the firmware design that is still required is substantially simplified, especially in large storage system environments and the real time handling requirements from the firmware is significantly reduced.
Further while the embodiments of the invention have been described with reference to illustrated embodiments, these descriptions are not intended to be construed in the limiting sense. Various modifications of the illustrative embodiments, as well as other embodiments of the invention, which are apparent to persons skilled in the art to which embodiments of the invention pertained, are deemed to lie within the spirit and scope of the invention.