Patent Application
20070002827
1. Field
Embodiments of the invention relate to the field of retry mechanisms in serialized protocols. More particularly, embodiments of the invention relate to an automated Serial (Small Computer System Interface (SCSI)) Protocol (SSP) target port transport layer retry (TLR) mechanism.
2. Description of Related Art
Serial Attached SCSI (SAS) is a protocol evolution of the parallel SCSI protocol. SAS provides a point-to-point serial peripheral interface in which device controllers may be directly linked to one another. SAS integrates two established technologies—SCSI and Serial Advanced Technology Attachment (SATA) technologies, combining the utility and reliability of the SCSI protocol with the performance advantages of SATA's serial architecture.
SAS is a performance improvement over traditional SCSI because SAS enables multiple devices of different sizes and types to be connected simultaneously in a full-duplex mode. In addition, SAS devices can be hot-plugged.
Computer devices, storage devices, and various electronic devices are being designed to comply with faster protocols that operate in a serial fashion, such as SAS protocol, to deliver the speed and performance required by today's applications.
In the SAS specification [e.g. Serial Attached SCSI-1.1 (SAS-1.1), American National Standard for Information Technology (ANSI), T10 committee, Revision 09d, status: T10 Approval, Project: 1601-D, May 30, 2005] [hereinafter the SAS standard] defines an SSP target port transport layer retry (TLR) requirements for SSP target ports.
According to the SAS standard, if a TRANSPORT LAYER RETRIES bit is set to one in a protocol-specific logical unit mode page, then the SSP target port should process link layer errors that occur while transmitting transfer ready (XFER_RDY) frames. This SAS standard protocol is described as follows.
The SSP target port first sets a RETRY DATA FRAME bit to one in each XFER_RDY frame. If the SSP target port transmits a XFER_RDY frame and does not receive an acknowledgement (i.e. an ACK/NAK timeout occurs), or receives a negative acknowledgement (NAK), then the SSP target port should retransmit the XFER_RDY frame with a different value in a target port transfer tag field with the RETRANSMIT bit set to one. For the ACK/NAK timeout case, the SSP target port is required to close the connection and open a new connection to retransmit the XFER_RDY frame. The SSP target port retransmits each XFER_RDY frame that does not receive an ACK at least one time.
If the SSP target port sends a read data frame for a logical unit that has its TRANSPORT LAYER RETRIES bit set to one in the logical unit mode page, then the SSP target port should process the link layer errors that occur while transmitting read data frames as described as follows.
If the SSP target port transmits a read data frame and does not receive an ACK/NAK (i.e. an ACK/NAK timeout occurs), or receives a NAK for that frame, the SSP target port retransmits all the read data frames from the last ACK/NAK balance point. For the ACK/NAK timeout case, the SSP target port is required to close the connection and open a new connection to retransmit the read data frames.
In this case, a CHANGE DATA POINTER is set to one in the first retransmitted read data frame and to zero in subsequent read data frames. The SSP target port should retransmit each read data frame that does not receive an ACK at least one time. The number of times the SSP target port retransmits each read data frame is typically vender-specific.
These fairly well defined rules set forth in the SAS standard for the SSP target port to handle transport layer retries are presently handled in firmware. Firmware implementation introduces a great deal of firmware overhead due to the large amount of required handshaking between firmware and hardware, and a great deal of processor compute cycle time.
In the following description, the various embodiments of the invention will be described in detail. However, such details are included to facilitate understanding of the invention and to describe exemplary embodiments for employing the invention. Such details should not be used to limit the invention to the particular embodiments described because other variations and embodiments are possible while staying within the scope of the invention. Furthermore, although numerous details are set forth in order to provide a thorough understanding of the embodiments of the invention, it will be apparent to one skilled in the art that these specific details are not required in order to practice the embodiments of the invention. In other instances details such as, well-known methods, types of data, protocols, procedures, components, electrical structures and circuits, are not described in detail, or are shown in block diagram form, in order not to obscure the invention.
Embodiments of the invention relate to an automated Serial (Small Computer System Interface (SCSI)) Protocol (SSP) target port transport layer retry (TLR) mechanism. Particularly, embodiments relate to a hardware automated SSP target port that employs a transport layer retry (TLR) mechanism in both a wide and narrow port configuration, as opposed to utilizing firmware, to thereby improve frame processing latency, reduce protocol overhead, and to improve overall system input/output (I/O) performance. For example, the SSP target port may be implemented as a circuit, such as, an integrated circuit.
Turning to
Device 102 may include a processor 107 to control operations in the device 102 and SAS controller 104 to control serial communication with device 110 in accordance with the SAS standard. Further, device 102 may include memory 109 coupled to processor 107 as well as a plurality of different input/output (I/O) devices (not shown).
Similarly, device 110 may likewise include processor 117 to control operations in device 110 and SAS controller 113 to control serial communication with the other device 102 in accordance with the SAS protocol. Further, device 110 may include memory 119 coupled to processor 117 as well as a plurality of different input/output (I/O) devices (not shown).
Each device may include a SAS controller 104 and 113, respectively. Further, SAS controller 113 may include an SSP target port 114 and an SSP initiator port 116 whereas SAS controller 104 may include an SSP target port 106 and an SSP initiator port 103. In accordance with this example, device 102 through SSP initiator port 103 may communicate a task over a link to SSP target port 114 of SAS controller 113 of device 110.
It should be appreciated that device 110 and device 102 may be any type of device such as a personal computer, laptop computer, network computer, server, router, expander, set-top box, mainframe, storage device, hard disk drive, flash memory, floppy drive, compact disk read-only memory (CD-ROM), digital video disk (DVD), flash memory, hand-held personal device, cell phone, etc., or any sort of device having a processor and/or memory.
Embodiments of the invention relate to a device 102 having an SAS controller 104 that includes an SSP initiator port 103 that communicates a task across a link to another device 110 and the structures and functions by which SSP target port 114 implements a transport layer retry (TLR) mechanism, as will be described in detail hereinafter. To aid in this description, a task nexus 120 may be defined as a nexus between SSP target port 114, SSP initiator port 103, a logical unit (comprising the devices, links, and nodes through which a task is transmitted), and the task itself (termed an ITLQ nexus).
Looking briefly at
In one embodiment, a plurality of I/O contexts are defined for each task or ITLQ nexus. With reference to
For example, the I/O context for the ITLQ nexus may include a retransmit bit 320, a target port transfer TAG 330, as well as a phantom target port and transfer TAGs 340. As will be discussed, the phantom target port transfer tag is used in a process to generate a different and an unused target transfer tag due to XFER_RDY frame retries.
The I/O context for an ITLQ nexus may further include dynamic fields 360, such as the current scatter gather list pointer (SGL_PTR) which may be a pointer to a local or host memory buffer; the current address length pair (A/L); the current I/O read/write data transfer count (I/O_XC); and the current I/O read/write data relative offset (I/O_RO).
Further, as well as the dynamic fields 360, the I/O context for the ITLQ nexus may further include snapshot fields 370, such as: snapshot SGL_PTR; snapshot A/L; snapshot I/O_XC; and snapshot I/O_RO. The snapshot fields are analogous to the dynamic fields except that they are previously saved fields for use in the SSP target port transport layer retry mechanism, as will be described. Additionally, other I/O context fields 310 may be utilized.
As will be described, the transmit transport layer of the SSP target port updates the dynamic fields 360 when it transmits a read data frame from the transmit buffer to the link and receives an acknowledgement (ACK). Further, the receive transport layer updates the dynamic fields 360 when the DMA processor transmits a write data frame from the receive buffer to the host or local memory.
With reference now to
It should be noted that it is assumed that an SSP target port 114 assigns a unique TAG for each ITLQ nexus. The TAG field is used by the SSP target port 114 to associate an I/O context to a particular ITLQ nexus. If the TAG is not unique across different remote nodes, the SSP target port 114 concatenates the remote node index with the TAG to form a unique I/O context ID to associate I/O context for a particular ITLQ nexus. Note that, each remote node is assigned a unique remote node index by the device.
With reference now to
In one embodiment, the target port may be hardware based. The target port 114 may be a circuit. For example the circuit may be an integrated circuit, a processor, a microprocessor, a signal processor, an application specific integrated circuit (ASIC), or any type of suitable logic or circuit to implement the functionality described herein.
Particularly, the target port 114 includes a transmit transport layer 508 and receive transport layer 504 both of which are coupled to a link 502. A transmit protocol processor 512 of the transmit transport layer 508 controls a TLR mechanism in a serialized protocol. A receive protocol processor 532 of the receive transport layer 504 is coupled to the transmit transport layer and likewise controls the TLR mechanism in the serialized protocol. As previously discussed, the serialized protocol is compatible with a Serial Attached (Small Computer System Interface (SCSI)) (SAS) protocol standard.
As will be discussed, if the transmit protocol processor 512 of the transmit protocol layer 508 transmits a XFER_RDY frame with the RETRY DATA FRAME bit set to one and does not receive an acknowledgement or receives a NAK for that XFER_RDY frame, the transmit protocol processor 512 retransmits the XFER_RDY frame with a different target port transfer tag and with the RETRANSMIT bit set to one. In one embodiment, the different transfer tag may be a phantom tag that includes a number not associated with any outstanding ITLQ nexus.
Both receive and transmit transport layers 504 and 508 are coupled to link and physical layers 502. Further, both the receive transport layer (RxTL) 504 and transmit transport layer (TxTL) 508 both utilize a direct memory access (DMA) processor 520 and common I/O context storage 530.
Looking more particularly at receive transport layer 504, receive transport layer 504 includes a receive frame parser 536 for parsing frames received from link and physical layer 502, a receive buffer 534 for storing receive frame data, an SAS receive protocol processor 532, and common I/O context storage 530 to store I/O contexts for ITLQ nexuses (as discussed with
Looking at the transmit transport layer 508, the transmit transport layer 508 includes common context storage 530 to store I/O contexts for the ITLQ nexuses (as discussed with reference to
The SAS transmit protocol processor 512 and the SAS receive protocol processor 532 are utilized in implementing SAS standard protocols as well as in implementing aspects of the transport layer retry (TLR) mechanism as will be described. The SAS transmit and receive processors may be any type of suitable processor or logic to accomplish these TLR functions. Additionally, each of the previously discussed components of the SSP target port 114 and their respective functionality in implementing aspects of the transport layer retry mechanism will now be discussed in detail with reference to
With reference now to
Looking particularly at
As illustrated at point 610 of
Utilizing the ACK/NAK timeout case as an example, the transmit transport layer 5080 requests that the link layer close the connection. However, before closing the connection, a retry sequence is initiated in which transmit transport layer 5080 of the SSP target port 114 under the control of the SAS transmit protocol processor 512 sets RETRANSMIT bit in the I/O context for that particular ITLQ nexus to one in order to remember that the transmit XFER_RDY frame is in a retry state. Also, at point 630 the SAS transmit protocol processor 512 causes the link layer to close the connection.
Thus, when a new connection is established, the transmit transport layer can check the RETRANSMIT bit in the I/O context for that particular ITLQ nexus, find that it is equal to one, and then start retransmitting the XFER_RDY frame with a different value in the target transport transfer tag with the RETRANSMIT bit set to one.
The number of times the transmit transport layer retries a XFER_RDY frame, may be a programmable value stored in a configuration space, a load page, or a chip initialization parameter.
In one embodiment, a different target port transfer tag due to a XFER_RDY retry may be generated by the transport layer requesting a target port transfer tag that is not associated with any outstanding ITLQ nexus from firmware.
In one particular embodiment, the different transfer tag may be created by a phantom target port transfer tag mechanism.
The SAS standard defines a 16-bit target port transfer tag field, which supports up to 64,000 (e.g. 64K) outstanding I/O commands. Typically, the SSP target port supports less than the 64K outstanding I/O commands, for example, 16K. In this example, the most two significant bits are not used in the target port transfer tag.
In this phantom tag embodiment, the SSP target port 114 simply changes the two most significant bits to generate a different target port transfer tag for the XFER_RDY frame retry. Therefore, the maximum number of times it can retry is 2ˆ2=4 (2ˆnumber of bits not used in the target port transfer tag). This mechanism is referred to as the phantom target port transfer tag mechanism. The phantom target port transfer tags may be stored and updated in the I/O context for that particular ITLQ nexus (e.g. at point 620) as illustrated in
In the phantom target port transfer tag embodiment, when the transmit transport layer 508 retransmits XFER_RDY frames with a different value in the target port transfer tag field and the RETRANSMIT bit set to one, a new phantom target port transfer tag is selected (e.g. by the SAS transmit protocol processor 512) and the phantom target port transfer tag field (e.g.
Continuing with the present example of an ACK/NAK timeout case, at point 640 in
Turning to
For example, if the write data frame's offset data offset field is less than the I/O context dynamic's I/O read/write offset field, the SSP target port 114 jumps to a discard mode (for that particular ITLQ nexus) and discards all the write data bytes received for that ITLQ nexus until the saved dynamics I/O read/write offset has been reached. It then switches back to a normal receive mode to save all future data bytes for the particular ITLQ nexus.
On the other hand, if the write data frame's data offset field is equal to the I/O context dynamic's I/O read/write offset field it just enters the normal receive mode to save all the data bytes for the particular ITLQ nexus.
In one embodiment, in order to handle write data frame retry receipt, the transmit transport layer 508 takes a snapshot of the dynamic field in common context buffer 530 (e.g. see
Therefore, the SSP target port 114 can choose not to discard all the previous received good write data bytes for that ITLQ nexus until the saved relative offset for the last good write data frame. Particularly, there is no penalty for the DMA processor to retransmit those already received write data bytes to the host or local memory buffers.
An example of recreating the write data frame for a ITLQ nexus will now be described. For example, at point 710 the SSP target port 114 receives A2 and responds with an ACK, but, in this example, the ACK is lost in transmission. Accordingly, an ACK/NAK timeout occurs at the SSP initiator port. The SSP initiator port reopens a new connection and retransmits all the write data frames from the previous XFER_RDY frame, starting at frame A1. It should be noted that each time the DMA processor 520 outputs a write data frame from the receive buffer, the receive transport layer 5040 updates the dynamic fields according to the size of the write data frame in the receive buffer. In this example, the last good received data frame is A2.
As shown at point 720 with the write data frames CHANGING DATA POINTER bit set to one and the write data frames data offset field being less than the I/O read/write data relative offset of the receive transport layer's I/O common context buffer 530, the receive transport layer 5040 enters discard mode and discards all the write data until the last good received write data frames relative offset.
At point 730, the receive transmit layer 5040 goes back to a normal mode to save all the new good write data bytes in the receive buffer 534.
Turning now to
If a SSP target port 114 transmits a read data frame and does not receive an ACK/NAK (i.e. and ACK/NAK timeout occurs), or receives a NAK occurs for that read data frame, the SSP target port retransmits all the read data frames since a previous time when ACK/NAK balance occurred. For the ACK/NAK timeout case, the SSP target port 114 opens a new connection before it starts the retry sequence.
As shown in
This enables the transmit transport layer 5040 to retransmit all the read data frames from the previous ACK/NAK balance point. It starts to retransmit all the read data frames starting at frame A4.
In order to accomplish this, the link layer 502 needs to provide an ACK/NAK balance indication to the transport layer. Thus, every time when an ACK/NAK balance occurs, the transport layer 508 takes a snapshot of the dynamic fields by copying the dynamic field's contents to the snapshot fields in the common context buffer 530.
For the ACK/NAK timeout case, the transmit transport layer 5080 requests the link layer to close the connection at point 840. In order to handle closing the connection due to the ACK/NAK timeout before the transmit transport layer starts the retry sequence, the transmit transport layer 5000 needs to set the RETRANSMIT bit in the I/O Context for that particular ITLQ to one to remember that the I/O read data sequence is in a retry state.
Thus, when a new connection is established, the transmit transport layer 504N at point 850 can check the RETRANSMIT bit and the common context buffer 530 to check that it is set to one and start servicing the I/O read data retry sequence by setting the CHANGING DATA POINTER bit to one for the first retransmitted read data frame. This may be done under the control of the SAS transmit protocol processor 512 and the read data may be transmitted through transmit buffer 514 onto the link.
The number of times that the transmit transport layer 504N retries a read data frame may be programmable via a specific configuration space or mode page or chip initialization parameter.
According to embodiments of the invention, a complete hardware automated mechanism to handle SSP target port transport layer retries, requiring virtually no assistance from firmware at all, is disclosed. In this way, firmware overheads are significantly reduced and there is a significant reduction in CPU compute cycle time and handshaking between firmware and hardware. This translates into improved overall system performance and improved SAS protocol control performance, especially, in multiple protocol applications. Moreover, the firmware design that is still required is substantially simplified, especially in large storage system environments and the real time handling requirements from the firmware is significantly reduced.
Further while the embodiments of the invention have been described with reference to illustrated embodiments, these descriptions are not intended to be construed in the limiting sense. Various modifications of the illustrative embodiments, as well as other embodiments of the invention, which are apparent to persons skilled in the art to which embodiments of the invention pertained, are deemed to lie within the spirit and scope of the invention.
