The invention pertains generally to integrated circuit technology and to integrated circuits used in data communications. More particularly, the invention relates to a method and apparatus for aligning multiple data channels and matching multiple transmissions with one receiving rate.
Recent developments in data communications have created the need for increasing the amount and speed of data transmitted. As the volume of data communicated increases, there is an increased desired for greater transmission speeds over various media. One limitation to greater transmission speeds is the decreased reliability of data signals at high speeds. That is, high speed transmission of digital data is limited by the reliability of cost effective transmission media.
One approach for overcoming this limitation has been the use of multiple parallel data channels. Multiple parallel data channels allow transmission in parallel over multiple carriers and/or media rather than in series over a single transmission medium. The effective transmission speed is then approximately the product of the single channel rate and the number of multiple channels. Generally, this allows much higher transmission rates over conventional media but requires more transmission channels.
Digital communication networks are one place where higher transmission speeds are desirable. Data is often broken into smaller segments, or packetized, at a transmitting device before it is transmitted over the network. A receiving device reconstructs the packets to rebuild the data into its original form. The packets may be transmitted over multiple physical or logical channels from the transmitting device to the receiving device on the network.
Each channel may represent a different transmission path between two or more devices on the network. All channels may comprise the same transmission medium or each channel may comprise a different transmission medium. Channel media may include twisted pair, coaxial cable, fiber optic, and/or air.
Ideally, packet transmissions would be synchronized throughout the network. However, it is often impractical or impossible to synchronize data transmissions between multiple devices and over multiple channels. In many cases it may not be possible to provide a single synchronizing clock to all devices on the network.
Even with synchronous transmissions over multiple channels, the different physical and/or logical channels employed may cause data packets to become skewed or misaligned. Generally, in order to process the received data, the packets must be aligned by the receiving device.
Another problem with multi-channel transmission schemes is the inherent rate differences between clocks. That is, even clocks with the same specified frequency have small differences or errors over many cycles. Over time, these small errors may result in processing errors of data received over multiple channels. For instance, differences between the write clocks for each transmission channel and the read clock at the receiving side may result in skipping or corrupting data at the receiving side.
Accordingly, there is a need for a method and/or apparatus that can align the data received over multiple channels and match the frequencies of transmitting and receiving clocks.
In the following detailed description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be obvious to one skilled in the art that the present invention may be practiced without these specific details. In other instances well known methods, procedures, components, and/or circuits have not been described in detail so as not to unnecessarily obscure aspects of the present invention.
Some aspects of the invention provide a novel scheme to align data received from multiple transmission channels and compensate for differences in transmit and receive clock frequencies.
In one embodiment, the network device 102 may be part of a computer, server, or other electronic device. In various implementations, the invention may be embodied within a multi-component circuit, one or more integrated circuit devices, one or more computer instructions, and/or a machine-readable medium.
In one embodiment of the invention, the channel aligning system 202 comprises an alignment control unit 210, a control code detector 212 for each channel, and a data buffer 214 for each input channel A, B, C, and D.
The alignment control unit 210 may be any number of devices including a general purpose processor, an application specific processor, a programmable device, and an application specific integrated circuit.
Similarly, the control code detector 212 may be any number of devices including a processor, an application specific integrated circuit, or other electronic device capable of detecting a particular character and/or bit pattern in a character stream or memory storage location.
The data buffer 214 may comprise any number of permanent or temporary memory storage devices to store one or more characters.
For purposes of illustration, 10Gbase-X Ethernet-compliant data frames will be used. The Institute of Electrical and Electronics Engineers (IEEE) Standard 802.3, 1998 edition and IEEE Draft P802.3ae/D1.0 are incorporated herein by reference. The IEEE 802.3 Standard specifies a physical coding sublayer (PCS). The invention may be part of an elastic receiving buffer which implements the standards specified by the PCS. It must be clearly understood however, that the invention is not limited to implementations of this standard and/or the elastic buffer and may be practiced with other standards, past, present, or future, and/or other devices.
Under the 10Gbase-X Ethernet-compliant standard, data packets are interleaved with one or more control or idle characters within an Inter-Packet-Gap (IPG). An IPG comprises one or more control or idle characters. Control or idle characters are generically denoted as /I/, and include characters /A/ (Align), /K/ (Synchronization), and /R/(Skip or Removable). As illustrated in
Particular control or idle characters may be uniquely or randomly located within an IPG. When the same IPG is transmitted across multiple channels, as illustrated in
Data across all transmission channels is said to be aligned, synchronized, or columnized when one character is transmitted in each channel at the same time reference or clock cycle. A column refers to the characters in every transmission channel driven by the same clock source on the same clock cycle. Thus when all four channels in
The minimum and maximum length of an IPG may vary according to the application. Generally, the IPG length must be large enough to include sufficient idle and alignment characters to enable the alignment process to function but small enough to avoid interfering with data transmission efficiency. Additionally, the minimum frequency of an aligning character, such as the /A/ idle character, within an IPG may be specified and its location and/or spacing within the IPG may be predefined.
According to one embodiment, the minimum length IPG in each transmission channel must contain at least four aligning characters /A/ and sixty-four idle characters in total. For multiple transmission channels, the aligning character /A/ spacing in the IPG is randomized in the range of sixteen (16) to thirty-two (32) columns. That is, the spacing between aligning characters /A/ may be as little as sixteen (16) idle columns or as much as thirty-two (32) columns in an IPG.
The alignment process starts when a first aligning character is received on any input channel. In one embodiment of the invention, the aligning process is started when a first aligning character /A/ is received in any of the input channels during an IPG transmission. The alignment process succeeds when an alignment character /A/ is received in every input channel within a maximum number of clock cycles.
For purposes of the example illustrated in
Accordingly, assuming each channel is read on the rising edge of each reference write clock cycle (RFWCLK), the aligning character /A/ in channel B, being first in time, would be detected first and the alignment process would begin at the first clock cycle. The subsequent aligning characters /A/ across channels C and A would be similarly detected on the rising edge of the fourth and seventh clock cycles respectively. However, detection of the last aligning character /A/ on channel D does not occur until the rising edge of the ninth clock cycle, more than the maximum skew of eight clock cycles or characters. Therefore, the aligning process would fail.
In the illustration of
Likewise, the same procedure is followed for all other channels and buffers until the last aligning character is received or the alignment process fails.
When the alignment process fails, the alignment character /A/ was not detected in all other channels within 8 cycles from the first /A/ being detected, then each buffer is flushed, all stalled read pointers (Read Ptr. A) are released and advanced to the un-stalled value (Read Ptr. B). As depicted in
In one embodiment, after flushing the buffers and/or resetting the read pointers, the alignment process may restart when the next aligning character /A/ is detected on any of the channels. So that lagging aligning characters from a previous alignment process are not mistaken as the start of a new alignment process, it is desirable to adequately space alignment characters within an IPG separately from each other. For example, where a maximum skew of 8 characters is utilized, a minimum spacing of sixteen characters between alignment characters may be implemented.
After successful alignment of the transmission channels, all output channels A, B, C, and D carry the same pattern of idle characters at the same output read clock cycle.
Note that the particular output read clock cycle on which the aligning characters /A/ are sent to the output channels A, B, C, and D may vary depending on the specific implementation. In one implementation, the buffered characters may be sent to each corresponding output channel concurrent with the detection of the last aligning character at the input channel. In other implementations, the buffered characters may be sent to their corresponding output channels one or more clock cycles after the detection of the last aligning character.
Even after all output channel data streams have been aligned, the problem of small differences in clock frequencies between the reference write clock and read clock remains. In one embodiment, the reference write clock may be taken from one of the input channels.
Once the alignment system 202 (
Referring again to
The rate matching control unit 220 (
The rate matching process is started when the rate matching control code detector 222 (
Frequency differences between a reference write clock (RFWCLK) and a read clock (RCLK) may be compensated by the rate matching system 204 (
If the read clock, RCLK in
If the read clock is slower than the reference write clock, then the FIFO buffer stack 702 may overflow as the characters are written faster to the FIFO buffer stack 702 than they are removed. The rate matching system 204 may remove an idle or filler character, such as an /R/ character, from each input channel, at the same location in the character stream, to prevent overflow of the FIFO buffer stack 702. In another embodiment, the filler character may be removed directly from the FIFO buffer stack 702 and the data counter 704 decremented.
It must be understood that this rate matching system 204 may also be implemented by inserting or removing characters from the FIFO buffer stack 702 rather than the incoming data channels.
Because a buffer has limited size, the rate matching system 204 may only be able to compensate up to a maximum frequency difference between a write clock and a read clock. In one implementation, this maximum frequency difference may be specified as plus or minus one cycle per ten thousand cycles. The rate matching system may compensate for larger frequency differences by increasing the size of the buffer 224.
The size of the buffer 224 may vary according to the particular implementation. Generally, a buffer may include sufficient storage locations to accommodate for synchronization latencies, overflow, and underflow. According to one embodiment, a buffer 224 may comprise a FIFO buffer stack, such as FIFO buffer stack 702, with one or more storage locations, each at least as wide as the total sum of the data bits on all channels. In one embodiment, the rate matching buffer 224 comprises a FIFO buffer stack six storage elements deep, each element forty-four bits wide.
In various embodiments, the rate matching control unit 220 may begin transmitting over the output channels, RM-Out (
According to the example shown in
Under normal operating conditions, the rate matching unit 220 (
If the buffer data counter 704 decrements to two, then the rate matching control unit 220 may assume an underflow condition and insert a filler character on the next clock cycle. According to one embodiment, the filler character may be an idle character /R/. Generally, the rate matching control unit 220 inserts a filler character during the interval between two data packets, such as during an IPG. In one embodiment, a filler character is inserted into the output channels of the rate matching control unit. In another embodiment, the rate matching control unit 220 inserts a filler character into the buffer and increments the buffer data counter 704 by one.
If the buffer data counter 704 increments to six, then the rate matching control unit 220 may assume an overflow condition and delete the next available filler character. In one embodiment, the rate matching control unit 220 removes a filler character from the FIFO buffer stack 702 and decrements the buffer data counter 704 by one. In another embodiment, the rate matching control unit 220 removes a filler character from the FIFO buffer stack 702 and decrements the buffer data counter 704 by one.
The channel alignment system may comprise an alignment state-machine 902 to control the reception and transmission of data. A control code detector may comprise combination logic 904 to detect an aligning character in a plurality of transmission channels. A logic device 914 may monitor when a first aligning character has been received on any input channel and when aligning characters have been received over all input channels. A counter 906 may monitor the number of characters between the first received alignment character on a first channel and subsequently received alignment characters on other channels. A read pointer generator and logic may serve to keep track of the storage location of each aligning character buffered on a FIFO 910. A flush generator may serve to flush the stalled data buffered in the FIFO 910 if an aligning character has not been detected on every input channel within a maximum number of characters after receiving the first aligning character. An insert generator 912 may serve to insert a filler character in each output channel where its corresponding input channel has received an aligning character.
The rate matching system may comprise a detector 1002 to detect idle characters and alignment characters. A pipelined device 1004 may further serve to monitor the number and type of characters detected. A rate matching FIFO 1006 may serve to buffer characters received synchronously with a reference write clock 1014 and transmit the buffered characters synchronously with a read clock 1016. An insertion generator 1008 may serve to insert filler characters into either the FIFO 1006 or the output channels (rm-dout) if an underflow condition is detected on the FIFO 1006. A skip generator 1010 may serve to skip or remove a filler character from the FIFO 1006 or the output channels (rm-dout) if an overflow condition is detected on the FIFO 1006. A FIFO data counter device 1012 may serve to monitor the number of data in the FIFO 1006 as new data is stored or stored data is removed. Additionally, the FIFO data counter device 1012 may be incremented or decremented depending on whether a filler character is inserted or removed by the insert generator or skip generator respectively.
While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention not be limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those ordinarily skilled in the art. Additionally, it is possible to implement the present invention or some of its features in hardware, programmable devices, firmware, integrated circuits, software or a combination thereof where the software is provided in a processor readable storage medium such as a magnetic, optical, or semiconductor storage medium.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5461621 | Little | Oct 1995 | A |
| 5802122 | Niegel | Sep 1998 | A |
| 6173380 | Jin et al. | Jan 2001 | B1 |
| 6397367 | Park et al. | May 2002 | B1 |
| 6606675 | Godbole | Aug 2003 | B1 |
| 6636993 | Koyanagi et al. | Oct 2003 | B1 |
| 6650638 | Walker et al. | Nov 2003 | B1 |
| 6738393 | Miki et al. | May 2004 | B2 |
| 6771670 | Pfahler et al. | Aug 2004 | B1 |
| 20020122435 | Mundkur et al. | Sep 2002 | A1 |
| 20020031202 | Teng | Feb 2003 | A1 |