Patent Application
20060031555
This invention relates generally to digital semantic processors, and more particularly to methods and apparatus for switching parsing contexts while parsing a data stream.
SONET (Synchoronous Optical NETwork) refers to a widely used standard for digital communication across fiber-optic cables, using a hierarchy of several different line rates. The SONET standards (ANSI T1.105.x) define line rates as STS-N, where “STS” is an acronym for Synchronous Transport Signal and “N” is commonly one of 1, 3, 12, 24, 48, 192, and 768. The line rate of an STS-N signal is N×51.84 Mbps (million bits per second), transmitted as 8000 frames/second, the frame size growing proportionally with N. The following background description contains a brief introduction to SONET concepts and terminology that will be helpful in understanding the embodiments described later in the application.
Referring to
Other SONET devices can reside along the path between STS terminal MUX 110 and PTE 120. For instance, two add/drop multiplexers (ADMs) 130 and 140, and two repeaters 150 and 160 are shown in the path between STS terminal MUX 110 and PTE 120.
ADMs 130 and 140 multiplex lower-rate STS-N signals to a higher-rate STS-N signal and vice-versa. For instance, ADM 130 could multiplex four STS-3 lines to an STS-12 line, and/or could extract (drop) some STS-3 lines from an incoming STS-12 and replace (add) other STS-3 lines to produce an outgoing STS-12 line.
Repeaters such as 150 and 160 can be inserted in lines too long for a single long fiber to reliably carry the SONET signal between endpoints. The repeaters cannot modify the SONET payload, but merely retime and retransmit it.
Three different layer terminologies are also illustrated in
A SONET link carries overhead bits for the path, line, and section layers. These overhead bits are referred to respectively as Path OverHead (POH), Line OverHead (LOH), and Section OverHead (SOH). SONET endpoints that are only section endpoints can generate and/or modify SOH, but cannot modify LOH or POH. Endpoints that are also line endpoints can additionally generate and/or modify LOH, but cannot modify POH. Path endpoints are the only endpoints allowed to create POH.
Overhead and payload occupy specific locations in a SONET frame. The general structure of a SONET frame 200 is illustrated in
The beginning location of the POH is actually specified by an offset stored in the first two bytes of the LOH, referred to herein as the “H1H2 pointer.” Path-Terminating Equipment (PTE) interprets the H1H2 pointer values to locate the beginning of the next POH first byte that follows the H1H2 pointer. For instance, an H1H2 pointer offset of 0 would indicate that the frame overhead begins at row 4, column 4, just to the right of the H1H2 pointer.
As illustrated in
One of the types of data that can be carried in a SONET SPE is packet data, such as is commonly carried on well-known computer networks. The packets carried in a SONET frame sequence need not be arranged in any synchronous fashion with respect to the SONET frames. For instance,
A further complication in the SONET hierarchy lies in the ability within the SONET framework to build a higher-level SONET frame stream from different combinations of lower-level SONET frame streams. For instance, a segment of a SONET network 500, illustrated in
In SONET network segment 500, a first STS-3 multiplexer (MUX) 510 combines three STS-1 streams A, B, and C to create an STS-3 stream AA. A second STS-3 MUX 520 combines three other STS-1 streams D, E, and F to create another STS-3 stream DD. STS-12 MUX 530 creates an STS-12 stream AAA by combining STS-3 streams AA and DD with two STS-3c concatenated streams BB and CC.
The overall frame structure 600 of a frame from STS-3 stream AA is illustrated in
The next 261 columns in frame structure 600 contain STS-3 payload, consisting of byte-interleaved content from the three received STS-1 SPEs. For instance, column 10 of frame structure 600 contains column 4 from an STS-1 stream A frame, column 11 contains column 4 from an STS-1 stream B frame, and column 12 contains column 4 from an STS-1 stream c frame. This byte-interleaved pattern then repeats for column 5 of each of the three STS-1 streams, and so on up to column 87 of the three STS-1 streams, which appear respectively in columns 268, 269, and 270 of the STS-3 frame.
Although the frame structure for STS-12 stream AAA is not illustrated, STS-12 multiplexer 530 takes four input STS-3 and/or STS-3c streams and byte-interleaves them in the same manner as just described. The output pattern for the STS-12 SPE in this example would repeatedly byte-interleave the four STS-3 input streams with the pattern AA, BB, CC, DD, AA, BB, . . . , CC, DD. Expanding this pattern to include the embedded STS-1s per
Conventionally, a set of demultiplexers exactly mirroring the multiplexers shown in
An STS-48 frame is created similarly to an STS-12 frame, except in place of the 4:1 multiplexer 530 of
The invention may be best understood by reading the disclosure with reference to the drawing, wherein:
In the example of
It would be desirable for a single device to be able to handle different STS-3, STS-12, STS-48, etc., and/or handle both STS demultiplexing and payload processing for an STS data stream. It has now been discovered that, with certain enhancements, a semantic processor having a direct execution parser can be configured to handle such tasks. For instance, the following description contains semantic processor embodiments and methods with the potential to receive a byte-interleaved STS frame structure such as that previously described, and completely interpret and process that structure, including the packet data carried in the SPE if desired. As an added benefit, at least some described embodiments can be reconfigured, by loading different parser grammar, to handle other STS frame structures and/or payload content.
One difficulty in directly parsing a received SONET frame structure is that the byte stream represents data from many different parsing contexts, intermixed in a fashion that typically changes parsing contexts at each received byte. SONET processing occurs in at least one context, and could be divided into section, line, path, and framing contexts. Each atomic STS-1 or STS-Nc payload stream also uses a different context.
The arrangement of the contexts also depends on how the STS-N stream is created, and also changes over time as the POH columns are allowed to shift. Accordingly, for general SONET processing the order of received contextual information cannot be predicted readily more than a frame in advance.
This problem of multiple parsing contexts in a single data stream is not unique to SONET parsing. Other networking streams exist that can be parsed more easily when parsing is not constrained to a single, linear context. Because SONET is believed to represent a difficult example of such a stream, the described embodiments focus on a SONET implementation. Those skilled in the art will recognize how multiple-context parsing can be applied to other parsing problems. Those skilled in the art will also recognize that not every multiple-context parser will need some of the functionality that is specific to SONET, such as a byte de-interleaver and POH counters.
The semantic processor embodiment illustrated in
Turning first to
A parser 900 is connected to receive data input from PIB 800. A Parser Table (PT) 740 stores parsing data particular to the input that semantic processor is configured to parse. A Production Rule Table (PRT) 750 stores grammatical production rules particular to the input that semantic processor is configured to parse. A Semantic Code Table (SCT) 770 stores microcode segments for a SPU cluster 760, which preferably contains multiple semantic processing units capable of executing the microcode segments when so instructed, e.g., by parser 900 or another SPU. A SPU memory 780 stores data needed by the SPU, such as session contexts, packet data currently being processed, etc. Preferably, PT 740, PRT 750, and SCT 770 are implemented with programmable on-chip memory, and SPU memory 780 is implemented with a cached DRAM external memory.
In operation, PIB 800 receives input data, such as a sequence of Ethernet frames, SONET frames, Fibre Channel frames, etc., from an appropriate PHY, such as the OC-N PHY 710 of
When parser 900 receives input data from PIB 800, it can do one of two things with the input data. First, it can literally match input data symbols with literal (“terminal”) symbols stored in the production rules. Second, and generally more common, parser 900 can “look up” additional production rules for the data input symbols, based on the current data input symbol(s) and a current production (“non-terminal” or “NT”) symbol. It is noted that terminal symbol matching can generally be performed using either production rule terminal symbols or parser table terminal match values.
When parser 900 is directed by the grammar to look up an additional production rule, parser 900 supplies one or more current data input (DI) symbols and non-terminal (NT) grammar symbols to parser table 740. Based on the (NT, DI) pairing supplied by the parser, parser table 740 issues a corresponding PR code to production rule table (PRT) 750. In some embodiments, parser table 740 is implemented with a Ternary Content-Addressable Memory (TCAM) consisting of entries of the form (NT, DI_match). Multiple (NT, DI_match) entries can exist for the same NT symbol and different DI_match values. The TCAM will return, as the PR code, the address of the entry with the correct NT value and DI_match value that matches the supplied data input DI. Because the TCAM allows individual bits or bytes in each TCAM entry to be set to a “don't care” value, each TCAM entry can be tailored to respond to the bits or bytes in DI that matter for the current production rule.
When parser table 740 locates the correct PR code, the code is passed to production rule table 750. PRT 750 locates a production rule corresponding to the PR code, and outputs the production rule. Although the production rule can contain whatever is desired, in
Based on a SEP supplied from PRT 750, SPU cluster 760 loads and executes a code segment to perform a task. For instance, a SPU could be directed to move the payload portion of an input packet from PIB 800 to SPU memory 780 and manipulate that payload portion in some fashion. SPU cluster 760 can also create output packet/frames and supply them to POB 720 for outbound transmission at PHY 710.
The preceding introduction to operation of one semantic processor embodiment is sufficient to allow understanding into the interoperation of PIB 800 and parser 900, which will be discussed in detail regarding multiple-context SONET processing, with the other blocks of an exemplary semantic processor. For further explanation of the other blocks of
Although parser 900 could conceivably operate directly on a byte-interleaved SONET stream, context-switching on every byte boundary during high-speed operation could waste a prohibitive number of parser cycles. Accordingly,
The goal for this input buffer embodiment is to, on a row-by-row basis, de-interleave the byte-interleaved SPEs present in an STS-N stream. Reference will be made to
Frame structure 1000 of
Referring back to
An Input FIFO 810 receives an STS-N data stream from an OC-N PHY. Input FIFO 810 detects framing bytes within the STS-N data stream in order to frame-sync. Input FIFO 810 then coordinates with a standard SONET descrambler 820 to descramble the portions of each SONET frame that were scrambled prior to transmission. The DQI output of input FIFO 810 outputs a byte DQI of descrambled SONET frame data each time it receives a data clock signal D_CLK from a VIB (Virtual Input Buffer) stage 830.
VIB stage 830 has the task of matching each byte of DQI with a corresponding VIBAddress entry from a programmable Byte De-Interleaver Pattern (BDIP) register 834. VIB stage 830 obtains one VIBAddress entry from BDIP register 834, and then BDIP register increments, each time VIB stage 830 asserts an address clock signal A_CLK.
BDIP register 834 contains a mapping for one row of SONET frame data to a group of VIB input buffers in an input buffer 850. For instance, the pattern for frame structure 600 of
An address decode stage 832 receives DQI/VIBAddress pairs and &/VIBAddress pairs from VIB stage 830. Address decode stage 832 supplies the VIBAddress to VIB pointer registers block 840, which returns a physical buffer address corresponding to the VIBAddress. Address decode stage 832 writes the DQI or “&” to the physical buffer address in input buffer 850.
The VIB pointer registers 840 are configured to operate as a plurality of ring buffer pointer registers, each storing a physical StartAddress, a physical EndAddress, a WritePointer, and a ReadPointer. When address decode stage 832 supplies a VIBAddress to VIB Pointer Registers 840, the correct WritePointer is retrieved and returned to address decode stage 832 as a physical buffer address. The WritePointer is then incremented, and reset to StartAddress when WritePointer reaches EndAddress.
On the output side of PIB 800, an output controller 860 interfaces with a parser through a FrameRowReady output signal, a parser output FIFO 862, and a DataRequest input. Output controller 860 also interfaces with a SPU or SPU cluster through a SPU output FIFO 864 and a SPU_IN FIFO 866. The operation of both interfaces will be described in turn.
When output controller 860 receives an S_Wrap signal from BDIP register 834, controller 860 asserts FrameRowReady to the parser. Output controller 860 can then respond to DataRequest inputs from the parser by transmitting DQO (DQ Output) values to the parser through parser FIFO 862.
Output controller 860 loads parser FIFO 862 by issuing D_REQ (data request) signals to an address decode stage 870. Address decode stage 870 is initially set internally to read out of VIB 0. Each time it receives a D_REQ signal, address decode stage requests the current ReadPointer for VIB 0 from VIB pointer registers 840. Address decode stage 870 supplies this ReadPointer as a buffer read address to input buffer 850 and receives a DQO. Meanwhile, VIB Pointer Registers 840 increment ReadPointer, and reset it to StartAddress when ReadPointer reaches EndAddress.
When address decode stage 870 receives a DQO from input buffer 850 that contains an “&” control character, it knows that it has reaches the end of that virtual buffer's input for the current SONET row. Accordingly, address decode stage 870 increments its internal VIBAddress from VIB 0 to VIB 1 and begins reading from VIB 1 on the next D_REQ. This same behavior is repeated as an “&” is reached in each VIB, until the “&” in the last allocated VIB is reached. At that point, the internal VIBAddress is reset to VIB 0 in preparation for reading the next frame row.
DQO values can be read out to a SPU in similar fashion through SPU FIFO 864, based on commands received from a SPU at a SPU_IN FIFO 866. It is also noted that “burst” read or skip forward commands can be efficiently implemented by storing pointers to the next few “&” characters in each VIB in the VIB pointer registers. Multiple consecutive DQO values can then be read at once as long as they do not read past the next “&” in the current VIB.
SPU_IN FIFO 866 can also be used to program PIB 800 at runtime. By issuing an appropriate CMD_IN, a SPU can instruct output controller 860 to receive a pattern and load that pattern to BDIP register 834 through the RegisterProgram signal interface. A SPU can also instruct output controller 860 to configure VIB StartAddress and EndAddress values for each active VIB in VIB Pointer Registers 840 and then initialize the ReadPointer and WritePointer values. A SPU can also instruct output controller 860 to configure input FIFO for the appropriate frame size and alignment character sequence, and load a seed to descrambler 820.
VIB stage 830 and/or Address Decode stage 832 can be pipelined as necessary to achieve an appropriate throughput for designed STS-N data rates. BDIP register 834 is designed with a size sufficient to hold a row mapping for the largest STS-N of interest. Input buffer 850 is designed with a size sufficient to hold at least two rows of the largest STS-N of interest. VIB pointer registers 840 are designed with enough pointer register sets to de-interleave the largest STS-N of interest were that STS-N composed entirely of STS-1s (for instance, 49 pointer register sets could be included to handle de-interleaving for any STS-48).
For non-byte-interleaved input, much of the functionality just described could be bypassed with a simple single-ring-buffer interface to input buffer 850. Optionally, the described hardware can be configured with a continuous VIB 0 pattern in BDIP register 834 and a VIB 0 register set with a StartAddress set to the left end of input buffer 850 and an EndAddress set to the right end of input buffer 850.
Given the STS-3 example of
Given the rearranged STS-3 frame structure of
The next 87 bytes are part of the STS-1 A payload context, and contain a POH byte and 86 payload bytes, which in this example are considered to be packet data (headers and/or packet payload). The actual location of the POH byte within these 87 bytes is part of context 1, and must be remembered from an H1H2 LOH field that was previously parsed. And the remaining 86 bytes in all likelihood do not start with the first byte of a packet, but could be a continuation of a previously unfinished packet parsing context that must be revisited.
The same observations hold for the middle 90 bytes and the last 90 bytes, respectively representing STS-1 B (contexts 3 and 4) and STS-1 C (contexts 5 and 6). The packet data in these last two byte segments will, for this example, represent different parsing contexts than the packet data in the first 87 bytes of SPE data. And the H1H2 values for these last two byte segments will indicate POH locations, within the 90 byte segments, unique to those segments.
At the end of the row shown in
A block diagram of a parser embodiment 900 is shown in
Input data interface 910 communicates with PIB 800. When PIB 800 asserts D_RDY to interface 910, interface 910 is allowed to send load or skip requests to PIB 800 until D_RDY is deasserted, signaling the end of a SONET frame row. In response to load requests, PIB 800 supplies input data to interface 910 over bus DIN.
Input data interface 910 requests data from PIB 800 in response to requests from parser state machine 930 (although interface 910 may cache not-yet-requested data as long as it responds correctly to external requests). Parser state machine 930 maintains the ID for the current context on the CTS bus. Whenever parser state machine 930 instructs input data interface to load input data to context symbol registers 920, row byte counters in a POH registers/counters 912 block, internal to input data interface 910, track the number of bytes read to that context.
Within register/counter block 912, two count registers are allocated for each context. The first, the previously mentioned row_byte_counter, counts the number of bytes read to the current context on the current frame row. The row byte counters for all contexts are reset each new frame row. The second, the SPE byte counter, counts the number of bytes read to the current context in the current SPE. The SPE byte counter counts are reset after each SPE is read in.
Four comparison registers are also allocated for each context in register/counter block 912. A row_count_max register defines the maximum number of symbols that should be read to that context on the current row, for instance 86 symbols for an STS-1 row, excluding overhead bytes. A SPE_count_max register defines the maximum number of symbols that should be read to that context for the current SPE, for instance 774 for an STS-1 SPE, excluding overhead bytes. A current_POH_pointer maintains a value for the column location of the POH in the current SPE, and a next_POH pointer maintains a value for the column location of the POH in the next SPE. The row_count_max register and SPE_count_max registers are loaded when parser table 740 and production rule table 750 are configured for the current input stream. The current_POH_pointer and next_POH_pointer are dynamic, and set by the load_POH_register signal from parser state machine 930.
A enable flag register also exists for each context; the value of the enable flag register determines whether the registers and counters for a particular context are enabled. When the registers and counters for a particular context are enabled and that context is active, the row byte and SPE byte counters increment as data is transferred to the context symbol registers 920. When the row byte counter value reaches the row_count_max register value, input data interface 910 signals a level-decrement grammar context switch to parser state machine 930 and stops any pending transfer to the current context. When the row byte counter value reaches the current_POH_pointer value, input data interface 910 signals a level-decrement grammar context switch to parser state machine 930 and stops any pending transfer to the current context. Also, when the SPE byte counter value reaches the SPE_count_max value, the next_POH_pointer is loaded to the current_POH_pointer, and the input data interface 910 skips bytes if necessary in the current row until it reaches the next_POH_pointer.
One final task of registers/counters 912 is to determine when the transmitter has signaled a positive or a negative byte stuff. A positive byte stuff is signaled by the transmitter inverting bits 7, 9, 11, 13, and 15 of the POH pointer, and a negative byte stuff is signaled by the transmitter inverting bits 8, 10, 12, 14, and 16 of the POH pointer. When parser state machine 930 loads a next_POH_pointer for a context, registers/counters 912 compare the next_POH_pointer value to the current POH_pointer value and signal a positive or negative byte stuff condition, as described, to parser state machine 930. Also, for a positive byte stuff, the row byte counter is incremented by one; for a negative byte stuff, the row byte counter is decremented by one.
Context symbol registers 920 store context symbols for each of N potential contexts. For instance, when context 2 is an IP packet context, context 2 may be exited and re-entered several times per packet before parsing is completed on that packet. Context symbol register 920 maintains the current symbol state in the interim for the suspended contexts. When register 920 receives input bytes from input data interface 910, it stores them to the context register indicated on the CTS bus. Further, the value output on the DI bus to parser state machine 930 and parser table 740 is the value contained in the context register indicated on the CTS bus. Context symbol register 930 also preferably maintains two values for each context symbol register, indicating how many of its input bytes are valid, and how many input bytes have been requested but not filled.
Parser state machine 930 coordinates operation of the other elements of parser 900, and signals all context switches. As previously mentioned, byte counter comparisons may in some circumstances signal the parser state machine to switch contexts. The example below will also demonstrate how context switches can be initiated by the grammar itself, when parser state machine receives special non-terminals that instruct it to switch contexts. Except for the context-switching function described herein, parser state machine 930 can in some embodiments function similarly to the parser state machine described in copending U.S. patent application Ser. No. 10/351,030.
Parser stack manager 940 performs pushes and pops of parser stack symbols to parser stack memory 960. In the illustrated embodiment, the CTS bus value is used by parser stack manager 940 to access a set of head/tail registers 950 for each context. The head/tail registers 950 contain two pointers for each context: a head pointer, which contains the address in parser stack memory 960 for the top-of-stack symbol for that context; and a tail pointer, which contains the address in parser stack memory 960 for the bottom-of-stack symbol for that context. When symbols are pushed to a context, the head pointer is incremented. When symbols are popped to a context, the head pointer is decremented.
The operation of and cooperation between the elements of parser 900 will be described further by example, with reference to the de-interlaced STS-3 frames of
Before parser 900 begins processing input from an STS-N stream, it can be specifically configured for the appropriate number of parsing contexts for that stream. Preferably, however, contexts are available and ready for use already, with movement between them directed by the grammar, with the STS-3 grammar being a “root” grammar and the required number of “branch” grammars for the input port.
For instance, the STS-3 root frame grammar for the illustrated embodiment can be defined as:
In the above grammar, the STS-3 stream is defined as a top of frame (TOF) symbol, followed by a de-interlaced STS-3 frame. The TOF grammar includes an instruction to “SKIP—1_BYTE,” in other words, to consume the CTL_NewFrame symbol from the symbol register 920 for context 0.
The STS-3 de-interlaced frame definition includes nine repetitions of a common pattern “@@L+1 CTS @@L+3 CTS @@L+5 CTS.” The CTS grammar looks for and consumes a CTL_ContextShift character, which was inserted in the incoming data stream between virtual input buffer segments by the PIB. The @@L+n grammar signifies a special parser directive to shift contexts, with relation to the present context, upwards n contexts. Thus on each row of a de-interlaced frame, the root grammar switches three times, to an STS-1 grammar, as defined below.
The STS-1 grammar processes STS-1 de-interlaced input on nine rows, for instance as follows:
Each row of the STS-1 frame is separately defined, such that each time an STS-1 context is called it will be re-entered at the proper location. Each row performs TOH processing, as will be described below, and then performs STS-1_SPE_row processing. The STS-1_SPE_row processing increments to the next context, which could be an IP context, ATM context, etc. The input data interface 910 will signal a context switch back to the STS-1_SPE_row grammar when the POH column is reached, at which time the POH byte is consumed. The STS-1_SPE_row grammar then increments back to the next context until input data interface 910 signals a context switch back to the STS-1_SPE_row grammar when the end of that SPE row is reached. The STS-1_SPE_row grammar then instructs the parser state machine to return to the root grammar with the @ @ Lroot command.
Each set of defined transport overhead bytes is parsed within the appropriate row.
Although not illustrated, several of the transport overhead bytes may be processed with their own sub-grammars, if desired. For instance, the D1 through D12 bytes can be used as another data channel, which could be parsed with an appropriate grammar.
One possible processing approach for TOH row 4 is illustrated. That row contains the POH pointer for the next SPE. The special directive @@XferPointer transfers the two pointer bytes to the appropriate next_POH_pointer register, causing input data interface 910 to assert Pos_Shift, Neg_Shift, or No_Shift back to parser state machine 930. Depending on the state of the asserted variable, either two, three, or four input bytes are then consumed.
Although special-purpose execution engines have been shown and described, alternative implementations can use the described multi-symbol parser as a front-end datagram processor for a general purpose processor. The STS-3 example presented above was used for simplicity—the concepts illustrated therein can be extended, e.g. to parsing STS-12, STS-48, etc., streams formed from any combination of STS-1, STS-3, STS-3c, STS-12, STS-12c, STS-48c streams.
One of ordinary skill in the art will recognize that the concepts taught herein can be tailored to a particular application in many other advantageous ways. Although one set of cooperating parser/input buffer hardware and grammar functionality has been described, others are possible. Also, not every multiple-context parsing approach will need hardware counters, as some parsing problems may contain only context-switching points that can be derived from the input data itself.
Those skilled in the art recognize that other functional partitions are possible within the scope of the invention. Further, what functions are and are not implemented on a common integrated circuit can vary depending on application.
Finally, although the specification may refer to “an”, “one”, “another”, or “some” embodiment(s) in several locations, this does not necessarily mean that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment.
Copending U.S. patent application Ser. No. 10/351,030, titled “Reconfigurable Semantic Processor,” filed by Somsubhra Sikdar on Jan. 24, 2003, is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60599830 | Aug 2004 | US |
