Patent Application
20250028672
The overall field of this invention relates generally to employing architecture, programming models, and Application Programming Interface (API) for serial data processing, and in particular for serial processing pipelines. The disclosed embodiments relate to a system and method for an architecture that allows concurrent processing of multiple stages in a serial processing pipeline. In concert with other techniques, including hardware accelerations and alternative methods for accessing memory, parallelism improves performance in dimensions of latency, throughput, and CPU utilization.
This paper describes an architecture that allows concurrent processing of multiple stages in a serial processing pipeline. In concert with other techniques, including hardware accelerations and alternative methods for accessing memory, parallelism improves performance in dimensions of latency, throughput, and CPU utilization. Parallelism has long been exploited as a means to improve processing performance in different areas of computing. For instance, in networking, techniques such as Receive Side Scaling (RSS) parallelize packet processing across different CPUs. Those mechanisms employ horizontal parallelism to process packets concurrently, however processing for each packet remains serialized. For instance, a QUIC/IPv4 packet consists of a stack of Ethernet, IPv4, UDP, and QUIC headers—the corresponding protocol layers are processed serially for each packet. Vertical parallelism allows concurrent processing of different layers of a packet thereby reducing latency and increasing throughput. The benefits of vertical parallelism become more pronounced with increased use of encapsulation, extension headers, Type Length Value lists (TLVs), and Deep Packet Inspection (DPI). Network protocol processing is an instance of a serial processing pipeline. A serial processing pipeline is characterized by a pipeline composed of some number of stages that are expected to be processed serially where one stage must complete its processing before moving to the next one. A serial processing pipeline is parallelized by running its stages in parallel. A threading and dependency model is required to facilitate this. This paper describes such a model for parallelizing serial pipeline processing. The fundamental elements of the model are data objects, metadata, external data, threads, and dependencies. Data objects are units of data processed by a serial processing pipeline. Metadata is data about an object that is accumulated as an object is processed. External data provides configuration and state that is shared amongst processing elements. Threads are units of execution created for each stage in a pipeline. Dependencies define dependencies between threads. Given a threading and dependency model, a design for parallelizing a serial processing pipeline of a network stack can be articulated. Packet processing begins with one of the threads such as the initial thread to process the first protocol layer. Each protocol layer thread parses the corresponding protocol headers and starts a thread to process the next layer. Wait points and resolve points are set in the code paths to handle dependencies between stages. Once processing for all protocol layers has been started, the initial thread waits for all the threads to complete and then performs any necessary serial completion processing.
In some aspects, the techniques described herein relate to a parsing system for parsing protocol headers, the parsing system including one or more computers, one or more storage devices on which are stored instructions that are operable, one or more memory and a parser engine, one or more parse nodes, one or more protocol tables, and one or more parsers, wherein an instance of a parser is included of a set of parse nodes and protocol tables, wherein the one or more parse nodes provide rules for parsing one or more protocol headers and the one or more parse nodes including additional rules for processing a protocol, wherein the one or more protocol tables describe relationships between the one or more parse nodes, wherein the parser engine processes the one or more protocol headers in a data object or packet per the rules of the one or more parse nodes and the one or more protocol tables, wherein to parse the one or more protocol headers, the one or more parse nodes determines a length of the one or more protocol headers being processed and a type of a next protocol header to be processed, wherein the length of the one or more protocol headers is determined by the one or more parse nodes that defines a minimum length attribute to give a minimum length of the one or more protocol headers, and wherein for a variable size protocol header, the one or more parse nodes define a length function that calculates the length of the one or more protocol headers, and wherein the length function includes a value of a length field in the one or more protocol headers as input, wherein the one or more parse nodes define a next type function to determine a type of the next protocol header to process, where the next type function includes a value of a next protocol field in the one or more protocol headers as input, wherein the parser engine uses the type of the next protocol header as input to a lookup in the one or more protocol tables that returns a next parse node or null when there is no next parse node, wherein an offset of the next parse node to process is given by a sum of an offset of a current protocol header being processed and a length of the current protocol header being processed, wherein when processing completes for the one or more parse nodes, the parser engine transitions to process the next parse node, wherein parsing of the data object or the packet is complete when the parser engine determines there is no next parse node to process.
In some aspects, the techniques described herein relate to a parsing system, wherein the parsing system is configured for parsing sub-protocol headers within a protocol header, wherein a sub-protocol defines a list of data elements each of which have one or more data headers, wherein the list of data elements are a Type Length Value list, a set of flag-fields, arrays, or other construct including multiple objects to be parsed, wherein the one or more data headers are parsed in a context of the one or more parse nodes, wherein the one or more parse nodes for the protocol with the sub-protocol includes one or more sub-parse nodes, one or more sub-protocol tables, and rules for parsing the one or more data headers of the sub-protocol, wherein the one or more sub-parse nodes provide rules for processing a data element, wherein the one or more sub-parse nodes define nested sub-protocols, wherein the one or more sub-protocol tables map types of data elements to the one or more sub-parse nodes, wherein the parser engine processes the sub-protocol in the protocol header by parsing and processing each of the data elements in the list of the sub-protocol, wherein to parse the one or more data headers, the one or more parse nodes determine a length and type of a current data header of the one or more data headers being processed, wherein the parser engine uses the type of the one or more data headers as input to a lookup in a sub-protocol table that returns the one or more sub-parse nodes for processing the data element, wherein the offset of a next data element to process is given by the sum of the offset of the one or more data headers being processed and the length of the data being processed, wherein when processing completes for the one or more sub-parse nodes, the parser engine transitions to process a next sub-parse node, wherein parsing of the sub-protocol for the one or more parse nodes is complete when all the data elements have been processed.
In some aspects, the techniques described herein relate to a parsing system, further including a set of parser instructions and one or more parser registers, wherein parser instructions are instructions in an Instruction Set Architecture that perform functions and operations related to parsing, wherein the one or more parser registers includes state variables for parsing, wherein the one or more parser registers are input to and processed by the parser instructions, wherein the parser instructions can be commingled with plain integer instructions, wherein the parsing system has instructions to move data from one or more integer registers to the one or more parser registers, wherein the instructions also move the data from the one or more parser registers to the one or more integer registers.
In some aspects, the techniques described herein relate to a parsing system, further including one or more parser exit codes which are a set of status codes returned when the parser exits, wherein the one or more parser exit codes include a success code and error code for conditions, wherein the one or more parser exit codes are stored in a parser status code register, wherein parser instruction processing cause the parser to exit prematurely, wherein an exit code set in a parser exit status register specifying a reason the parser exited.
In some aspects, the techniques described herein relate to a parsing system, further including state information describing the current protocol header being processed or the current data header being processed, wherein the state information for the current protocol header being processed includes the offset of a first byte of the protocol header being processed relative to a start of the packet and the length of the current protocol header being processed, wherein a current header parser register of the one or more parser registers holds the offset and the length of the current protocol header being processed, wherein a pointer to the current protocol header or the current data header being processed is derived from the offset and a base address pointer for the data object or the packet, wherein the state information for the one or more data headers being processed includes the offset of the first byte of the one or more data headers being processed relative to the start of the packet and the length of the one or more data headers being processed, where in the current header parser register of the one or more parser registers holds the offset and the length of the one or more data headers being processed, wherein a pointer to the one or more data headers being processed is derived from an offset and the base address pointer for the data object or the packet.
In some aspects, the techniques described herein relate to a parsing system, further including limit bounds of parsing, wherein the length of the data object or the packet implies a maximum length of the one or more protocol headers, wherein the length of the data object or the packet is held in a packet length register, wherein when the protocol header with its length exceeds the limit bounds set by the length of the packet or an instruction attempts to access data beyond the limit bounds then the parser will exit on an error, wherein a parse node sets a databound for the sub-protocol that is the maximum length of all the data elements included within the protocol header, wherein when the length of the one or more data headers exceeds the bound limits set by the databound or the instructions attempt to access data beyond the databound in the context of the sub-protocol then the parser exits on the error.
In some aspects, the techniques described herein relate to a parsing system, further including end of node processing that is performed at an end of a node for an instruction sequence, wherein the end of node processing includes checking that parsing is complete, checking a for loop, jump to loop head, exiting loops, jump to next node, and overlay handling, wherein end of node processing first checks a loop register, wherein when the loop register is set to an address a data header offset is advanced by the length of the current data header and then a jump is performed to that address, or wherein when the loop register is a status code indicating an error then the parser exits and reports the error, or wherein when the loop register is set to an okay status code, the loop is not being processed and a next register is checked, wherein when the next register is set to an address, a current header offset is advanced by the length of the current data header, wherein when the one or more parse nodes are marked as an overlay node then the current header offset does not advance, wherein the jump is performed to that address the next register, or wherein when the next register is a status code indicating an error then the parser exits and reports the error, or wherein when the loop register is set to the okay status code then the parser exits normally with the okay status code, wherein a limit is configured for a number of loop iterations and when the limit is exceeded then the loop exits with an error, wherein a limit is configured for the one or more parse nodes to process and when the limit is exceeded then the parser exits with an error.
In some aspects, the techniques described herein relate to a parsing system, wherein the parser instructions are augmented with an end-of-node attributes, wherein once a marked instruction completes its primary processing it executes common end of node processing, wherein the parser instructions set the next register or loop register to be processed by the end of node processing.
In some aspects, the techniques described herein relate to a parsing system, further including loop instructions including basic loops that are defined by a loop head, which sets the loop register with the address, wherein at the end of node processing when the loop register is set an address then the jump is made to the address to process the next loop iteration, wherein in end-of-node processing a loop terminates when the loop register has been set to sub node stop okay or the loop register is set to an error code when an error being encountered during loop processing, wherein an optional jump to post loop processing is allowed.
In some aspects, the techniques described herein relate to a parsing system, wherein an encapsulation level is maintained in the one or more parser registers, wherein when transitioning to a parse node marked as encapsulation in the end of node processing the encapsulation level is incremented, where in when the encapsulation is incremented a pointer to a metadata frame is advanced by the size of the metadata frame, wherein a limit for a number of encapsulations is set and when the limit is exceeded the parser exits with an error, the one or more parser registers include one or more counters that count events and the encapsulation level.
In some aspects, the techniques described herein relate to a parsing system, wherein the one or more parser registers include one or more counters that count events, wherein the parsing system includes an increment counter instruction that increments the one or more counters, wherein a limit is configured for a counter, of wherein when the limit is exceeded then the parser engine takes an action that could be stop the parser, stop the parser with error, exit loop, don't increment counter, wherein counters are automatically reset to zero when parsing commences for the packet or the data object, wherein the counter is optionally configured to be reset when an encapsulation parse node is encountered.
In some aspects, the techniques described herein relate to a parsing system, further including a load from header instruction that loads some number of bytes from the current protocol header or the data being processed into an accumulator register, wherein an attribute of the instructions indicates whether a source is the current protocol header or the one or more data headers, wherein an offset indicates the offset to load from relative to a start of the current data header, wherein an address pointer for the load can be derived by adding the offset to the pointer for the current header or the one or more data headers, wherein the attribute of the instructions indicate a loaded value is to by endian swapped, wherein an optional shift value indicates a number of bits to shift left the loaded value, wherein an optional mask value indicates a number of high order bits in the loaded value to zero, wherein the parsing system checks current header of data header length as part of the load, wherein when the load would access bytes beyond a length limit then parsing system jumps out of the parser on error condition, wherein when the length is acceptable but beyond the current header or data length then extend the loaded value in the one or more parser registers.
In some aspects, the techniques described herein relate to a parsing system, further including a store to metadata instruction that stores some number of bytes from the one or more parser registers or immediate value to metadata memory, wherein source data is sub-register of the one or more parser registers, wherein a target of the store is either common metadata or a metadata frame, wherein the offset indicates the offset to store data relative to start of the common metadata or the metadata frame, wherein a counter register is specified to use as an array index and the counter is configured to be associated with an array element size, wherein the offset into the array is derived by multiplying the value of the counter by the array element size, wherein the offset for storing data is the offset indicated in the instructions plus the offset of the array when the array index is specified, wherein the one or more parser registers include base addresses of the common metadata and the metadata frame so that fully qualified address pointer for a destination is derived by adding the base addresses and a computed store offset.
In some aspects, the techniques described herein relate to a parsing system, further including hardware parser length instructions, to set and check current header length data header length, and databound, wherein the length is derived from an immediate length, a variable field loaded in a sub-register of the one or more parser registers, or a sum of an immediate value and a variable length, wherein when the variable length is set it can be left shifted, wherein once the length is computed it is checked against appropriate bounds, wherein when a bound is exceeded, the parser stops with code depending on whether the length is for the current protocol header or the one or more data headers.
In some aspects, the techniques described herein relate to a parsing system, further including a Content Addressable Memory that is used as a protocol table, wherein each entry is composed of a key and a target value, wherein the Content Addressable Memory used to perform next protocol lookups and can be used for other purposes as well, wherein instruction are used to program entries of the Content Addressable Memory, wherein the Content Addressable Memory lookup instructions perform the lookup on the value in an accumulator sub-register as the key, wherein Content Addressable Memory instructions set returned value in a next register, set the returned value in an accumulator register, or jump directly to a returned address, wherein the Content Addressable Memory instructions indicate a table selector that allows different Content Addressable Memory tables, wherein the different Content Addressable Memory tables are consolidated into a single Content Addressable Memory table by making the table selector to be part of the key, wherein for the single Content Addressable Memory table, where the table selector is deduced by a low order bits program counter to reduce a number of bits needed to express a table identifier in the instructions.
In some aspects, the techniques described herein relate to a parsing system, further including lookup arrays that are used as a protocol table, wherein the lookup arrays are used to perform next protocol lookups and can be used for other purposes as well, wherein instruction are used to program entries of the array, wherein parser array lookup instructions perform a lookup using a value in a sub-register as an index, wherein array lookup instructions set a returned value in a next register, set the returned value in the one or more parser registers, or jump directly to a returned address, where the value includes a base index into a sub-table to consolidated different lookup arrays in into a single array table.
In some aspects, the techniques described herein relate to a parsing system, further including Type Length Value loops that are implemented using a loadtlvloop instruction, which combines a functionality of loading a Type Length Value type from the one or more data headers and serving as a loop head, wherein at each iteration the one or more parser registers is set to an index of a next set flag bit to process, wherein a “jump loop” function performs the lookup and jump in the context of a loop, wherein a “jump TLV loop” function performs the lookup and the jump in the context of a Type Length Value loop.
In some aspects, the techniques described herein relate to a parsing system further including comparison instructions that perform a comparison operation between a value in sub-register of the one or more parser registers and an immediate, wherein a result of the comparison is false then behaviors include one of a following: stop the parser, stop processing the current node, stop processing a current sub-node, of jump to a handler.
In some aspects, the techniques described herein relate to a parsing system further including runthread instructions that requests that work be performed to process a protocol layer in one or more worker threads, wherein a work item indicates a function to run in the one or more worker threads to process a protocol layer and includes the parser state describing the protocol layer to be processed, wherein when a runthread instruction is executed, a snapshot of a material parser state is taken and placed in an allocated work item which is a memory object, wherein the one or more parser registers are overlaid with data of the allocated work item such that taking the snapshot is done by a block copy for the one or more parser registers to an address of the allocated work item in memory, wherein the parser engine sends these messages to a thread scheduler initiate scheduling of the one or more worker threads, wherein the scheduler processes the message and schedules the one or more worker threads to run all the work items in the list, wherein the one or more worker threads thread are scheduled asynchronously and runs in parallel with the parser.
In some aspects, the techniques described herein relate to a parsing system further including data extraction instructions that performs a copy from header data to metadata to perform data extraction, wherein the data extraction instructions encapsulate load and store operations and move more than eight bytes in one instruction, wherein the data extraction instructions invokes pseudo instructions, wherein the pseudo instructions include an index of the instructions in memory, and a total number of the pseudo instructions to execute.
Embodiments of the present disclosure are described in detail below with reference to the following drawings. These and other features, aspects, and advantages of the present disclosure will become better understood with regard to the following description, appended claims, and accompanying drawings. The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations and are not intended to limit the scope of the present disclosure. Also, the drawings included herein are considered by the applicant to be informal.
Protocol parsing is essential in network processing. It can be defined as the operation of inspecting network packets to identify and process their protocol layers, and a protocol parser is then an entity that parses a set of protocols. A protocol parser may be represented as a parse graph that indicates the various protocol layers that may be parsed and the relationships between layers.
The goals of the PANDA parser are: •“Turing complete”—any protocol that can be parsed by a CPU can be parsed by the PANDA parser. •Native support for parsing TLVs and flag-fields. Allow users to code in a language convenient to them and leverage standard tool chains and tools. The same parser source can be compiled to arbitrary software and hardware targets without code change. For any given target environment, provide the highest performance possible given its capabilities. The following paragraphs explain the design of the PANDA parser including core data structures and the parser engine that processes them, the programming model including compilers and an Intermediate Representation for parsers, the hardware implementation that employs Domain Specific [XX] CPU instructions, and a discussion of related work and opportunities.
The PANDA Parser is a facility that implements generic and programmable parsing. A parser instance is defined by a set of data structures as nodes that are linked together by protocol tables to represent the parse graph for a parser. A parser engine parses packets per the rules and attributes of the parse graph. The parser engine is effectively a Finite State Machine (FSM) where the input data are packets, the nodes of the parse graph are the states, and protocol tables define the state transitions. A parse graph for a PANDA parser is specified by a set of parse nodes and protocol tables. Parse nodes describe how to parse the header of a specific protocol and are annotated with ancillary customizable functions to process the results of parsing. Protocol tables define the links between parse nodes. The attributes of a parse node includes a protocol node, reference to a protocol table, metadata extraction rules, and backend processing handlers.
A parse node includes a reference to a protocol node that provides the standard rules for parsing a protocol. Parsing a protocol requires two fundamental pieces of information: 1) the length of the protocol header (and hence the offset of the next header), and 2) the type of the next header protocol for non-leaf protocols.
A protocol node defines the min_len attribute to give the minimum length of a header; this must be non-zero. For variable length protocols, a len function is specified that returns the length of a protocol header. A length function can be specified by a parameterized function
Some variable length protocols lack an explicit header length field. For instance, in GRE [XX], header length is determined by summing the sizes of flag-fields. A FLAG_FIELDS_LENGTH function can be defined for this that takes flags and a flags descriptor table as input.
The READ function reads data from the header (HDR). pfield-off is the offset of the length field, and pfield-len is the size of the length field in bytes. pmask is a mask, pshift-right is number of bits to shift right, pmultiplier is a multiplier value, and padd is a value added to the final result. An instance is given by <pfield-off, pfields-len, pmask, pshift-right, pmultiplier, padd>. For example, the function to compute the IPv4 header length is denoted by HLEN<0, 1, 0xf, 0, 4, 0>, and the function to compute the length of an IPv6 Hop-by-Hop Options Extension Header is denoted by HLEN<1, 1, 0xff, 0, 8, 8>.
Protocol nodes for non-leaf protocols specify a function to extract a protocol number from a field in a packet. This can be defined as a parameterized function (where the input parameters have same semantics as described above): PTYPE<pfield-off, pfield-len, pmask, pshift_right>=(READ (HDR, pfield-off, pfield-len) & pmask)>>pshift-right An instance is given by <pfield-off, pfields-len, pmask, pshift-right>>. For example, the function to derive the next protocol from an IPv4 header is denoted by PTYPE<9, 1, 0xff, 0>, and the function to derive the next header for an IPv6 Hop-by-Hop Options Header is denoted by PTYPE<0, 1, 0xff, 0>. 2.1.3 Other protocol node properties A protocol node contains additional optional attributes. The encap attribute is a boolean indicating that the protocol is a network encapsulation; this indicates that frame index (Section 2.3) is incremented when proceeding to the next protocol. The overlay attribute is a boolean indicating that the protocol is an overlay; this indicates that cur_ptr (Section 2.7) does not advance when proceeding to the next protocol. The check_fields attribute refers to a list of conditional expressions for validating a protocol.
Protocol tables provide the links between parse nodes in a parse graph. A protocol table maps a protocol number to a parse node. Protocol tables may be implemented as arrays, hash tables, or CAMs (Content Addressable Memory). Metadata consists of data about packets that is collected and recorded as a packet is parsed. Metadata includes fields that are extracted from protocol headers, and also ancillary information such as header length or encapsulation level. Parse nodes may be annotated with rules to “extract” metadata and record it in a metadata buffer. Metadata is consumed by protocol handlers and post parser processing.
A metadata buffer is composed of two sections: common metadata and metadata frames (
A parse node may specify functions to perform backend processing of a protocol layer. These functions are invoked inline by the parser and the arguments to the function are a pointer to the protocol header, the length of the header, and a pointer to the metadata buffer for the packet. Handler functions can perform arbitrary protocol processing and they can run in parallel with the parser.
The PANDA Parser supports parsing Type Length Values (TLVs). TLVs parse nodes and TLVs protocol nodes extend parse nodes and protocol nodes with attributes for parsing TLVs.
The parser engine in the PANDA parser performs the work of parsing protocols in packets. The processing flow of the parser engine is illustrated in
The parser engine uses some variables for parsing TLVs (and flags-fields): data_off is the offset (from the beginning of the packet) of the current TLV being parsed, data_len is the computed length of the current TLV, and data_bnd is maximum extent of the TLVs.
The programming flow of the PANDA parser is depicted in
The following illustrates CPR by an example: the json below describes a parser for Ethernet, IPv4, TCP with options, and GRE with flag-fields. The parsers property declares my_parser with root node eth_node (not shown) and okay-target indicates that the okay node (not shown) is invoked when the parser completes. Parse nodes are defined in parse-nodes. The protocol table for the eth_node parse node includes an entry that maps 0x800 EtherType to ipv4_node. In ipv4_node, min-hdr-len indicates the minimum header length (20 bytes for IPv4). hdr-length sets parameters for the IPv4 header length function, and next-proto gives the parameters for the next header type function. The ents sub-field of next-proto is an inlined protocol table that matches TCP and GRE to ipv4_node and gre_node The tlvs-parse-node in tcp_node provides the rules and attributes for parsing TCP options. tlv-type and tlv-length provide the function parameters to determine the type and length of a TCP option (the minimum TLV length is inferred). The starting offset of TLVs is taken to be the minimum header length (20 bytes). tcp_opt_table maps option type 8 to tcp_opt_tstamp_node, and that node records the timestamp value in the metadata. flag-fields-parse-node in gre_node gives the rules for parsing GRE flag-fields. For each possible GRE flag-filed there is an entry in gre_flags_table that specifies the flag value, mask, and field size. Flag value 0x4000, the KeyId flag, is mapped to gre_key_id node.
The code may be illustrated below
The PANDA parser ecosystem. A parser definition is coded by a user in some frontend language (shown on the left); The parser may be part of a larger data path program. The panda-compiler front-end compiles the user's code into the Common Parser Representation IR (shown in the middle), and non-parser code is compiled into a suitable IR like LLVM IR for a C++ program. A backend compiler then compiles both the parser IR and non-parser IR into an optimized binary executable for the desired hardware or software targets (shown on the right).
PANDA-C is an API and library to program parsers in C. A parser is specified by a set of C data structures for protocol nodes, parse nodes, and protocol tables.
The PANDA parser compiler infrastructure consists of two phases: 1) A compiler frontend that converts parser source code or other representations of parsers to Common Parser Representation, and 2) A compiler backend that transforms CPR into executable images for different backend targets.
As shown in
The backend can be divided into two phases, this division allows flexibility and gives more optimization options to be applied at the right abstraction level for better performance. The backend starts with a simple conversion from CPR, a declarative intermediate representation, to an MLIR representation, with a dialect defined specifically to define parsers as a higher abstraction concept. This allows the MLIR output to be used for augmentation by other tools, such as debugging injection and optimization passes that can happen at a conceptual level of the parser definition. The second phase of the backend compiler reads the MLIR representation, applies optimization passes and translates MLIR to LLVM IR. If the target supports parser instructions, then the compiler will translate the high-level MLIR parser dialect into parser intrinsics (internal functions defined in LLVM). The LLVM backend will generate parser instructions from the intrinsics when it compiles the LLVM IR translated code.
The PANDA Parser is implemented in hardware as an Instruction Set Architecture (ISA) extension for RISC-V CPUs [XX]. The extension defines a set of domain specific parser instructions and a register file with parser registers that parser instructions act upon. Parser instructions are highly optimized for performance, and Section 5.2 provides a performance evaluation of parser instructions. Parser instructions are prefixed by prs. in assembly. There are thirty-two sixty-four-bit parser registers. Their logical names are in italics with the first letter capitalized in this paper, and their names in assembly start with a ‘p’ and are denoted in bold lowercase. Parser registers may have sub-fields that are denoted by Reg.Field. The full specification of PANDA Parser instructions is in [XX]. In this section we provide an overview and example assembly.
There are several classes of parser instructions. •Move instructions: prs.mv moves a parser register to another parser register, prs.mv.x.p moves an integer register to a parser register, prs.mv.p.x moves a parser register to an integer register. •Load instructions prs.load* loads a header field at some offset from the current header (pcurptr) or data header (pdatptr). Variants allow loading one, two, four or eight bytes. •Length instructions prs.lenset* set the length of the current or data header and perform a bounds check. There is a variant to compute the length function value. •Lookup instructions perform protocol number lookup to return the next node. prs.cam* does a CAM lookup, prs.arr* does an array lookup. •Loop ins. prs.loadtlvloop* and prs.*loop, support TLVs and flag-fields loops. •Store instructions store data in a metadata buffer. The destination can be common metadata or a frame. •Compare instructions (prs.cmpi*) compare values in the accumulator to immediate values. •Run thread instruction. prs.runthread schedules an external thread to perform backend processing.
Parser instructions are used to manifest the parser engine. Parse nodes are processed as a sequence of instructions similar to a function, however, instead of being terminated by a return instruction, a .stp instruction indicates the “end-of-node”. The Next register indicates the action to take at end-of node. If Next is not NULL then it contains the address of the next parse node to process, else if Next is NULL that indicates parsing is complete. CAM and array lookup instructions are used to lookup the next protocol and set the Next register.
Parsing state is mainly contained in a few parser registers. CurHdr.Offset and CurHdr. Len give offset and length of the current header. DataHdr.Offset and DataHdr. Len give the offset and length of the data header. PktLen holds the length of the packet, and DataBound holds the maximum length for data.
A program may perform a lookup on the TLV type or flag-field and process the data item by invoking a TLV or flag-field parse node (similar to calling normal a function). .stp instructions indicate the last instruction in a node or loop iteration. Common end of node processing is done to jump to the next node, continue a loop, or exit when parsing is complete (jump to okay-target for instance). The metadata layout is given in the ParserConfig register, and FrameOff.Offset contains the offset of the current metadata frame (
Configuration registers are used to set various properties and limits. These include ParserConfig, LoopSpec, TlvSpec, Counter*Config,
The example program below implements the parser defined in CPR in Section 3.1.
my_parser:
eth_node:
1 prs.load.h paccum, pcurptr+12
2 prs.cam.h.stp pnext, paccum[0], 1
ipv4_node:
3 prs.load.b paccum, pcurptr
4 prs.lensetmin.n pcurhdr, paccum[1], 4:20
5 prs.load.b paccum, pcurptr+9
6 prs.cam.b.stp pnext, paccum, 2
tcp_node:
7 prs.load.b paccum, pcurptr+12
8 prs.lensetmin.n pcurhdr, paccum[0], 4:20
9 prs.tlvfastloop pdathdr, pdatptr, 1:0
10 prs.cmpnei.h.stopsub paccum[1], 0xA08
11 prs.load.w paccum, pdatptr+2
12 prs.store.w.stp pmdbase+4, paccum
gre_node:
13 prs.load.w paccum, pcurptr
14 prs.cam.h pnext, paccum[1], 1
15 prs.andmask.b paccum[0], 0xF0
16 prs.flagsloop.rev pflags, paccum, paccum
17 prs.camjumploop.b.stp paccum[0], 3
gre_key_id:
18 prs.storereg pmdbase+8, dathdr
gre_flgfld_node
19 prs.lenset.stp pdathdr, 4
okay:
20 prs.runthread.stp 17
The CAM table would be programmed as below. Note the lookup key has two parts: the protocol number to match in the low order 16 bits, and a table identifier in the high order 4 bits. When a lookup is done the full key needs to be exactly matched—this allows one CAM table for all uses.
Before executing this sequence, a few configuration registers would be set. TLVSpec would be configured for TCP options. Okay Target would be set to the address of okay. Thread #17 refers to a backend processing function when the parser exits with an “okay” status. Parsing commences at the root node which is ether_node. Lines #1-2 load and lookup the Ethertype in CAM from sub-table 1; the load implicitly sets and checks the header length (14 for Ethernet). The .stp qualifier in Line #2 indicates the “end of node”, and a jump is made to the Next node set by CAM lookup. In ipv4_node, Lines #3-4 compute and check the IPv4 header length—this implements the HLEN function for IPv4
Lines #5-6 look up the next protocol node, and a jump is made to the Next node set by CAM lookup since Line #6 is a .stp instruction. In tcp_node, lines #7-8 compute and check the TCP header length (implements the HLEN function for TCP). Lines #9-12 implement a TLV loop to parse TCP options per the processing in
Line #10 compares the option type to 8 and the option length to 10 (type and length of the Timestamp option), if there is a not match then processing loops back to Line #9, else if there is a match then lines #11-12 record the timestamp in metadata. The .stp at line #12 indicates the end of the loop iteration and the thread loops back to line #9. When all options have been processed the parser exits at the prs.tlvfastloop instruction in line #9 since TCP is a leaf node. In gre_node, line #13 loads the base four GRE byte header which implicitly sets and checks the minimum length. Line #14 looks up the EtherType and sets the result in Next. Line #15 prepares the GRE flags by masking them, and lines #16-17 perform a flag-fields loop (per processing in
System may include a hardware parser in a SemiDynamics Avispado RISC-V CPU in a Xilinx U200 FPGA [XX]. The design is shown in
As a baseline for performance measurement, we ported the Linux kernel flow dissector to user space. Flow dissector [xx] is a software parser in C code that performs protocol parsing and metadata extraction in a similar manner as the PANDA parser. We implemented an equivalent parser with the PANDA parser that supports the same protocols and performs the same metadata extraction. The PANDA parser instance is compiled into user space C code, we employ a common test program to inject packets into both parsers, and all C code is compiled with the same options to make an “apples to apples” comparison.
The system extrapolates to make performance projections for parser instructions in CPU hardware. A single parser instruction replaces between five and three hundred standard RISC-V integer instructions, with an expected average ratio of parser instructions to equivalent integer instructions to be about 1:15 and expected instructions Per Cycle, or IPC, for parser instructions to be about 0.4 on average, and an IPC of 1.4 for integer instructions.
A good example of a well featured software parser is the Linux kernel flow_dissector [XX]. Flow dissector is a kernel function that parses packets to extract metadata and is used in various places including Receive Packet Steering [xx] and TC Flower [xx]. Flow dissector parse many protocols including several of those in
The PANDA Parser provides an alternative that addresses the issues. It is proposed that a PANDA Parser compiled into eBPF could replace the Linux flow dissector [XX]. As discussed in section 5.1, the PANDA parser has better performance than flow dissector. The PANDA parser is also offloadable due to the Common Parser Representation that decouples the frontend language from backend targets. 6.2 P4 and the PANDA Parser Programming Protocol-Independent Packet Processors, or P4, is a high-level language and hardware environment for packet processing. P4 includes a parser that is programmed in the P4 language. While P4 has made inroads in datapath programmability, it has several drawbacks that impede adoption. P4 was originally designed for network routers that are concerned with a limited set of protocols, whereas host networking requires a wider range of protocol support, including support for TLVs and flag-fields that P4 naively lacks. P4 intertwines programming language with hardware so it is difficult to support new backend targets, or use alternative frontend languages. The biggest impediment is the use of a Domain Specific Language replete with its own build tool chain and debug tools—these tend be unfamiliar to programmers and have a steep learning curve resulting in high development and maintenance costs.
The PANDA Parser can productively augment P4. P4 could be compiled to the Common Parser Representation IR, and the CPR representation could be compiled to P4 hardware, thereby facilitating flexibility at both the frontend and the backend. If a user is already writing programs in P4, the model expands the set of potential hardware targets. Similarly, if a user has P4 hardware they could program it using alternative languages of their choice.
The SiPanda Parser is a domain specific hardware parser for parsing serial data headers such as network packets. The SiPanda Parser builds on top of the base SiPanda architecture and leverages the program flow it defines.
There are two variations of the SiPanda Parser: the first uses 32-bit RISC-V custom instructions mapped into the custom-0 primary opcode (0xb). The second uses 64-bit RISC-V custom instructions using the opcode space defined for instructions larger than 32-bits. This specification describes the 32-bit hardware parser instructions; the 64-bit variant will be specified in a companion document.
This specification introduces a new register file to RISC-V denoted “parser registers” or just p registers. An instruction to move data from an integer register to a parser register, and one to move data from a parser register to an integer register are defined per the coprocessor specification. Coprocessor instructions use custom-3 opcode (0x7b), and the parser specific coprocessor instructions are denoted by cpreg equal to zero.
This specification covers four topics: new parser registers, memory model, alignment, CAM, and helper macros, the normative description of the parser instructions including the instruction format, semantics, pseudo code, and assembly for the instructions, background information about the PANDA parser, mapping to hardware instructions, example of key parser parameters in action, pseudo data extraction instructions, and a sample program with disassembly, and description of the interaction between the Parser and SPDU. This includes the parser event loop, receiving “start parser” messages from the cluster frontend, and mechanisms to request scheduling of worker threads, and sending messages to the cluster scheduler to start a thread set for processing a PDU
Pseudo code describes instruction semantics in courier font. Hardware names for parser registers have the first letter capitalized and are in bold; for example: Accum. Assembler ABI names for parser registers are lower case, start with a ‘p’ and are printed in italics; for example, paccum. Temporary variables in pseudocode are prefixed by “Temp”. Field names in instructions are capitalized and italicized, for example: Address.
Hardware instructions are denoted by all capital letters, in bold and italics, and prefixed with ‘P’; for example PSTORE. The mnemonics for assembler instructions and any fixed fields for instructions are in bold typeface and variable arguments for assembly instructions are in italics and enclosed by < > brackets; for example:
prs.loadsb paccum, pcurptr+<offset>, <blen>:<shift>
A one character selector in assembly descriptions is denoted by a set of characters enclosed by [ ] brackets—for example, prs.loadsb.[bhw] pflags, pdatptr+<offset>, <blen>:<shift> indicates [bhw] is replaced by b, h, or w. Optional components of an instruction are enclosed by { }brackets—for example prs.loadsb pflags, pdatptr+<offset>, <blen>{:<shift>} indicates that :<shift> is optionally present.
Some registers have a structure containing some number of bit fields. In pseudo code fields of structured registers are denoted by <register>.<field>; for example, LoopSpec.MaxNon refers to the half-word at bits 16 to 31 of register LoopSpec. Register fields can be read or written in pseudocode where the appropriate bit operations are performed on the fields.
An example read operation could be denoted:
which is equivalent to:
And an example store operation might be denoted:
which is equivalent to:
Common macros and functions may take some number of arguments that are used in the pseudo code. The notation for a macro argument is _ARG_Name, and if the argument is derived from an instruction field then the notation is _ARG_Name_. For example, the CAM lookup macro has logical prototype:
CommonCAMLookup(_ARG_Value_, _ARG_Sz_, _ARG_Pos_, _ARG_F_,
_ARG_Share_)
where the arguments for _ARG_Sz_, _ARG_Pos_, _ARG_F_, and _ARG_Share_ are derived from the Sz, Pos, F, and Share fields in a CAM instruction.
In macros, ## is used to represent token substitution from arguments in variable names (this is similar to use of ## in the C preprocessor). For example, if a macro is invoked with the _ARG_Cntr_ argument set to 3 then the register field Counters.Cntr ##_ARG_Cntr_ would be Counters.Cntr3 after the substitution.
The parser works on data being delivered as a stream or serial data. It is assumed that this data will not be modified while the parser is working on it. This means that the parser can maintain a buffer of incoming streaming data with no coherency checking.
The parser's 32-bit instructions can only target 4-byte aligned targets. If 16-bit instructions are supported and being mixed in, then a 16-bit NOP may be required to make sure the target of any parser instruction is 4-byte aligned. Note that in an assembler, .balign 4 may be used to align 32-bit parser instructions.
The 64-bit instructions must be 8-byte aligned, which means that if they are mixed with 32-bit instructions a NOP may be required to meet alignment requirements. The target of 64-bit instructions are also 8-byte aligned. Note that in an assembler, .balign 8 may be used to align 64-bit parser instructions. 64-bit instructions and 32-bit instructions can branch to each other and fall through to each other in execution as long as alignment rules are followed.
Addresses are assumed to be sixty-four bits, including any PC targets, pointers to the packet payload in external memory, pointers to the metadata and parsing buffer header data, and pointers to other memory used by the parser such as the lookup array.
The CAM returns instruction addresses as relative offsets. These instruction base relative addresses are encoded in twenty-four bit values. For example, a fully qualified absolute address is derived from a twenty-four bit offset with the canonical base address for the parser as: ParserInstrBase|4*<24-bit address>
PC relative addresses, such as that expressed in the PNEXTNODE instruction, are encoded as sixteen bit values. A fully qualified absolute address is derived as:
The PLOAD instructions load one byte, one halfword (two bytes), one word (four bytes), or one double word (eight bytes) into a parser register. The source memory is the “packet buffer” which has a base address in PktHdrBase. This memory is the only memory read by parser instructions and is not written (i.e. it is read-only memory from the parser instructions' perspective).
The PSTORE instructions store one byte, one halfword (two bytes), one word (four bytes), or one double word (eight bytes) from a parser register or immediate value. The destination memory is a “metadata frame” which has a base address derived by adding MetadataBase and 4*FrameOffFnunSeqno.FrameOffset register values, or the “common metadata” (general metadata for the whole object being processed) which has a base address in MetadataBase. This memory is the only memory written to by parser instructions and is not read (i.e. it is write-only memory from the parser instructions perspective).
Sub-registers allow referencing byte, nibble, and word components of a register explicitly in instructions. Several parser instructions use sub-registers. There are two parameters to describe a sub-register: size and position. Size and position are expressed in the assembly for an instruction and set the Sz and Pos fields in the instruction code.
Assembly Instructions using sub-register operands are annotated with size and position information. In an instruction mnemonic definition, size is indicated by [nbhw] and position is indicated by <reg>[<pos>]. For example, the mnemonic format for the prs.lenset instruction is:
and an example use might be:
which has the effect of computing the length of the current header based on byte number six in the Accum register.
If {[<pos>] is in the menonomic format and [<pos>] is not present in an instruction, then the sub-register position is taken to be zero. For instance:
prs.lenset.b pfcurhdr, paccum is equivalent to prs.lenset.b pcurhdr, paccum[0]
For most instructions that use sub-registers, the Sz field corresponds to 0 for nibbles, 1 for bytes, 2 for half-words, and 3 for words (shown in the second to last column of the above table); such that the number of bits in the sub-register value is:
For parser load and store instructions, the Sz field corresponds to 1 for bytes, 2 for half-word, 3 for word, and 0 for double word (shown in the last column of the above table); such that the number of bits in the sub-register value is:
The instructions that use the alternate meaning for Sz==0 are denoted as such below.
The position of a sub-register indicates the position of the nibble, byte, half-word or word. Sub-registers are counted from the first byte in memory being the zero position (low order byte in little endian word). Nibbles are numbered such that the high order four bits in a byte are a lowered number nibble than the four low order bits of a byte; e.g. nibble number zero is the four high order bits of the first byte, and nibble number one is the low order four bits of the first byte.
A side effect of several parser instructions is that they may set the PC to perform a jump. The most common jumps occur at the end of a node when the stop bit (S-bit) is set in an instruction, and jumping to a handler returned by a CAM lookup. The stop bit processing is described in the “Common_End_of_Node” section below. The other cases of jumps are for exception and error handling. Note that there are no CPU generated traps or interrupts defined in this architecture.
The hardware parser assumes the data for parsing is streamed into cluster and CPU local memory using the data streaming mechanism defined by the SiPanda Base Architecture. The headers will be in the region of memory defined by the system with some base address. The PktHdrBase register contains a pointer to the base address for the packet headers of one packet in the stream receiving memory region, and the PktLen register contains the length of the whole packet in PktLen.AllLen, and the length of the packet headers in PktLen.ParseLen. As data streams in, these values are monotonically increasing until all the data is received or the limit of the parser buffer is reached in the case of PktLen.ParseLen. PktLen.F is a flag indicating that whole packet is received and PktLen.AllLen is at its final value, PktLen.P is a flag indicating that either whole packet is received of the size of the parsing buffer has been received (ParserConfig.PrsBuff) and PktLen.ParseLen is at its final value.
In the current design of the parser, it is assumed that packets are received in their entirety so when the parser runs PktLen.ParseLen and PktLen.AllLen are set to their final values and PktLen.F and PktLen.P are set.
The headers buffer, metadata block, and work item that the cluster front end sends to the parser constitute the packet state necessary for parsing. A parsing header buffer contains the headers of a packet for parsing. The size of an allocated buffer is in ParserConfig.PrsBuff (the real byte size is (ParserConfig.PrsBuff+1)*64). Metadata is any information that is derived from a packet as it is parsed and the data is saved in a “metadata block” for consumption by down stream processing. Metadata blocks are allocated by the cluster front end from shared cluster memory with some base address. The allocated size of a metadata block is (note rounding up to sixty-four bytes): ((4*ParserConfig.FrameOffset+4*(ParserConfig.FrameSize+1)+63)/64)*64.
A high performance CAM is integrated into the CPU and is used for protocol number lookups to determine the next node, TLV type lookups for processing TLVs, flags lookup for processing flags, and general CAM lookup to load a value into Accum. A CAM entry has a 20-bit key and a 32-bit target. The target may be an encoded address or a parser code. The CAM Key is structured in one of two ways as indicated by
If the high order for bits of the key are non-zero, then the key is for a shared table and the Shared structure in the above union for the key is used. The Shared field indicates one of fifteen tables numbered one through fifteen, and the Match field is the primary field to be matched which can be up to sixteen bits in length (for instance this could be an 8-bit or 16 bit protocol field such as an IP protocol number or EtherType respectively). Shared tables are used for common lookups in different protocol nodes; for instance the lookup for EtherType might be shared between the root Ethernet node and a node for GRE encapsulated Ethernet. Also, if the protocol lookup requires more than eight bits to match then a shared table is used.
If the four high order bits of the key are zero, the Selector field is used to select a 8-bit logical, non-shared, CAM sub-table. The Match field is the primary field to be matched which can be up to eight bits in length (this could be an 8-bit protocol field such as an IP protocol number). The selector for a non-shared table is derived from the PC of the instruction for a CAM lookup as:
The selector for two different non-shared tables must be unique. If the PC derived selector for two different tables is identical, meaning the addresses of the respective instructions invoking the CAM are equal in the second through the ninth bit, then this is a non-shared keyed collision and it is not allowed. If a non-shared collision occurs then one mitigation is to insert nop's before the second instruction to increase the value of the PC and selector.
When the target of a CAM entry is an address, for instance the address of the next node instruction, then it is commonly formatted as
When the target of a CAM entry is a code, it is formatted as
Code is a seven bit code as defined in the Parser Codes section below. The E, V, NE, NV have the same meaning as described above. Maintaining the set of control bits when a code is conveyed allows setting control bits before an address is determined. If the code is returned to the caller, e.g. being set in ParserExitCode.Error, the control bits are filled in with 1's so that the whole value is a number between −1 and −127 (i.e. the code is a negative value sign extended to the width of the data type in use).
In addition to the CAM, a high performance lookup array is integrated into the CPU and is used for protocol number lookups to determine the next node, TLV type lookups for processing TLVs, flags lookup for processing flags, and general Array lookup to load a value into Accum. Array lookups are appropriate where the key value space is a small number of bits (about one to eight bits). The advantage of an array over a CAM is that an array is a simple indexed memory lookup; the downside is that all possible index values need to be set in the array.
The lookup array is an array of 32-bit values. The array value may be an encoded address or a parser code. The encoding of an address or code in the target is the same as the encodings described for the CAM above. A single lookup array can hold multiple sub-arrays for different uses. A sub-array is identified by a base index and the number of entries in the sub-array. There is no concept of an “array miss” so all possible values in the index range for a table must be set. The default array value is PANDA_STOP_OKAY code.
The Hardware Parser defines a new set of 64-bit registers, referred to as p regs. In assembly these registers are preceded by ‘p’ as illustrated in
Several parser registers employ an encoding to contain either an address or a parser code. The base encoding is thirty-two bits where the high order bit, bit 31, indicates an address or code is encoded. When bit 31 is zero a twenty-four bit relative address is encoded, and when bit 31 is one a code is encoded. To represent the encoding in a sixty-four bit value, bit 31 is signed extended (PANDA parser codes are negative values −1 to −128 such that a code can be cast as a 16 bit, 32 bit, or 64 bit value by simple sign extension of the code in a signed byte). This is illustrated in
ObjectRef (p0, pobjrej) This register holds a fully qualified sixty-four bit opaque object reference (typically a pointer to the PvBuf in external memory for the current PDU). This value is initialized when the parser starts a new packet and is not changed as the packet is parsed.
CurHdr (p1, pcurhdr This register holds the current header offset from the beginning of the packet for the current node being processed, and the header length of the current node being processed Offset, as illustrated in
Length is the length of the current header. PktHdrBase plus CurHdr.Offset plus CurHdr.Length gives a pointer to the next header following the current header
DataHdr (p2, pdathdr). This register holds the data offset from the beginning of the packet for the current node being processed, and the data length of the current node being processed (e.g. the offset is the offset of a TLV and length is the length of the TLV).
As illustrated in
PktLen (p3, ppktlen) Length of the packet. This encodes both the parse buffer length (length of header data the parse can process) and also the length of the whole PDU. This is illustrated in
FrameOffFnumSeqno (p4, pfofnsq)
This register holds the offset of the current metadata frame, the sequence number of the packet, and the function number to run. To derive a pointer to the current frame add MetadataBase and FrameOffFnumSeqno.FrameOffset. The sequence number is only set in the work item and not used operationally by the parser as illustrated in
FrameOffset is the byte offset divided by four of the current metadata frame from the beginning of the metadata for the packet being processed. To derive a pointer to the current frame add MetadataBase and 4*FrameOffFnumSeqno.FrameOffset
FuncNum is the function number to run in a worker thread. This is set by prs.runthread instruction before sending the work item to the cluster scheduler.
Seqno is the sequence number assigned to the packet by the dispatcher
The example diagram in
In this example there is some space reserved for meta metadata which contains generic metadata for the whole packet, and three metadata frames of some configured size. MetadataBase plus 4*FrameOffFnumSeqno.Offset points to the second frame which indicates that the parser is currently processing the first level of an encapsulation and hence Counters.Encap is currently set to one.
PktInfo (p5, ppktinf)
General packet information for a created work item. PktCtx is used to initialize PktHdrBase and MetadataBase. These values may be set by the parser, but otherwise are not operationally used in parsing. This is illustrated in
PktCtx is set by the cluster frontend. This refers to the allocated packet state for the packet being parsed. The value is used as an index in the packet header base memory and metadata base memory to get the header parsing buffer (where the header data is) and the metadata block, which are respectively PktHdrBase and MetadataBase. When the parser starts parsing a packet, these are initialized from PktInfo.PktCtx as:
PktHdrBase=SysHeadersBase( )+(PktInfo.PktCtx*size_of_parsing buffer)
MetadataBase=SysMetadatBaseo+(PktInfo.PktCtx*size_of_metdata block) where SysHeadersBase( ) returns the base address of header buffers, and MetadataBase returns the base address metadata blocks (see description of SysHeadersBase( ) and SysMetadataBase( ) in “Helper macros and functions” section below.
Checksum is the packet checksum computed at ingress
NextWorkItem is the next work item index in a list of work items. This is used by the clustr scheduler and not the parser
IFID is the ingress interface identifier. This is set by the dispatcher and passed to the cluster scheduler in work items. The parser does not process this field otherwise.
L: Last thread in the thread set. The parser set this in the work item for the last thread requested for a packet (i.e. the last instance of prs.runthread for a packet)
N: Indicates that the worker thread cannot be killed (that is, it is run to completion and impervious to the kill threads signal). This is set by prs.runthread.nokill
D: Indicates that the current header is a data header when set, and a current header if not set. This is used by worker threads to compute the pointer to the header to be processed
NodeLoopCnt (p6, pndlcnt)
Holds the running node count and various counters for iterating in a loop as illustrated in
NumLoops counts all iterations of a loop; it works in conjunction with LoopSpec.MaxCnt to enforce a limit on the number of iterations through a loop
NonPadCnt counts the number of non-padding TLVs encountered when processing a TLV loop. This works in conjunction with LoopSpec.MaxNon to enforce a limit on the number of non-padding TLVs to process
PadLen counts the number of consecutive bytes of padding encountered in a TLV loop; this works in conjunction with LoopSpec.MaxPlen to enforce a limit on the number of consecutive bytes of padding in a TLV loop
ConPad counts the number of consecutive padding encountered options in a TLV loop; this works in conjunction with LoopSpec.MaxCPad to enforce a limit on the number of consecutive bytes of padding in a TLV loop
NodeCnt counts the number of nodes encountered in the current parse walk; this works in conjunction with ParserConfig.MaxNodes to enforce a limit on the number of nodes processed in a parse walk
Counters (p7, pcount) This register contains user defined counters for the current parse walk. This includes the encapsulation level and parser counters (Cntr1-Cntr7) as illustrated in
Encap contains the current encapsulation layer. For each protocol encapsulation encountered, this value is incremented. This works in conjunction with ParserConfig.MaxEncap to limit the number of encapsulation levels processed.
Cntr1, Cntr2, Cntr3, Cntr4, Cntr5, Cntr6, and Cntr7 are user defined counters. These counters are incremented by the prs.inc.cntr instructions. These work in conjunction with CounterLimitsConfig.Cntr* to limit the counters. These counters may be used as an array index in the prs.store and prs.storereg instructions, and these work in conjunction with CounterArrayConfig.Cntr* to limit the number of elements that can be indexed in an array.
PktHdrBase (p8, phdrbas) This register holds a fully qualified sixty-four bit base address of packet headers for the current packet being processed. Basically, this is a pointer to the first byte of the first packet header. This value is initialized when the parser starts a new packet and is not changed as the packet is parsed.
MetadataBase (p9, pmdbase)
This register holds a fully qualified sixty-four bit base address of the metadata block for the packet being processed. The metadata block is composed for the “common metadata” followed by an array of metadata frames; see diagram below. This value is initialized when the parser starts a new packet and is not changed as the packet is parsed.
ParserInstrBase (p10, pinbase)
The 64-bit fully qualified base address for parser code, this is a 64M aligned address. That is: ParserInstrBase & 0x3FFFFFF==0.
Next (p11, pnext)
The next node in the parse graph that the parser should go to at the end of this node. This register contains an address/code encoded value.
The fully qualified address is derived by:
if (!IS_RET_CODE(NextNode))
TempAddress=ParserInstrBase|(NextNode & 0xFFFFFF)
PendingWork (p12, ppendwk).
This register holds the index of the pending work for prs.runthread, as illustrated in
DataBndLoop (p13, pdbndlp.
This register holds the data bound which is the maximum length allowed for data in subnodes; and the address of the first instruction of a loop or a code to terminate a loop, as illustrated in
DataBound is the databound length. Initially, this value is set to infinity (0xFFFFFFFF). As the parser processes data headers, like TLVs, this register is updated accordingly
Loop is the beginning of an iterative loop for processing flags or TLVs. When a loop is executing this register holds the address of the first instruction for a loop, or a code to terminate the loop. This register contains an address/code encoded value. The default value, meaning not in loop execution, is the OKAY_RET code. Node and code encodings are illustrated in
The fully qualified address is derived by:
if (!IS_RET_CODE(DataBndLoop.Loop))
TempAddress=ParserInstrBase|
(DataBndLoop.Loop & 0xFFFFFF)
ParserExitCode (p14, pexcode)
This register holds the exit code for the parser when it exits. This register contains a parser code, see “Parser Codes” table, and the address of the parser instruction where the parser exited, as illustrated in
Address is the address offset of the instruction that caused the parser to exit relative to ParserInstrBase. The sixty-four bit address for the instruction can be derived by: TempAddress=ParserInstrBase|(4*ParserExitCode.Address) Error is a parser exit code, this will be a 16-bit representation of a value from the “Parser Codes”.
Accumulator register for working values.
Register for holding the flags being processed in a flags loop. The register also serves a second accumulator in some instructions
ParserConfig (p17, pconfig)
Register containing parameters for parser configuration, as illustrated in
MaxNodes is the maximum number of nodes to visit. This works in conjunction with NodeLoopCnt.NodeCnt to enforce a limit
MaxEncap is the maximum number of encapsulation levels. This works in conjunction with Counters.Encap to enforce a limit
MaxFrames is the maximum number of metadata frames
FrameSize specifies the frame size that is calculated by: RealFrameSize=4*(ParserConfig.FrameSize+1)
FrameOffset specifies the byte offset of the first metadata frame from MetaDataBase. Offset is calculated by: RealFrameOffset=4*MetaDataBase
EE: Bit flag that when set indicates that if the maximum number of encapsulations is exceeded then it is an error
EO: Bit flag that when set indicates that the last metadata frame is overwritten when the encapsulation level exceeds the maximum number of frames. If the bit is not set, stores to metadata when the encapsulation level exceeds the maximum number of frames have no effect.
NumPfuncs: Number of encapsulation functions in the ParserFuncs array
PrsBuff indicates the size of the parsing buffer in units of sixty-four bytes. The size of the parsing buffer is (PrsBuff+1)*64. Note that PrsBuff may be set by hardware and so this field could be read only
CounterLimitsConfig (pcntlim)
Configuration for maximum counter values. This contains the maximum value for each of the seven user counters and an indication for each counter as to whether it is an error when the counter value exceeds the maximum value, as illustrated in
CounterArrayConfig (pctarcf) Configuration for maximum counter index values. This contains the maximum value for each of the seven user counters when they are used to index an array in prs.store instructions. There is also an indication for each counter as to whether the last element of an array should be overwritten when a counter exceeds the maximum array index value, as illustrated in
O1, O2, O3, O4, O5, O6, and O7 indicate that if the respective counter exceeds the maximum array index value then the last element in the array is overwritten.
Cntr1, Cntr2, Cntr3, Cntr4, Cntr5, Cntr6, and Cntr7 provide the maximum index for the respective counter. The work in conjunction Counters.Cntr* with to enforce limits on counter array indices.
CouterArraySzResEncConfig. Configuration for the array element sizes associated with the seven user counters. Each field is the element length minus one so that the possible array sizes are in the range one through 256. The array element size is applied in an indexed reference in prs.store instructions. Additionally, there is a flag bit for each counter that indicates the counter is to be reset when encapsulation is encountered. This is illustrated in
R1, R2, R3, R4, R5, R6, and R7 indicates that the respective counter (Counters.Cntr*) is to be reset to zero when an encapsulation layer is encountered
Cntr1, Cntr2, Cntr3, Cntr4, Cntr5, Cntr6, and Cntr7 provide array element size minus one so that the range of element size is one to 256 bytes.
Holds the configuration parameters for processing a loop as illustrated in
MaxCnt is the limit for the maximum number of loop iterations. In conjunction with NodeLoopCnt.NumLoops, a simple for loop can be logically implemented as in:
for (NodeLoopCnt.NumLoops=0; NodeLoopCnt.NumLoops<LoopSpec.MaxCnt; NodeLoopCnt.NumLoops++) { . . . }
MaxNon is the limit for the maximum number of TLVs encountered in a TLV loop. This works in conjunction with NodeLoopCnt.NonPadCnt to enforce the limit
MaxPlen is the limit for the maximum number of consecutive bytes of padding in a TLV loop. This works in conjunction with NodeLoopCnt.PadLen to enforce the limit
MaxCPad is the limit for the maximum number of consecutive padding options in a TLV loop. This works in conjunction with NodeLoopCnt.ConPad to enforce the limit
Disp: Disposition when a loop limit is exceeded. See loop Common_Loop_Limit_Exceeded section for usage
E: Indicates that when loop count limit is exceeded it is an error
TLVSpec Holds the TLV parameters for TLV processing. This is a structured register and works with the PTLVFASTLOOP and PCAMJUMPTLVLOOP instructions, as illustrated in
IgnVal specifies a value in the type that indicates an unknown option is to be ignored.
IgnMask indicates a mask applied to the TLV type before comparing it to IgnVal. If the value is zero then the ignore value is ignored
PAD1: Indicates the type number for one byte padding. Valid when P bit is set.
PADN: Indicates the type number for multi-byte padding. Valid when the N bit is set.
EOL: Indicates the type number for one byte “end of list”. Valid when the E bit is set.
Disp: Disposition when a loop limit is exceeded. See loop Common_Loop_Limit_Exceeded.
P: PAD1 field is valid.
N: PADN is valid.
E: EOL field is valid.
This register holds the fully qualified address to jump to when the parser exits normally.
This register holds the fully qualified address to jump to when the parser exits normally.
Wildcard for CAM lookups. This register contains an address/code encoded value.
The fully qualified address is derived by
if (!IS_OKAY_RET(WildCard))TempAddress=ParserInstrBase (WildCard & 0xFFFFFF)
Alternate wildcard for CAM lookups. This register contains an address/code encoded value.
The fully qualified address is derived by
if (!IS_OKAY_RET(AltWildCard))TempAddress=ParserInstrBase (AltWildCard & 0xFFFFFF)
This register holds the fully qualified PC address to jump to for “at encapsulation” processing when an encapsulation node is encountered. A value of zero (NULL) indicates no “at encapsulation” processing is set.
This register holds the fully qualified PC address to jump to for post loop processing in a node. A value of zero (NULL) indicates no post loop processing is set.
This register holds the fully qualified PC address for code to execute when a comparison instruction evaluates to false.
DataExtractBase (p30, pdexbas)
This register holds the fully qualified base of pseudo instructions for the data extraction pseudo instructions.
This register returns a high precision object received timestamp. The timestamp is generated at ingress and set in the work item from the dispatcher to the cluster scheduler. Register initialization is illustrated in
The p registers can be read to integer registers and written from integer registers using the coprocessor read and write instructions CPPRSRD and CPPRSWR instructions where CoP is set to zero to indicate the parser coprocessor. The cpreg specifies the p register. This illustrated in
Moving values between the integer registers and p registers allows software to perform any transformations that are not directly supported by the parser instructions. CPPRSRD reads a value from a p register into an integer register. CPPRSWR writes a value from an integer register into a p register. CPPRSWRIMM writes an eleven bit immediate to a p register. CPPRSWRCAM writes or removes an entry in the protocol CAM by its index: if D is not set Cpreg register contains the key, and the Rs register contains the target; if D is set then the Cpreg register contains the key of a CAM entry to be removed. CPPRSRDCAM reads an entry from the CAM lookup (performs a lookup on the input key).
Assembly for Parser coprocessor read and write instructions is illustrated in
<ireg> is an integer register x0-x31 (ABI names zero, ra, sp, gp, tp, t0-t6, s0-s11, a0-a7)
<preg> is a parser register p0-p31 (ABI names pobjref phdrbas, pmdbase, pcurhdr, pdathdr, ppktlen, pfofnsq, ppktinf pinbase, pnext, ppendwk, pdbndlp, pexcode, paccum, pflags, pndlcnt, pcount, pconfig, pcntlim, pctarcf pctarsz, ploopsp, ptlvsp, pokay, pfail, pwild, palwild, patent, ppostlp, pcmpfal, ptimstp). <imm> is a value between −1028 to 1027 inclusive (<imm> is sign extended when moving to a register. <offset> is a relative PC offset in range shift right by two so effective range is −4096 to 4092 (note that targets in parser instructions are assumed to be four byte aligned)
The hardware parser has a standard set of codes to indicate failure conditions, and okay conditions. Codes are negative bytes from −1 to −127, or 0xFF to 0x80. Codes are naturally represented in half word, word, and double words simply by extending the sign bit. A check for a code is performed by checking the high order bit is set (i.e. check for a negative value). This is illustrated in
STOP_* codes greater than STOP_FAIL (−12) are considered normal parser exit codes, codes less than STOP_FAIL are considered abnormal conditions to stop the parser.
32-bit Parser Instructions
The 32-bit Hardware Parser instructions use custom-0 for the opcode and have a 4-bit function field that specifies the specific instruction. This is illustrated in
This section describes macros for common pseudo code.
These are helper macros used in the specification:
# define IS_RET_CODE(X)((X)<0)
# define IS_NOT_OK_CODE(X)((X)<=PANDA_STOP_FAIL)
# define IS_OK_CODE(X)(IS_RET_CODE(X) && (X)>PANDA_STOP_FAIL)
These are hardware helper functions mentioned in the pseudo code for instructions:
Load number of bytes into a register from the memory address referred to by <Address>. <NumberOfBytes> may be 1 (byte), 2 (half-word), 4 (word), or 8 (double word). Returns the loaded value.
Store the contents of a register to the memory address referred to by <Address>. <NumberOfBytes> may be 1 (byte), 2 (half-word), 4 (word), or 8 (double word)
Logical hardware function to wait for more data to arrive. This is invoked when data is streaming in such that PktLen.F or PktLen.P is not yet set. (In the current design this not required since it is assumed that whole packet is received before starting the parser)
Perform a CAM lookup and return the result or 0xFFFFFFFFFFFFFFFF on a miss
Remove the CAM entry corresponding to the index in the CAM table
Write the CAM entry corresponding to the index
Return the value from the lookup array corresponding the index
Set the array entry corresponding to the index to 0xFFFFFFFF
Write the array entry corresponding to the index with a value
Calls the external work item object allocator to get a sixty-four byte thread work item. A pair is returned: the first value is a sixty-four bit pointer to a work item, the second value is the sixteen bit index of the work item. The index will be sent in a “start thread set” message to the cluster scheduler
Perform a block store of registers p0 through p7 to the target memory address. This stores a work item in the memory allocated by AllocWorkItem
Enqueue a sixty-four bit message on the indicated FIFO. The parser enqueues messages on the pars_to_clussched_fifo in the prs.runthread instruction and when the parser completes parsing a packet
Dequeue a sixty-four bit message on the indicated FIFO. The parser dequeues messages from the clusfend_to_pars_fifo in the parser event loop
Dequeue a sixty-four bit message on the indicated FIFO. The parser dequeues messages from the clusfend_to_pars_fifo in the parser event loop
Assert an invariant condition is true. If the condition is false this considered a fatal error and the system should take appropriate action such as a reset. This is for debugging, and may be disabled a well tested system from production
Returns the base metadata address for the system. The metadata base contains an array of metadata blocks that are allocated via a cluster allocator. PktInfo.PktCtx references a metadata object as an index into the array. Presumably, the metadata base address is a system constant that doesn't need to be exposed as a register
Returns the base headers address for the system. The headers base contains an array of header buffers that are allocated via a cluster allocator. PktInfo.PktCtx references a header buffer as an index into the array. Presumably, the headers base address is a system constant that doesn't need to be exposed as a register
Returns the base work items address for the system. The work items base contains an array of work items that are allocated via a cluster allocator. PktInfo.PktCtx references a work item as an index into the array. Presumably, the work items base address is a system constant that doesn't need to be exposed as a register
Returns the base memory address for parser functions array. Presumably, the parser functions base address is a system constant that doesn't need to be exposed as a register
Convert Relative Instruction Address (Relative_Ins_Addr_to_FQA)
Pseudo registers. In addition, the registers described above there are some pseudo registers used in Assembly instruction as illustrated in
Load from Header Instructions
These instructions load a value from the header. The header is assumed to be streamed into the packet memory space via the SiPanda streaming datagram infrastructure. They are illustrated in
The Offset is relative to the address specified by either the current header pointer (PktHdrBase+CurHdr.Offset) or the data header pointer (PktHdrBase+DataHdr.Offset). Specifically, if the X bit is set the address loaded from (PktHdrBase+DataHdr.Offset+Offset). If the X bit is not set the address loaded from is (PktHdrBase+CurHdr.Offset+Offset). The number of bytes is specified by Sz. If Sz equals zero, then the number of bytes is eight, else the number of bytes is 1<<(Sz−1) (1, 2, or 4 bytes).
Once the value is fetched from memory the E-bit specifies if the byte ordering should be swapped (only applicable if more than one byte is being loaded). If the E-bit is set the target is treated as a big endian value that is swapped before being set in the register; if the E-bit is not set the target is treated as a little endian value that is set in the register as is.
The Shift and Blen fields specify transformations performed on the loaded value (after optional byte swapping). The fetched value is left shifted by the value in Shift and then masked to zero for the number of high order bits specified by Blen. If Sz is 0, that is 8 bytes is being fetched, then Blen is multiplied by two so as to allow masking to zero up to thirty high order bits with a multiple of two.
The load instructions check if there is sufficient length to perform the load. If the load is performed from the current header pointer (PktHdrBase+CurHdr.Offset) the check is that CurHdr.Offset+Offset+number_of_bytes is less than or equal to PktLen.ParseLen, and when the load is from the data pointer (PktHdrBase+DataHdr.Offset) the checks are that Offset +number_of_bytes is less than or equal to DataBndLoop.DataBound and DataHdr.Offset+Offset+number_of_bytes is less than or equal to PktLen.ParseLen.
If the extent of bytes being loaded is greater than CurHdr.Length when loading from the current header pointer (PktHdrBase+CurHdr.Offset), or greater than DataHdr.Length when loading from the data header pointer (PktHdrBase+DataHdr.Offset) then CurHdr.Length or DataHdr.Length is increased to the extent of the bytes being loaded. This effectively allows the load instruction to perform a header length check up to the length covering the last byte being loaded.
Loads a value from the header buffer into the accum register to be used as TLV type. It also is the head of the TLV loop. Note that the load may include additional bytes such as the TLV length field.
If a size qualifier is present in [bhw] or [hw]; then if b is present Sz instruction field is set to 1, if h is present Sz instruction field is set to 2, if w is present Sz instruction field is set to 3, else Sz instruction field is set 0. Blen is set based on <blen> or defaults to zero if the <blen> is not present in the arguments. If b or h is in the instruction mnemonic then <blen> is a value in the range 0 to 7 inclusive; if w is in the mnemonic then <blen> is a value in the range 0 to 15 inclusive; else <blen> is a value in the range 0 to 30 inclusive and must be a multiple of two. <shift> is a value in the range 0 to 7 inclusive. If .swp is present then the E bit is set indicating that bytes being loaded are swapped for endianness.
As illustrated in
If a size qualifier is present in [bhw] then if b is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 3 inclusive, if w is present Sz instruction field is set to 3 and <pos> if present is in the range 0 to 1 inclusive else Sz is set to 0 and. If .rev is present then the R bit is set and the order of the bits are reversed to match the logical numbering. <mask> is in the range 0 to 0xFFFF inclusive and is set in the Mask field. If <Mask> is not present then the Mask field is set of 0xFFFF
This is a specialized instruction to handle the common case of TLVs where the first byte is the type and the second byte is the length. This format is common in several protocols with TLVs including IPv4 options, IPv6 Hop-by-Hop Options, IPv6 Destination Options, SRv6 options, TCP options, etc. This is a loop head instruction for TLV loop processing.
The instruction does several things:
If this the first instruction in the loop then DataBndLoop.Loop and NodeLoopCnt loop values are initialized (note that the Common_Loop_Head is not used)
Check DataBndLoop.DataBound is zero which signifies normal end of TLV processing. If it is equal to zero then proceed to Common_End_of_Node handling
Load two bytes frompdatptr (PktHdrBase+DataHdr.Offset). If only one byte is available it is loaded to check for single byte TLVs (PAD1 and EOL).
Check if the type is PAD1 or EOL. If type is PAD1 check padding limits, increment data pointers, and continue loop with next TLV; if the type is EOL then normally exit loop.
If type is not EOL or PAD1 and only one byte was loaded then exit the parser on STOP_TLV_LENGTH
Compute the TLV length as the second loaded byte left shifted by Shift plus Len
Check if type is PADN; if it is then check padding limits, increment data pointers, and continue loop with next TLV
Otherwise a well formed non-padding option is found. Check non-padding option limit, set DataHdr.Length to the computed TLV length, and fallthrough to next instruction which is typically a PCAMJUMPTLVLOOP instruction. This illustrated in
Shift is the number of bits to shift the extracted length field by, and Len is the number to add to the length after being shifted.
<len> is a value in the range 0 to 511 inclusive and <len> is set in the Len field of instruction. <mult> is 1, 2, 4, 8, 16, or 32 and is set in the Shift field of the instruction as log2(<mult>). If <mult> is not present then Shift is set to 0.
The store instructions move data from a p register or an immediate value to a metadata frame (MetadataBase plus 4*FrameOffFnumSeqno.FrameOffset) structure or the common metadata (MetadataBase) at some offset.
PSTORE stores the contents of the Accum or Flags register or sub-register at an offset from Metadata Base or Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset and an optional array index from user defined counters #1, #2, #3, #4, #5, #6, or #7. This illustrated in
If the F bit is not set then the target destination base address is Metadata Base, else the des address is the frame pointer, Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset. The Offset is relative to the address specified by the base destination address. The number of bytes to store is specified by Sz; if Sz is 0 then eight bytes are stored, else the number of bytes stored is 1<<(Sz−1) (1, 2, or 4 bytes). Pos in indicates the sub-register (e.g. if Sz=1 and Pos=5 then the fifth byte in the Accum of Flags register is stored). The J-bit indicates the source is Accum if not set, and Flags if set. If Sind is non-zero, an array offset is added to the offset and Sind corresponds to counter Cntr1, Cntr2, . . . , Cntr7 where the counter's value serves as the array index. CounterArraySzResEncConfig.Cntr<cntr> contains the array element size of the counter. The S-bit is a stop bit that indicates that this instruction is the end of a node.
PSTOREREG stores the contents of a p register or sub-register at an offset from Metadata Base or Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset and an optional array index from user defined counters #1, #2, #3, #4, #5, #6, or #7. This is illustrated in
If the F bit is not set then the target destination base address is Metadata Base, else the dest base address is the frame pointer, Metadata Base plus 4*
The Offset is relative to the address specified by the base destination address. The number of bytes to store is specified by Sz; if Sz is 0 then eight bytes are stored, else the number of bytes stored is 1<<(Sz−1) (1, 2, or 4 bytes). Reg indicates the source p register. If Sind is non-zero, an array offset is added to the offset and Sind corresponds to counter Cntr1, Cntr2, . . . , Cntr7 where the counter's value serves as the array index. The element size of the array associated with a counter index is in CounterArraySzResEncConfig.Cntr<cntr>. The S-bit is a stop bit that indicates that this instruction is the end of a node.
PSTOREIMM stores an immediate byte at an offset from Metadata Base or Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset. This is illustrated in
If the F bit is not set then the target destination base address is Metadata Base, else the dest base address is the frame pointer, Metadata Base+4*
The Offset is relative to the address specified by the base destination address. The number of bytes to store is specified by Sz; if Sz is 0 then eight bytes are stored, else the number of bytes stored is 1<<(Sz−1) (1, 2, or 4 bytes). Value is the immediate byte value to store. The S-bit is a stop bit that indicates that this instruction is the end of a node.
Assembly for store instructions is illustrated in
If a size qualifier is present in [bhw]; then if b is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 3 inclusive, if w is present Sz instruction field is set to 3 and <pos> if present is in the range 0 to 1 inclusive, else size instruction field is set 0. If [<pos>] is not present, Pos is set to zero in the instruction. If stp is in mnemonic then S=1 else S=0. <offset> is an unsigned value in the range of 0 to 511 inclusive and is set in the Offset instruction field. <reg> is a parser register p0-p31 (ABI names pobjref phdrbas, pmdbase, pcurhdr, pdathdr, ppktlen, pfofnsq, ppktinf pinbase, pnext, ppendwk, pdbndlp, pexcode, paccum, pflags, pndlcnt, pcount, pconfig, pcntlim, pctarcf pctarsz, ploopsp, ptlvsp, pokay, pfail, pwild, palwild, patent, ppostlp, pcmpfal, ptimstp). If [cntr[1234567]] is present, then one of the seven user counters (Cntr1, Cntr2, Cntr3, Cntr4, Cntr5, Cntr6, or Cntr7) is being used as an array index where Sind is set as the corresponding value in the instruction. Note that pframe and pmdbase are virtual registers only used in this instruction; when pframe is the destination of the store the target address is Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset, and when pmdbase is the destination of the store the base target address is Metadata Base.
The hardware parser length instruction performs a number of different operations. There are two basic variants that determine how the Len field is processed: 1) If D is not set, the operation works by taking the value in the Accum register, shifting it by the value in and adding it to Len with all arithmetic truncated to 9 bits; if Shift is 7 then the operation is a constant length check against the value in Len, the data offset is then set to CurHdr.Offset plus Len. 2) else when D is set, the length field is the minimum length, the computed length is checked that it is greater than or equal to Len; if the minimum length check is okay then the data offset is set to CurHdr.Offset plus Len. The length field is taken from a sub-register (nibble, byte, half-word, or word) as indicated by Sz and Pos. The S-bit is a stop bit that indicates that this instruction is the end of a node. This is illustrated in
This instruction sets the CurHdr.Length based on Accum.
This instruction sets the DataHdr.Length for a non-TLV sub-node.
This instruction sets the DataBndLoop.DataBound relative to the current DataHdr.Offset. It must set it to a lesser value than that already set to or it causes an error.
This instruction sets the DataHdr.Length for a non-padding TLV option. This should be called at most once for processing a TLV.
This instruction sets the DataHdr.Length for padding. Note that PLENDATAPAD should be called at most once for a TLV and should not be called if PLENDATA is called (lest the number of consecutive padding bytes is undercounted).
This instruction processes an “End of List” or EOL option. Perform length checks for options, and then do a normal stop of the sub-node.
Assembly for length instructions are illustrated in
If a size qualifier is present in [bhw] then if b is present Sz instruction field is set to 0 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 3 inclusive, if w is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 1 inclusive, else Sz instruction field is set 3. If stp is in mnemonic then S=1 else S=0. <pos> refers to the sub-register and is set in the Pos instruction field. <len> is an unsigned value in the range of 0 to 255 inclusive and is set in the Len instruction field. <mult> is 1, 2, 4, 8, 16, or 32 and is set in the Shift field of the instruction as log2(<mult>). If <mult> is not present then Shift is set to 0. <len min> is an unsigned value in the range of 1 to 256 inclusive and is set in the Len instruction field. <mult min> is 1, 2, 4, 8, 16, 32, or 64 and is set in the Shift field of the instruction as log2(<mult min>). If <mult min> is not present then Shift is set to 0.
The next instruction, set immediate instruction, set code, and the and-mask instruction perform operations that have a 16-bit argument in the instruction. The V-bit for PNEXTNODE indicates the next node is an overlay, the V-bit for PANDMASK and PSETIMM indicates to operate on Flags or Accum. The S-bit (stop bit) indicates the instruction is an end of the current node. Pos is the sub-register half-word position for PANDMASK. Next is the next node address or code for PNEXTNODE, Mask is the mask for PANDMASK. This is illustrated in
This instruction sets the Next register with the value from the Next field, preserving the encapsulation and next-encapsulation bits. The V bit indicates to set the overlay bit.
This instruction loads a 16-bit immediate value into the Flags or Accum register as indicated by the V bit. Note that the immediate value is not sign extended
This instruction sets Next to a code value
This instruction performs a stop node.
Common_End_of_Nodeo;
This instruction reads an integer from the DataHdr that is a variant Protocol Buffers type. If the V bit is set then this indicates that a zigzag operation is done on the return value.
This instruction ANDs a mask with Flags or Accum as indicated by the V bit and stores the value in Flags or Accum respectively. It is used to consume flags (such as from a secondary CAM handler for a multi-bit flag).
Assembly for next instructions, as illustrated in
If stp is in mnemonic then S=1 else S=0. If ov, indicating the next node is an overlay, is in the mnemonic for PNEXTNODE then V=1 else V=0. <reloc-address> is in the range 0 to 0x3FFFC and must be a multiple of four; this is set in the Next instruction field with the value right shifted by two bits. If alt is in the mnemonic then the V bit set; this indicates the alternate value is returned by PVARINT. <mask> is a sixteen bit mask value set in Mask, <imm> is a sixteen bit mask value set in Imm.
The PEXTRACT instruction extracts an arbitrary set of contiguous bits from a parser register, and the PLOOP instruction starts a general loop. This is illustrated in
This instruction extracts an arbitrary set of contiguous bits from a parser register, and stores them in pflags or paccum based on the V-bit.
This instruction starts a general loop (simple counter loop for instance).
Pseudo Code for PLOOP:
Common_Loop Heado;
Assembly for extract and loop instructions. This illustrated in
If stp is in mnemonic then S=1 else S=0. <preg> is a parser register p0-p31 (ABI names pobjref phdrbas, pmdbase, pcurhdr, pdathdr, ppktlen, pfofnsq, ppktinf pinbase, pnext, ppendwk, pdbndlp, pexcode, paccum, pflags, pndlcnt, pcount, pconfig, pcntlim, pctarcf pctarsz, ploopsp, ptlvsp, pokay, pfail, pwild, palwild, patent, ppostlp, pcmpfal, ptimstp). <bit-pos> is a value in the range 0 to 63 inclusive, and <bit len> is a value in the range 1 to 64 where <bit_pos>+<bit len> is less than or equal to sixty-four.
There are three instructions to manipulate the seven user defined counters. PINCCNTR increments a counter, PSETCNTRBIT sets a bit in a counter (as a flag for instance), and PRESETCNTR resets a counter to zero. Optionally, the value of the counter before or after the operation (indicated by ValO) may be returned in Flags or Accum (as indicated by F). The S-bit is a stop bit that indicates that this instruction is the end of a node. This illustrated in
This instruction increments the encapsulation depth or one of the seven user defined counters. Cntr indicates the counter where a value of zero is for encapsulation depth and values one to seven correspond to counters 1 to 7. ValO indicates if the per or post operation value in the counter is returned. If ValO is one then the pre operation value is returned, if ValO is two then the post operation value is returned, else no value is returned. If a counter value is returned, it is set in Accum if F is zero, or Flags if F is one. The S-bit is a stop bit that indicates that this instruction is the end of a node.
This instruction sets a bit in a counter for one of the seven user defined counters. Cntr indicates the counter where values one to seven correspond to counters 1 to 7 (if Cntr is zero no operation is performed). Bnum is the bit position to set in the counter. ValO indicates if the per or post operation value in the counter is returned. If ValO is one then the pre operation value is returned, if ValO is two then the post operation value is returned, else no value is returned. If a counter value is returned, it is set in Accum if F is zero, or Flags if F is one. The S-bit is a stop bit that indicates that this instruction is the end of a node.
This instruction can be used to track occurrences of an event such as an instance of a particular TCP option when parsing a packet. For instance, bit #0 might be set when the MSS option is seen, bit #1 might be set when the window scaling option is seen, etc. If the limit exceeded bit is set for the counter, that is CounterLimitsConfig.E<cntr>, then if a bit is already set in the counter it is considered an error. This is useful to enforce that only one occurrence of an event is allowed, for instance in the TCP options case the parser could be configured to fail if two MSS options are in the same TCP packet.
This instruction resets one of the seven user defined counters. Cntr indicates the counter where values one to seven correspond to counters 1 to 7 (if Cntr is zero no operation is performed). If ValO is one then the pre operation value is returned, else no value is returned. If a counter value is returned, it is set in Accum if F is zero, or Flags if F is one.
Assembly for counter instructions is illustrated in
If stp is in mnemonic then S=1 else S=0. [1234567] indicates one of the seven user-defined counters. <bit-pos> is a value between zero and seven and is set in Bnum. If preval is present in the mnemonic then ValO is set to 1, else if postval is present in the mnemonic then ValO is set to 2, else ValO is set to 0. If paccum is present as the destination register then F is set to 0, if pflags is present as the destination register then F is set to 1, else F is set to 0.
One of the key features of the hardware parser is a CAM structure that can be used for quickly looking up what should be the next node. The CAM structure has a 20-bit key and returns a 32-bit value. If Share is non-zero then that indicates one of fifteen shared subtables numbered one to fifteen; the key is composed of the four bit Share value followed by a sixteen bit match value. If Share is zero then that indicates a non-shared sub-table; the key is composed of four zero bits, followed by an eight bit subtable selector that is derived from the PC address of the CAM instruction, followed by an eight bit match value. The match value, either up to eight bits for a non-shared table, or up to sixteen bits for a shared table, is taken from a sub-register of the Accum of Flag register, depending on the F bit, as indicated by Sz and Pos. The S bit is the stop bit, the A bit indicates the alternate wild card is selected. This illustrated in
This instruction does a CAM lookup on either an Accum or Flags sub-register and places the result in the Accum register so it can be used for comparison or length computations. It also has a stop bit to potentially signal the end of a node.
This instruction performs a CAM lookup on either an Accum or Flags sub-register and places the result in the Next register thereby setting the next node to process. It also has a stop bit to potentially signal the end of a node.
This instruction performs a CAM lookup on a sub-register for either Accum or Flags and jumps to the resultant address. It also has a stop bit to potentially signal the end of a node.
This instruction performs a CAM lookup and jumps to the resultant address in the context of a loop iteration. It also has a stop bit to potentially signal the end of a node.
PCAMJUMPLOOP is called for plain loops, TLV loops, and Flags fields loops. Accum or Flags is expected to contain the lookup value. In the case of a TLV loop this will be a TLV type that was loaded by PLOADTLVLOOP, and for a Flags loop the Accum register contains the index of the flag to lookup that was determined by PFLAGSLOOP.
This instruction performs a CAM lookup and jumps to the resultant address in the context of a TLV iteration. PCAMJUMPTLVLOOP is called in conjunction with PTLVFASTLOOP. It also has a stop bit to potentially signal the end of a node.
Assembly for CAM instructions as illustrated in
{**miss**} indicates an action to take on a CAM miss and is one of:
{**miss**} not present: indicates to continue
.wild: indicates to use WildCard
.alt: indicates to use AltWildCard
.stop: indicates to stop the parser with success
.stopsub: indicates to stop the current sub-node or loop iteration with success
.fail: indicates to stop the parser on with failure
.failsub: indicates to stop the current sub-node or loop iteration with failure
If a size qualifier is present in [nbh] then if n is present Sz instruction field is set to 0 and <pos> if present is in the range 0 to 15 inclusive, if b is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 3 inclusive. <pos> refers to the sub-register and is set in the Pos instruction field. <share> is a value in the range 1 to 15 inclusive and is set in the Share instruction field if present. If pc is the share argument then Share is set to 0 (indicating that the PC is used to derive table specifier)
These instructions are used to lookup a table of thirty-two bit entries in an array. The match index is taken from a sub-register of the Accum of Flag register, depending on the F bit, as indicated by Sz and Pos. The array memory is contained in the hardware at base address in SysArrayBase( ) and for each lookup a Base offset is provided that is an element offset into a subarray. The S-bit is a stop bit that indicates that this instruction is the end of a node. This is illustrated in
This instruction does an array lookup on either an Accum or Flags sub-register and places the result in the Accum register so it can be used for comparison or length computations. It also has a stop bit to potentially signal the end of a node.
This instruction does an array lookup on either an Accum or Flags sub-register and places the result in the Next register thereby setting the next node to process. It also has a stop bit to potentially signal the end of a node.
This instruction performs an array lookup on a sub-register for either Accum or Flags and jumps to the resultant address. It also has a stop bit to potentially signal the end of a node.
This instruction performs an array lookup and jumps to the resultant address in the context of a loop iteration. It also has a stop bit to potentially signal the end of a node.
PARRJUMPLOOP is called for plain loops, TLV loops, and Flags fields loops. Accum or Flags is expected to contain the lookup value. In the case of a TLV loop this will be a TLV type that was loaded by PLOADTLVLOOP, and for a Flags loop the Accum register contains the index of the flag to lookup that was determined by PFLAGSLOOP.
Assembly for array instructions. This is illustrated in
If a size qualifier is present in [nbh] then if n is present Sz instruction field is set to 0 and <pos> if present is in the range 0 to 15 inclusive, if b is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 3 inclusive. <pos> refers to the sub-register and is set in the Pos instruction field. <base> is the base offset in units of words (thirty-two bits) and is in the range 0 to 511 inclusive.
Compare a half word in a sub-register of the Accum to an 16-bit immediate value. Pos field indicates the halfword sub-register (0, 1, 2, or 3). Value is the value for comparison. Er describes action to take when the compare evaluates to false. If the N bit is not sent the compare is for inequality, when Nis not set the compare is for equality. This is illustrated in
Assembly for compare halfword immediate instructions. This is illustrated in
<pos> indicates the half word sub-register position and is a value from 0 to 3 inclusive and is set in the Pos field in the instruction. <value> is a value in the ranges 0 to 0xFFFF inclusive and is set in Value in the instruction. If stop is present in the mnemonic then er is set to 0, if stopsub is present in the mnemonic then er is set to 1, fail is present in the mnemonic then er is set to 2, if cmpfail is present in the mnemonic then er is set to 3.
Compare a byte in a sub-register of the Accum to an 8-bit Value with a Mask. If Mask is 0xFF then the instruction performs a simple comparison to Value. Er describes action to take when the compare evaluates to false. Pos is the position of the byte sub-register. This is illustrated in
PCMPIB instruction
Compare a byte sub-register in Accum with a mask applied to an immediate value for equality.
Compare a byte sub-register in Accum with a mask applied to an immediate value for equality.
Assembly for compare byte immediate instructions. This is illustrated in
<pos> indicates the byte sub-register position and is a value from 0 to 7 inclusive and is set in the Pos field in the instruction. <value> is a value in the ranges 0 to 255 inclusive and is set in Value in the instruction, <mask> if present is in the range 0 to 0xFF inclusive and is set to the Mask field in the instruction, it <mask> is not present the default value of 0xFF is set in the Mask field of the instruction. If stop is present in the mnemonic then er is set to 0, if stopnode is present in the mnemonic then er is set to 1, if stopsub is present in the mnemonic then er is set to 2, otherwise if no descriptor is present in the mnemonic then er is set to 3.
Compare a nibble, byte, half-word, or word sub-register in Accum, as indicated by Sz and Pos, to an immediate byte Value for inequality (less than, less than or equal to, greater than, or greater than or equal to). Er describes action to take when the compare evaluates to false. The S-bit is a stop bit that indicates that this instruction is the end of a node. This is illustrated in
Compare a sub-register in Accum to be less than a byte immediate value.
Compare a sub-register in Accum to be less than or equal to a byte immediate value.
Compare a sub-register in Accum to be greater than a byte immediate value.
Compare a sub-register in Accum to be greater than or equal to a byte immediate value.
Assembly for compare byte immediate instructions. This is illustrated in
If a size qualifier is present in [nbhw] then if n is present Sz instruction field is set to 0 and <pos> if present is in the range 0 to 15 inclusive, if b is present Sz instruction field is set to 1 and <pos> if present is in the range 0 to 7 inclusive, if h is present Sz instruction field is set to 2 and <pos> if present is in the range 0 to 3 inclusive, else Sz instruction field is set 3 (for w) and <pos> if present is in the range 0 to 1 inclusive. <pos> refers to the sub-register and is set in the Pos instruction field. <value> is in the range 0 to 255 inclusive. If stop is present in the mnemonic then er is set to 0, if stopnode is present in the mnemonic then er is set to 1, if stopsub is present in the mnemonic then er is set to 2, otherwise if no descriptor is present in the mnemonic then er is set to 3.
Initialize parser for next packet instruction
The PINITPARSER instruction initializes the parser state to process a PDU. This is illustrated in
The arguments to this instruction are in the “a” registers following standard C calling conventions in RISC-V. Note the “a” registers are registers number 10 through 17.
The arguments are:
a0: <address_of_packet>, base address of the packet headers
a1: <packet len>, full length of the packet
a2: <metadata_address>, base address for metadata
a3: (<seqno><<32)|<checksum>, sequence number assigned by the dispatcher and full packet checksum computed on ingress
a4: <IFID>, interface identifier of the ingress
a5: <object_reference>
a6: <timestamp>
a7: <pkt_ctx>: Packet context received in the work item
InitializeParser(regs[10], regs[11], regs[12], regs[13], regs[14], regs[15], regs[16], regs[17]);
Assembly for initialize parser instruction. This is illustrated in
PRUNTHREAD, EVENTLOOP, and PEVENTLOOPEND are specialized instructions to implement worker thread scheduling and the parser event loop in the SDPU. See section below “SiPanda Parser and the SDPU” for a description of the parser's place in the SDPU architecture.
The PRUNTHREAD instruction requests that work be performed to process a protocol layer in a worker thread. A work item indicates a function to run in a worker thread to process a protocol layer and includes the parser state describing the protocol layer to be processed. When PRUNTHREAD is executed, a snapshot of the material parser state is taken and placed in an allocated work item which is a memory object. To simplify this, parser registers zero through seven are overlaid with the work item data structure such that taking the snapshot is done by a block copy for the parser registers, sixty-four bytes, to the address of an allocated work item in memory. The parser sends these messages to the cluster to initiate scheduling of the worker threads. The cluster scheduler processes the message and schedules threads to run all the work items in the list. This is illustrated in
FuncNum indicates the function number that indexes into a table of functions to run. S is the stop bit.
Assembly for run thread instructions. This is illustrated in
The PEVENTLOOP instruction implements the start of the parser event loop for the SDPU. The instruction listens on FIFOs for “start parser” messages from the cluster frontend, initializes the parser for the next packet, and parses packets as requested by the cluster frontend by jumping to a parsing function. PEVENTLOOPEND performs the event loop end processing upon return from a parser. The instruction checks for pending work in PendingWork.PendingWork, and if there it sends a “last thread in thread set” message on a FIFO to the cluster scheduler; if there is no pending work item then the packet is simply freed (i.e. this is a silent drop). The instruction then loops to the head of the event loop. This is illustrated in
Return address is a signed PC relative branch address that is set as the return address when a parser function is run. This is set as <address_offset>/4.
The pseudo code for these instructions is in the “SiParser and the SDPU” section below.
Assembly for Parser Event Loop instructions. This is illustrated in
<return address> is the return address when the parser completes. If the value is four, then the return address is the next instruction after prs.eventloop.
prs.eventloop and prs.eventloopend work in conjunction to implement the parser event. The code for the tightest possible loop would be:
j prs_start
prs_end: prs.eventloopend
prs_start: prs.eventloop prs_end
The first time this code is run, the jump to prs_start starts the parser event loop for the first iteration. When the parser invokes a parser function, the return address is prs_end which is the prs.eventloopend instruction. Subsequently, when the prs.eventloopend completes the next instruction is prs.eventloop thus starting the next iteration of the event loop.
PDATAEXTRACT runs a set of pseudo instructions to optimize metadata extraction. The pseudo instructions are specialized thirty-bit instructions that are not in RISC-V format. A pseudo instruction performs a copy from header data to metadata to perform data extraction. These instructions encapsulate both the load and store operations, and they can move more than eight bytes in one instruction.
An example pseudo instruction to save the IPv6 addresses to metadata is:
Assembly for Data ExtractInstructions. This is illustrated in
InsIndex indicates the index of the first pseudo instruction to execute, NumIns plus one is the number of pseudo instructions to run. S is the stop bit.
Data Extraction pseudo instructions are specialized non-RISC-V instructions that perform metadata extraction, or writes of immediate data to metadata. These are thirty-two bit instructions that don't use canonical RISC-V opcodes.
PSEUDOMOVE moves data from the current header or data header to metadata for some number of bytes. The destination may use a counter array index, and endian swap before a store may be requested. This instruction would be used in lieu of pairs of PLOAD and PSTORE instructions. PSEUDONIBBMOVE and PSEUDONIBBMOVE moves a nibbles from data from the current header or data header to metadata for some number of nibbles. The destination may use a counter array index, and endian swap before a store may be requested. The instructions are used in lieu of pairs of PLOAD and PSTORE instructions where Shift and Blen are set appropriately in PLOAD to load nibbles. PSEUDOMOVEI16 and PSEUDOMOVEI16 store an eight or sixteen bit immediate value. These instructions can be used in lieu of PSTOREIMM instructions. This is illustrated in
If the F bit is not set then the target destination base address is Metadata Base, else the dest address is the frame pointer, Metadata Base plus 4*FrameOffFnumSeqno.FrameOffset. The DstOffset is relative to the address specified by the base destination address. The SrcOffset is relative to the address specified by either the current header pointer (PktHdrBase+CurHdr.Offset) or the data header pointer (PktHdrBase+DataHdr.Offset). Specifically, if the X bit is set the address loaded from (PktHdrBase+DataHdr.Offset+Offset). If the X bit is not set the address loaded from is (PktHdrBase+CurHdr.Offset+Offset). If Sind is non-zero, an array offset is added to the offset and Sind corresponds to counter Cntr1, Cntr2, . . . , Cntr7 where the counter's value serves as the array index. The element size of the array associated with a counter index is in CounterArraySzResEncConfig.Cntr<cntr>. For PSEUDEOMOVE length is in bytes, for PSEUDONIBBMOVE and PSEUDONIBODMOVE, length is number of nibbles. E indicates that bytes are endian swapped before being stored.
These pseudo instructions. These are not normal RISC-V instructions so an assembler would treat these as a different ISA. This is illustrated in
The pseudo instructions are expected to be run in a near accelerator. The pseudo code for this is:
The data extraction pseudo instructions are invoked by the PDATAEXTRACT instruction. The pseudo instruction can run in a coprocessor. It is also possible for the pseudo instructions to execute concurrently with other parser instructions subject to the following rules.
When PDATAEXTRACT a snapshot of CurHdr and DataHdr registers is saved for processing the pseudo instructions. This is done to allow CurHdr and DataHdr to be updated by subsequent parser instructions
When PRSRUNTHREAD runs (specifically when a work item message is sent to the cluster scheduler), the pseudo operations for any preceding PDATAEXTRACT must be complete. This ensures that when a work thread runs it is able to see the metadata.
The PANDA Parser is a framework and API for programming protocol parser pipelines that utilizes the mechanisms and PANDA API for parallelism and serial data processing as described in this architecture. Protocol parsing is a fundamental operation in serial data processing such as networking processing. A protocol parser can be represented as a parse graph that shows various protocol layers that may be parsed and the relationships between layers. The processing of one data object can be thought as one “walk in the parse graph”. At each node in the graph the corresponding protocol layer of a data object (protocol header in networking parlance) is parsed and processed. Processing may include validations, extracting of metadata from the protocol layer, and arbitrary protocol processing. Parsing is driven by a parser engine that performs the parse walk and calls processing functions for each layer. The parser engine parsers top level protocols, TLVs, and flag-fields.
The fundamental data structures of the PANDA parser are:
Protocol nodes provide the properties and functions needed to parse one protocol in a parse graph to proceed to the next protocol in the parse graph for a packet. A parse node contains common characteristics that reflect the standard protocol definition (for instance there is only one standard procedure to determine the length of an IP header). The parse walk over a protocol node requires determining the protocol type of the next node and the length of the current node. A protocol node has two corresponding functions that are implemented per a specific protocol:
len: Returns the length of the current protocol layer (or protocol header)
next_proto: Returns the protocol type of the next layer
A parse node is an instantiation of one node in the parse graph of a parser being defined. A parse node includes a reference to the protocol node for the specific protocol, as well as customizable processing functions. A parse node allows defining two optional functions:
extract_metadata: Extracts metadata, e.g. protocol fields, from a protocol header and saves it in the metadata memory and perform arbitrary protocol processing. This function might implement the full logic of protocol processing
A protocol table is a lookup table that takes a protocol number as input as the protocol type of the next protocol layer, and returns the parse node for the next layer. The protocol numbers can be the canonical protocols numbers, for instance a protocol number might be an IP protocol number where the table contains parse nodes for various IP protocols (e.g. for TCP, UDP, etc.). Non-leaf parse nodes have a reference to a corresponding protocol table, for instance, a parse node for IPv6 would refer to a protocol table that takes an IP protocol number as input and returns the parse node for the corresponding IP protocol.
A parser defines a parser and includes a set of parse nodes, each having a reference to a protocol node. Non-leaf parse nodes have a reference to a protocol table. The parse nodes are connected to be a graph via the relationships set in the protocol tables. The parser can be represented as a declarative data structure in C and can equivalently be viewed as a type of Finite State Machine (FSM) where each parse node is one state and transitions are defined by next protocol type and associated protocol tables. A parser defines a root node which is the start node for parsing an object (for networking the root is typically Ethernet).
Type-Length-Value tuples (TLVs) are a common networking protocol construct that encodes variable length data in a list. Each datum contains a Type to discriminate the type of data, a Length that gives the byte length of the data, and a Value that is the bytes of data. TLVs are parsed in the context of a top level protocol, for instance TCP options and IPv4 options are represented by TLVs parsed in the context of a TCP header and IPv4 header respectively.
A protocol node with TLVs is an extended protocol node that describes a protocol that includes TLVs. A protocol node with TLVs provides the properties and functions to parse TLVs in the context of a top level protocol and includes three operations: tlv_len, tlv_type, and tlv_data_offset. The tlv_len function returns the length of a TLV (and therefore the offset of the next TLV), tlv_type returns the type of a TLV, and tlv_data_offset returns the offset of the data within a TLV. Note that tlv_len returns the length of the whole TLV including any TLV header, so the length of just the data in a TLV is the total length of the TLV as given by tlv_len minus the offset of the data as given by tlv_data_offset.
A parse node with TLVs is an extended parse node that has reference to a protocol node with TLVs and a TLV table. A TLV table is a lookup table that takes a TLV type as input and returns a TLV parse node for the TLV.
A TLV parse node describes the processing of one type of TLV. This includes two optional operations: extract_tlv_metadata and handle_tlv. These have the same function prototypes as the similarly named functions defined for a parse node (see above) where extract_tlv_metadata extracts metadata from a TLV and places it into the metadata structure, and handle_tlv allows arbitrary processing of the TLV.
Flag-fields are a common networking protocol construct that encodes optional data in a set of flags and data fields. The flags indicate whether or not a corresponding data field is present. The data fields are fixed length and ordered by the ordering of the flags indicating the presence of the fields. Examples of protocols employing flag fields are GRE and GUE.
A flag-field structure defines one flag/field combination. This structure includes: flag, mask, and size fields. The flag value indicates the flag value to match, the mask is applied to the flags before considering the flag value (i.e. a flag is matched if flags & mask==flag), and size indicates size of the field.
A protocol node with flag-fields is an extended protocol node that describes a protocol that includes flag-fields. A protocol node with flag-fields has two flag-fields related operations: flags returns the flags in a header and fields_offset returns the offset of the fields.
A parse node with flag-fields is an extended parse node that has a reference to a protocol node with flag-fields and a flag-fields table. A flag-fields table is an array of flag-field structures that define the parseable flag-fields for a protocol. A flag-fields table may be defined in conjunction with a protocol node definition and is used by functions of the protocol node or parse nodes for the protocol.
An instance of a PANDA parser can be mapped to the parser instructions defined in this specification. The goal is that the developer would write a parser in a high level language such as C and an optimizing compiler would emit the sequence of parser instructions that instantiate the parser to run in hardware with high performance. This is facilitated by the design where elements of the declarative representation of a parser in the high level language directly map to specific constructs in the instruction set (following the principles of Domain Specific Architecture).
The nodes of a parser are implemented in a parser as a sequence of instructions that process the node where the sequence is terminated by a .stp instruction (typically an instruction with the S-bit set, but could also be terminated at the end of a loop in a loop instruction). The implementation of a node encompasses the processing functions of both a protocol and a parse node. Protocol tables are mapped to CAM tables which provide the linkage between different nodes in the parse graph.
The basic structure of a node would be:
Determine the length of the header and set CurHdr.Length accordingly. For a variable length protocol this might entail loading a field from the packet header and then executing a length instruction. The instructions of interest for this are:
prs.load (both to load length field from pcurptr, PktHdrBase+CurHdr.Offset, as well as to set CurHdr.Length)
prs.lenset pcurhdr, prs.lensetadd pcurhdr, prs.lensetmin pcurhdr
Perform optional compare functions on packet fields. The instructions of interest are:
prs.load (to load fields frompcurptr, PktHdrBase+CurHdr.Offset)
prs.cmpi.h, prs.cmpnei.h compare half-word sub-register to a constant
prs.cmpi.b, prs.cmpnei.b compare byte sub-register to a constant with a mask
prs.cmplti*, prs.cmpltei*, prs.cmpgti*, prs.cmpgtei* compare a sub-register to a constant for less than, less than or equal to, greater than or greater than or equal to Determine the next protocol and set Next. The instructions of interest for this are:
prs.load (to load the next header field)
prs.camnext
prs.setaddr pnext
Save metadata, invoke thread processing. The instructions of interest for this are:
prs.load (to load fields frompcurptr, PktHdrBase+CurHdr.Offset)
prs.store, prs.storei, storereg save metadata in the current frame or meta metadata
prs.action invoke thread processing for a protocol layer (details TBD)
Optionally execute a loop to process sub-nodes such as TLVs. Details are described in the next sections
End current node processing and proceed to next in Next. The instructions of interest for this are:
*.stp: those instructions that set the S-bit
camjump* instructions may invoke end of node processing
*loop instructions invoke end of node processing unless post loop processing is configured
Loops are defined from parsing protocol constructs such as TLVs, lists, or flag-fields. LoopSpec (ploopsp) contains configuration for a loop including limits on number of iterations. NodeLoopCnt.NumLoops (ploopct) counts the number of iterations performed, and NodeLoopCnt.NonPadCnt (ploopct) counts the non-padding TLV sub-nodes.
The general flow of a loop is:
Create a loop head. Parse loops are create using
prs.loop: starts a simple loop that performs LoopSpec.MaxCnt iterations
prs.tlvloop starts a loop to process TLVs
prs.flagsloop starts a loop to process flags in flags-fields
prs.tlvfastloop is a special instance of TLV processing
Process on iteration of a loop as a “sub-node”, for instance one particular TLV would be processed as a sub-node. The strategy for processing a sub-node is similar to those for processing a node as described above. This is done differently depending on the type of loop as described below.
Perform a lookup on the type and jump to the sub-node processing. The instructions of interest for this are:
Pcamjumploop
Pcamjumptlvloop
Process the sub-node. Typical flow is:
Perform optional compare functions on packet fields. The instructions of interest are:
prs.load (to load fields from the pdatptr, PktHdrBase+DataHdr.Offset)
prs.cmpi.h, prs.cmpnei.h compare half-word sub-register to a constant
prs.cmpi.b, prs.cmpnei.b compare byte sub-register to a constant with a mask
prs.cmplti*, prs.cmpltei*, prs.cmpgti*, prs.cmpgtei* compare a sub-register to a constant for less than, less than or equal to, greater than or greater than or equal to Save metadata, invoke thread processing. The instructions of interest for this are:
prs.load (to load fields from the pdatptr, PktHdrBase+DataHdr.Offset)
prs.store, prs.storei, storereg save metadata in the current frame or meta metadata
prs.action invoke thread processing for a sub-node (details TBD)
Determine the length of the sub-node header and set DataHdr.Length accordingly. For a variable length protocol, such as a TCP option, this might entail loading a field from the packet header and then executing a length instruction. The instructions of interest for this are:
prs.load (both to load length field from pdatptr, PktHdrBase+DataHdr.Offset, as well as to set DataHdr.Length)
prs.lenset pdathdr, prs.lensetadd pdathdr, prs.lensetmin pdathdr
prs.lensettlv pdathdr, prs.lensettlvadd pdathdr, prs.lensettlvmin pdathdr
prs.lensetpad pdathdr, prs.lensetpadadd pdathdr, prs.lensetpadmin pdathdr
prs.lenseteol pdathdr, prs.lenseteoladd pdathdr, prs.lenseteolmin pdathdr
At a .stp instruction, perform the end of sub-node processing and jump to the loop head to handle the next iteration. Appropriate conditions are checked for exiting or breaking the loop. When the loop is terminated normally, jump to post loop processing if set PostLoop contains an address) or perform end of node processing. The instructions of interest for this are:
*.stp: those instructions that set the S-bit
TLV loops are a variant of loop processing with the context of a TLV loop.
A loop head is created by prs.loadtlvloop or prs.loadtlvloopmb instructions; these instructions load the TLV type field into Accum (paccum).
A type lookup and jump to sub-node processing is performed by prs.jumploop or prs.jumptlvloop
In the case of prs.jumploop, a CAM lookup is performed and a jump made to the return address. CAM miss processing is performed for a CAM miss.
In the case of prs.jumptlvloop, a CAM lookup is performed and a jump made to the return address. On a CAM miss an extra check is performed to evaluate if the unknown TLV is to be ignored. This is done by and'ing the TLV type in Accum with TLVSpec.IgnMask and comparing the result to TLVSpec.IgnVal; if they are equal then jump is performed to the loop head instruction to process the next iteration. If the values are not equal, the TLV is not ignored and CAM miss processing is invoked.
A sub-node is processed as described above with respect to performing additional compare checks, saving metadata, and invoking processing threads
The length of the subnode is set by one of these instructions being called (only one invocation of any them per sub-node)
prs.lensettlv pdathdr, prs.lensettlvm pdathdr set the length for a non-padding TLV
prs.lensetpad pdathdr, prs.lensetpadm pdathdr set the length for a padding TLV. This also checks limits concerning padding such as number of consecutive padding options and number of consecutive bytes of padding
prs.lenseteol, pdathdr, prs.lenseteolm pdathdr set the length for an “End of List” TLV. This also breaks the loop and will jump to either post loop processing or will proceed to the next node.
Flags loops are a variant of loop processing in the context of processing flag-fields in a loop.
A loop head is created by prs.flagsloop; at the first execution the flags to be processed are copied from a sub-register in Accum to the Flags register
At each iteration, including the first, the prs.flagsloop instruction runs. It examines the Flags register. If Flags is zero, the loop terminates normally and either a jump is made to post loop processing or end of node processing; else the first set bit in Flags is located. The index of the first bit is set in Accum and the bit is zeroed in the Flags register.
Do a lookup on the index of the flag to process, i.e. the value set in Accum, and jump to sub-node processing is performed by prs.jumploop
The flag-fields sub-node is processed as described above with respect to performing additional compare checks, saving metadata, and invoking processing threads
The length of the sub-node header, that is the length of the data field for the flag, is typically set in DataHdr.Length by prs.lenset pdathdr with a constant length argument
As described for sub-node processing above, at a .stp instruction, perform the end of sub-node processing and jump to the loop head to handle the next iteration
TLV fast loops are a fast variant of TLV loops.
A loop head is created by prs.fasttlvloop. This instruction:
Checks if DataBndLoop.DataBound is zero meaning the end of the TLV list is reached. If it is zero then and the loop exits normally and either a jump is made to post loop processing or end of node processing is performed
Loads the first two bytes atpdatptr (PktHdrBase+DataHdr.Offset). This is the type byte and length byte. If only byte is available for the limit of DataBndLoop.DataBound or packet length, load only one byte
If the type byte is equal to TlvSpec.PAD1 (and TlvSPec.P is set) then one padding byte is processed. Padding limits defined in TlvSpec for the number of consecutive padding options and number of consecutive bytes of padding are checked. If any limits are exceeded the parser exits abnormally; else the data offset and point advance by one byte and the next type byte is loaded (go to step b.)
If the type byte is equal to TlvSpec.EOL and TlvSPec.E is set then the end of loop is processed; the loop exits normally and either a jump is made to post loop processing or end of node processing is performed
Otherwise, if only one byte was able to be loaded exit the parser on a malformed TLV
If the type is equal to TlvSpec.PADN and TlvSPec.N is set then N padding bytes are processed. Padding limits defined in TlvSpec for the number of consecutive padding options and number of consecutive bytes of padding are checked. If any limits are exceeded the parser exits abnormally; else the data offset and point advance by the number of padding bytes plus two account for the type and length bytes and the next type byte is loaded (go to step b.)
Otherwise, DataHdr.Length is set to the determined TLV length
A type lookup and jump to sub-node processing is performed by prs.jumploop or prs.jumptlvloop. Note that the Accum contains the length as well so a lookup could be performed on the full Type and Length which is convenient in some cases
In the case of prs.jumploop, a CAM lookup is performed and a jump made to the returned address. CAM miss processing is performed for a CAM miss.
In the case of prs.jumptlvloop, a CAM lookup is performed and a jump made to the returned address. On a CAM miss an extra check is performed to evaluate if the unknown TLV is to be ignored. This is done by and'ing the TLV type in Accum with TLVSpec.IgnMask and comparing the result to TLVSpec.IgnVal; if they are equal then jump is performed to the loop head instruction to process the next TLV. If the values are not equal, the TLV is not ignored and CAM miss processing is invoked.
A sub-node is processed as described above with respect to performing additional compare checks, saving metadata, and invoking processing threads.
The length of the subnode does not need to be set by the sub-node since the prs.tvfastloop already handles the TLV length.
As described for sub-node processing above, at a .stp instruction, perform the end of sub-node processing and jump to the loop head to handle the next iteration (go to step 1.b.)
The hardware parser handles protocol encapsulation by managing the Counters.Encap register. The register is incremented when transitioning to a new encapsulation layer. As discussed in the description of the MetadataBase (pmdbase) register, the Counters.Encap register serves as the index of the metadata frame where frame pointer is =&frame[Counters.Encap](equals MetadataBase plus 4*FrameOffFnumSeqno.FrameOffset). The maximum encapsulation depth is limited by ParserConfig.MaxEncap. If this limit is reached, then an error is triggered if ParserConfig.EE is set; else Counters.Encap does not increment for additional layers of encapsulation and neither does FrameOffFnumSeqno.FrameOffset change which has the effect that the last metadata frame contents may be overridden by nested encapsulations if ParserConfig.EO is set (this may be desirable in certain circumstances such as when the caller is only interested in the outermost and innermost headers).
Encapsulation depth is incremented in one of two ways:
In common end of node processing (Common_End_of_Node), if the Next's encapsulation bit is set (i.e. masked bit 0x40000000) then ParserConfig.Encap is incremented when jumping to the next node
The prs.inc encap instruction increments ParseConfig.Encap and the effect is immediate upon return of the instruction.
When transitioning to the next node, processing Next in Common_End_of_Node, if the next node is marked as an overlay node (i.e. masked bit 0x20000000 is set in Next) then overlay processing is performed. For overlay processing, the current header and data offsets, pointer, and lengths don't change (as opposed to non-overly processing in which case CurHdr.Offset advances and the other pointers, offsets, and lengths are set accordingly.
This section provides guidance and strategies on programming the CAM and hardware lookup array.
Both the CAM and lookup array are presented as arrays for which entries can be set and deleted.
The lookup array is straightforward to program. Target values are written at specific indices in the array using the prs.array.write instruction where the first source operand contains the index and the source second operand contains the thirty-two bit value to write at that index. Entries can be removed using the prs.arr.delete instruction where the source operand is the index of the entry to be deleted. The effect of deleting an array entry is to write a STOP_OKAY code in the entry for the index.
The CAM is programmed as an array of entries where each entry is composed of a thirty-two bit key and a thirty-bit target value. CAM entries are written using the prs.cam.write instruction where the first source operand is the index, and the second operand encodes the key and the target value; the key occupies the high order thirty-two bits of the second operand, and the target occupies the low order thirty-two bits. CAM entries are removed using the prs.cam.delete instruction where the source operand is the index of the entry to be deleted. The effect of deleting a CAM entry is that the key is written with a value of 0xFFFFFFFF and the target value is set to zero; this makes the key an invalid value that should never match any possible CAM lookup.
Note, similar to the programming of the lookup array, it is the prerogative of the software to manage the CAM as an array with some known number of elements. For instance, when adding an entry to the CAM table it's up to the software to determine an unset entry in the table and set the new entry at that index. The software needs to handle the case where there are no free entries in the table, and also needs to ensure that all keys are in the table are unique. Maintaining a shadow table in software of the CAM table may be prudent for table management.
Shared CAM tables, non-shared CAM tables, and arrays may be used in tandem to implement various protocol lookups.
The advantage of shared tables is that one sub-table can be used for lookup in different instructions, and a shared table allows a 16-bit match value. For instance, a common table for looking up 16-bit EtherType can be used both in Ethernet parse nodes and GRE parse nodes. The limitation of shared tables is that there is a maximum of fifteen shared tables.
The advantage of non-shared tables is that there can be more of them than shared tables. A non-shared table can only be used by one CAM instruction, so shared tables are suitable for “one-off” lookups. For instance, a lookup of the GRE version number is likely only performed by a GRE node so such a table wouldn't need to be shared. Non-shared tables allow only eight bit lookups and there is a risk of key collisions in the 8-bit selector between different instructions.
Arrays have the advantage of simplicity and space compared to CAMs. The caveat is that all possible indices for a match value must be possible in an array even if the lookup value is ignored. For instance, the GRE version number is a three bit field, so a version number lookup could be implemented as an array with eight elements. Version 0 and 1 of GRE are defined so that elements in the array would be populated with node addresses, the other elements would be populated with a parser code indicating no match.
Given these limitations and tradeoffs, some general guidance can be provided:
If the lookup is on a 16-bit value or is common amongst multiple instructions, then a shared CAM table should be considered
If the lookup is on a small value, say up to 4-bits, then using an array should be considered
If the lookup is a “one off” for an instruction and the lookup is on eight bits or less then a non-shared CAM table should be considered
The non-shared table selector is 8-bits so in a large program with several non-shared tables the chances of key collisions may be high. As discussed above, inserting nop's is one mitigation. Another possibility is to increase the key size which would require hardware implementation. For instance, a 21-bit key might be used to increase the selector size to twelve bits. This is illustrated in
This section provides an example to illustrate the behavior and semantics of the critical parameters for hardware parsing. These fundamental parameters are in the registers: CurHdr.Offset (phoff), pcurptr pseudo register (PktHdrBase+CurHdr.Offset), CurHdr.Length (phlen), pdatptr pseudo register (PktHdrBase+DataHdr.Offset), DataHdr.Length (pdlen), DataHdr.Offset (pdoff), and DataBndLoop.DataBound (pdbnd).
The assembly for the example is listed below (line numbers are in blue). In this example there are four parse nodes: ether_node, ipv4_node, ip_option_node, and tcp_node. For this example, metadata extraction is omitted, and it is assumed that two protocol tables are populated where for shared table #1 an EtherType lookup is performed and there is once entry that maps IPv4 EtherType to ipv4_node, and for shared table #2 there is one entry that maps TCP protocol number to tcp_node. For the IP options lookup, a PC table is used and it may be assumed that the table is empty and all IP options are just parsed and otherwise ignored.
To illustrate the flow, we assume a TCP/IPv4 packet is input to the parser with one IPv4 option having eight bytes length. There is no TCP data so the total length of the packet is sixty-two bytes and it's assumed that the whole packet is received such that PktLen.ParseLen equals sixty-two. When the parser runs for such a packet, thirteen instructions are executed and have the following order per the line numbers: 2, 3, 5, 6, 7, 8, 9, 12, 13, 9, 10, 15, 16. The register states for key points in processing is described below where Point X references the instructions duly annotated above. This is covered by
Point 0: Initial state when ether_node is called. The initial state for parsing a packet is that CurHdr.Offset, DataHdr.Offset, CurHdr.Length, and DataHdr.Length are set to zero. Pseudo registerers pcurptr and pdatptr (pseudo registers) are logically set to PktHdrBase by virtue of setting CurHdr.Offset and DataHdr.Offset to zero. DataBndLoop.DataBound is set to infinity (−1ULL).
Point 1: After prs.load.h paccum, pcurptr+12
When the halfword load is performed at offset twelve, which loads the EtherType field from the Ethernet header, the expanse of bytes being loaded exceeds the CurHdr.Length but not the packet length. CurHdr.Length is incremented by the end of the data being loaded minus its current value. In this case CurHdr.Length is set to 14.
Point 2: After prs.cam.h.stp pnext, paccum[0], 1 (at ipv4_node)
.stp indicates a transition to the next node and the pointers and offsets are advanced to the next node and the lengths are reset. In this case, CurHdr.Length was equal to fourteen, the length of an Ethernet header, so CurHdr.Offset is set to fourteen as well as DataHdr.Offset, pcurptr, and, pdatptr are updated accordingly. CurHdr.Length and DataHdr.Length are set to zero, and DataBndLoop.DataBound is set to infinity.
Point 3: After prs.lensetmin.n pcurhdr, paccum[1], 4:20
lensetmin indicates both a minimum constant header length and a variable header length which is derived from a length field in the packet. In the case of IPv4, the minimum header length is twenty and the variable length is computed from the second nibble of the header multiplied by four. For this example, the value in the second nibble is seven which makes the length of the IPv4 header 28 bytes. CurHdr.Length is set to the computed variable length, that is 28 for this example. DataHdr.Offset is set CurHdr.Offset plus the minimum length, so in this example DataHdr.Offset is set to 34. pdatptr (pseudo register) is adjusted to reflect the new DataHdr.Offset. DataBndLoop.DataBound is set to the new CurHdr.Length minus the minimum length which equals eight in this example. After this instruction completes pdatptr, DataHdr.Offset, and DataBndLoop.DataBound are primed to commence processing the IP options.
The program continues through the prs.loadtlvloop and prs.camjump instruction to reach the ip_node_node. The next instruction to affect the parser offsets is then prs.lensettlv at point 4.
Point 4: After prs.lensettlv.b.stp pdathdr, paccum[1]
(before .stp processing is applied)
lensettlv determines the length of a non-padding option being processed by inspecting the length field in a sub-register. The IP option length is 8 bytes, so DataHdr.Length is set to eight.
Point 5: After prs.lensettlv.b.stp pdathdr, paccum, 0
(after .stp processing is applied)
When .stp processing occurs for a TLV, the data pointer, offset, length, and data bound are set for processing the next TLV. DataHdr.Offset is advanced by the value in DataHdr.Length making DataHdr.Offset equal to 42 in this example, and DataBndLoop.DataBound is reduced by the value in DataHdr.Length so in this example DataBndLoop.DataBound is set to zero. pdatptr is set accordingly, and DataHdr.Length is set to zero. At this point, the next option can be processed, however in this example DataBndLoop.DataBound is now zero which indicates there are no more IP options to process.
Point 6: After second execution of
prs.loadtlvloop paccum, pdatptr (at tcp_node)
At the second iteration of loadtlvloop, DataBndLoop.DataBound is zero indicating the end of options has been reached. In this example there is no post loop processing (PostLoop is assumed to be NULL) so the pointers offsets, and lengths are set up to process the next node. In this example, CurHdr.Length was equal to 28, the computed variable length of the IPv4 header, plus the original CurHdr.Offset value of 14 makes the new CurHdr.Offset set to 42. DataHdr.Offset, pcurptr and pdatptr are updated accordingly. CurHdr.Length and DataHdr.Length are set to zero, and DataBndLoop.DataBound is set to infinity.
Point 7: After prs.lensetmin.n.stp pcurhdr, paccum, 4:20
lensetmin computes the variable length of the TCP header with a minimum constant check that the TCP header is at least twenty bytes. In this example, the constant length and computed length of the TCP header are both twenty bytes so CurHdr.Length is set to twenty. The data pointer, offset, and data point are set accordingly and in this example DataHdr.Offset is set to 62, DataHdr.Length and DataBndLoop.DataBound are both zero. In this example packet there are no TCP options, and this simple program doesn't process them anyway. When .stp processing is performed for this instruction, there is no Next set so the parser terminates normally with STOP_OKAY.
Below is an example of a simple parser in parser instructions. This parse is composed of four nodes:
ether_node: Parses the Ethernet header and extracts the EtherType into metadata. It then performs a CAM lookup on the Ethernet using share table #1
ipv4_node: If Ethertype is IPv4, then the IPv4 header is parsed. First the IP version number is checked to equal four. A length check is performed that the minimum length is twenty bytes and determines the variable length from the IPv4 header. The source and destination addresses are extracted to metadata and the IP protocol field is extracted and CAM lookup is performed on the value using share table #2
ipv6_node: If Ethertype is IPv6, then the IPv6 header is parsed. First the IP version number is checked to equal six. The source and destination addresses are extracted to metadata and the next header is extracted and CAM lookup is performed on the value using share table #2. Note that setting the header length to twenty bytes is performed implicitly by load in the destination address
ports_node: If the IP protocol or next header is UDP or TCP (as set in share table #2) then the port numbers are extracted to metadata. The port numbers occupy the first four bytes of the transport layer header, so a single four byte load is performed that also implicitly verifies there is at least four bytes of length for the header in the packet.
The SiPanda Hardware parser is an integral component in the SDPU architecture. The parser provides two outputs: metadata and requests to schedule worker threads.
Metadata is any information derived for parsing a packet including values of protocol fields, offsets of protocol headers, lengths of protocol headers, and general packet information such as packet length and a receive timestamp. Metadata is saved into a metadata block of memory via the prs.store* instructions. The saved metadata is consumed by downstream processing by reading the memory containing metadata.
Requests to schedule worker thread is accomplished by the prs.runtrhead instruction. This instruction allocates a work item object in memory, via an external object allocator. Work items are sixty-four byte structures that are overlaid onto the first eight parser registers (p0 to p7).
As depicted in the diagram, the Parser is architecturally positioned between the Cluster Front End and the Cluster Scheduler. The input to the parser are work items from the Cluster Front End that provide the information needed for parsing received packets. A work item from the Cluster Front End is sent in a PARSER_START_MSG message on the clusfend_to_parser_fifo FIFO. A work item includes a reference to the packet context which includes the parsing buffer holding the first N bytes of data and a Metadata block. The Parser parses the headers in the parsing buffer and writes metadata to the Metadata block.
When the parser receives a PARSER_START_MSG message, a lookup is performed to determine which parser program to run. The work item from the Cluster Front End contains a parse function number that the parser uses to lookup in a table. The returned value is that address of the program that the parser runs.
As the parser runs, the program schedules worker threads by invoking the prs.runtrhead instruction. This instruction allocates a thread work item object in cluster local memory via an object allocator. These work items are sixty-four byte structures that are overlaid on the first eight parser registers. Once the work item is allocated, a block copy is performed of the first eight parser registers. Effectively, this is taking a snapshot of the current parser state that is needed for running the work thread (e.g. the pointer to the base address of the packet headers, the offset and length of the current header being processed, the pointer to the current metadata block, etc.).
The thread work item created by prs.runthread is processed as follows:
If PendingWork.PendingWork is equal to 0xFFFF then this is the first thread scheduled for a packet. prs.runthread sets PendingWork.PendingWorkto the index of the thread work item
Else, if PendingWork.PendingWork is not equal to 0xFFFF then this is not the first thread for the packet. The parser creates a START_THREAD_MSG message with a reference to the work item in cluster local memory. The message is sent to the cluster scheduler on the parser_to_clussched_fifo. When the cluster scheduler receives the message it can schedule a thread to process the work item
When the parser completes and PendingWork.PendingWork is not equal to 0xFFFF then the parser creates a LAST_THREAD_MSG message with a reference to the work item in cluster local memory. The message is sent to the cluster scheduler on the parser_to_clussched_fifo. When the cluster scheduler receives the message it can schedule a thread to process the work item and also marks the thread set as parsing complete. A bit in this last work item also can indicate that the cluster scheduler should close the thread set for the packet.
When the parser completes parsing a packet, PendingWork.PendingWork is set to 0xFFFF in preparation for parsing the next packet
The parser processes two types of work items: it sends thread work items and receives cluster work items.
Thread work items are sent from parser to cluster scheduler on the pars_to_clussched_fifo in messages of type START_THREAD_MSG or LAST_THREAD_MSG; these describe the request for processing a protocol layer in a worker thread. Note that these work items are overlaid on the first eight parser registers (this facilitates a simple block store for the register file to initialize a thread work item).
Cluster work items are sent from the cluster from end to the parser in START_PARSER_MSG type messages on the clusfend_to_pars_fifo; these describe a packet that is to be parsed by the parser.
The parser receives messages from the cluster front end via the clusfend_to_pars_fifo. The expected message type is PANDA_SDPU_CLUSFEND_TO_PARSER_START_MSG. The structure of the messages is:
pfunc is the parser function number, this indicates which parser program to run. work is a reference to the packet work item; values from the work item are used to initialize the parser registers for each packet. seqno is a sequence number used to ensure proper ordering of messages when there are multiple parsers; the parser does not process this and just passes it in thread work items sent to the cluster scheduler.
Packet work items reside in cluster shared memory. The address of the work item in shared memory is computed by:
The parser sends messages to the cluster scheduler via the pars_to_clussched_to_pars_fifo. The two message types are PANDA_SDPU_PARS_TO_CLUS_START_THREAD_SET and PANDA_SDPU_PARS_TO_CLUS_THREAD_SET_CLOSE_MSG. Both of these are sent as a result of prs.runthread being invoked. The first is sent to start a thread for a thread set for all threads except the last one in the thread set; the second type is sent to start the last thread in the thread set (i.e. the last thread closes the thread set). The structure of these message is:
seqno is the parser sequence number and id just copied from the packet work item received by the cluster scheduler, work is a reference to the thread work item, hash is the return result of a hash function being called for the last thread in the thread set (that is the hash value is only set in when message type is PANDA_SDPU_PARS_TO_CLUS_THREAD_SET_CLOSE_MSG).
PANDA_SDPU_PARS_TO_CLUS_THREAD_SET_START_MSG=1
PANDA_SDPU_PARS_TO_CLUS_THREAD_SET_CLOSE_MSG=12
PANDA_SDPU_CLUSFEND_TO_PARSER_START_MSG=12
In the SDPU, the parser, as all the other components, is driven by an event loop. The event loop could be implemented in instructions or hard logic for highest performance.
The PEVENTLOOP instruction (see PEVENTLOOP description above) is used to initiate an event loop. The pseudo code for the PEVENTLOOP instruction is:
The PEVENTLOOPEND instruction (see PEVENTLOOPEND description above) is used to handle the end of an iteration of an event loop. The instruction would normally be run by PEVENTLOOP setting the return address register to point to an prs.endloop instruction.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated. The present invention according to one or more embodiments described in the present description may be practiced with modification and alteration within the spirit and scope of the appended claims. Thus, the description is to be regarded as illustrative instead of restrictive of the present invention.
This application is a continuation in part which claims priority to U.S. patent application Ser. No. 17/233,149 filed Apr. 16, 2021 which claims priority to U.S. Provisional Patent Application No. 63/011,002 filed Apr. 16, 2020 which is incorporated in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63011002 | Apr 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17233149 | Apr 2021 | US |
| Child | 18762396 | US |
