The present invention relates generally to data communication, and particularly to methods and systems for efficient packet parsing.
In various communication devices and applications, the headers of packets contain information indicative of how the packets should be processed and forwarded.
Methods for parsing packets are known in the art. For example, U.S. Pat. No. 7,623,468 describes a parsing processor that provides instruction-driven content inspection of network packets at 10-Gbps and above with a parsing engine that executes parsing instructions. A state-graph unit traces state-graph nodes to keyword indications and/or parsing instructions. The parsing instructions can be derived from a high-level application to emulate user-friendly parsing logic. The parsing processor sends parsed packets to a network processor unit for further processing.
As another example, in a paper by Glen Gibb et al. entitled “Design Principles for Packet Parsers,” ANCS '13: Proceedings of the ninth ACM/IEEE symposium on Architectures for networking and communications systems, October 2013, Pages 13-24, the authors describe trade-offs in parser design, identify design principles for switch and router designers, and describe a parser generator that outputs synthesizable Verilog that is available for download.
An embodiment of the present invention that is described herein provides a parsing apparatus including a packet type identification circuit and a parser. The packet-type identification circuit is to receive a packet to be parsed, and to identify a packet type of the packet by extracting a packet-type identifier from a defined field in the packet. The parser is to store one or more parsing templates that specify parsing of one or more respective packet types. When the packet type of the packet corresponds to a parsing template among the stored parsing templates, the parser is to parse the packet in accordance with the stored parsing template. When the packet type of the packet does not correspond to any of the stored parsing templates, the parser is to parse the packet using an alternative parsing scheme.
In some embodiments the parser includes (i) a first parsing circuit to parse packets in accordance with the stored parsing templates, and (ii) a second parsing circuit to parse, using the alternative parsing scheme, packets that do not match any of the stored parsing templates. In a disclosed embodiment, the parser is to parse packets in accordance with the stored parsing templates at a first parsing rate, and to parse packets using the alternative parsing scheme at a second parsing rate that is slower than the first parsing rate.
In an example embodiment, the defined field in the packet, which holds the packet-type identifier, is a Layer-1 Start-of-Packet (SOP) field. In another embodiment, the defined field in the packet, which holds the packet-type identifier, is a Layer-2 Ethertype field. In some embodiments, the parsing template includes one or more anchors that specify locations of one or more header fields, and the parser is to parse the packet in accordance with the anchors.
There is additionally provided, in accordance with an embodiment of the present invention, a network device including a host interface and a packet processor. The host interface is to receive packet data and packet-type indications from a host. The packet processor is to send packets corresponding to the packet data to a network, including writing, in at least some of the packets, packet-type identifiers to a defined field in the packets, responsively to the packet-type identifications.
There is also provided, in accordance with an embodiment of the present invention, a parsing method including receiving a packet to be parsed. A packet type of the packet is identified by extracting a packet-type identifier from a defined field in the packet. When the packet type of the packet corresponds to a stored parsing template, the packet is parsed in accordance with the stored parsing template. When the packet type of the packet does not correspond to any stored parsing template, the packet is parsed using an alternative parsing scheme.
There is further provided, in accordance with an embodiment of the present invention, a method for packet parsing. The method includes storing one or more parsing templates in a memory, each parsing template specifying how to parse packets of a respective packet type. A packet to be parsed is received. A packet type, to which the received packet belongs, is identified by extracting a packet-type identifier from a field in the packet. If the packet type of the received packet, as indicated by the extracted identifier, corresponds to one of the stored parsing templates, the packet is parsed in accordance with the corresponding parsing template. If the packet type of the received packet does not correspond to any of the stored parsing templates, the packet is parsed using an alternative parsing scheme.
The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
Various types of network devices and applications perform packet parsing for determining the processing and forwarding required for the packet. Parsing typically involves identification of certain fields in a header of the packet, e.g., fields related to the underlying communication protocols used.
In modern network devices, parsers are typically required to support a large variety of packet types. Moreover, increasing bandwidth demands such as in data centers and in High-Performance Computing (HPC) increase parsing rate demands accordingly.
Header fields of the packet header may indicate various packet parameters. As an example, a packet in an Ethernet network may have a packet type denoted MAC-IPV6 or MAC-IPV4, for example. As another example, a packet encapsulated in a Virtual Local Area Network (VLAN) may have a packet type denoted MAC-VLAN-IPV4-UDP. In specifying a packet type, a header field of one protocol typically points to the next field of another protocol.
A parser supporting a large variety of packet types may identify a packet type by scanning the packet header in accordance with a predefined graph in which the nodes correspond to the supported packet types. Parsers of this sort, however, are typically implemented using a Finite-State Machine (FSM), which is not fast enough to meet parsing rate demands.
In principle, parsing rate could be increased, for example, by assigning multiple parsing-engines to different subsets of the packets belonging to a common packet stream. This solution would typically involve packet reordering before or after parsing. Another approach for increasing the parsing rate could be based on parsing packets on demand, e.g., parse only when a certain field of a packet header is needed. The main disadvantages of the example approaches described above include increased complexity, increased latency and high cost in terms of ASIC area and power consumption.
The packet types of packets traversing a communication network are typically distributed nonuniformly. Characteristics of transport protocols used in data centers are described, for example, by Jun Cheng & Karl-Johan Grinnemo, in paragraph 4.2 of a survey entitled “Telco Distributed DC with Transport Protocol Enhancement for 5G Mobile Networks,” Karlstad University, Faculty of Health, Science and Technology, December 2017.
The inventors discovered experimentally that traffic in real-life networks may be dominated by only a small subset of the entire packet types supported. For example, a field experiment showed that 99% of the packets in some ports had a MAC-VLAN-IPV6-TCP type. In other ports, 70% of the packets had a MAC-VLAN-IPV6-UDP type (possibly with high-layer protocols that need not be parsed for forwarding), and 30% had a MAC-VLAN-IPV6-TCP type.
Parsing of packets is done predominantly by parsing the packet headers (e.g., extracting in the packet header). We will refer hereinbelow to “header parsing” and “packet parsing” interchangeably. We will further refer interchangeably to the terms “packet type” and “header type” (i.e., the various protocols indicated by the header fields).
Embodiments of the present invention that are described herein provide improved methods and systems for packet parsing. Based on the assumption that, in a given network, a small number of header types that can be a-priori defined dominates most of the traffic (“frequent headers”); in some embodiments an initiating network device, such as a Network Interface Controller (NIC), marks the frequent headers by changing a reserved field in the packet header. In embodiments, the initiating network device further encodes the header type of the frequent headers in reserved fields of the packet header. The reserved header fields may be, in an embodiment, an L1 Start-of-Packet (SOP) field and/or an L2 Ethertype field.
According to some embodiments, a network device that receives communication packets (e.g., a network switch or a NIC at a destination node) may comprise a template-based parser (TBP); the TBP receives packet headers and identifies marked frequent headers. If the header is marked, the TBP parses the header at high-speed, using a predefined template that is selected responsively to the header type (the header type is encoded using reserved combinations of pre-defined header fields). If the header is not marked, the TBP parses the header using an alternative “conventional” parser (which typically takes much longer).
In some embodiments, the TBP parses marked headers using a template, and unmarked headers using a “conventional” parser, at the same time, for yet higher parsing rates. In other embodiments the TBP completes the parsing of a non-marked header before proceeding to parse a marked header, and vice-versa.
In embodiments, each parsing template comprises anchors for the various header fields, and, hence, template-parsing comprises the selection of bit fields in the header that start and end at anchor values independently of each other; thus, bit-field extraction may be done in parallel for all bit fields of the header (although, in practice, as the header received serially, bit-field extraction for some of the fields may be delayed. until the network device receives the respective fields)
According to an embodiment, an initiating network device comprises a packet builder; the packet builder receives, from a host, the header-parameters, and the payload of packets to be sent, over the network, and, for frequent headers, a frequent header indication and a packet-type. The packet builder builds a packet header according to the header-parameters; if the header is a frequent header, the packet builder encodes the packet type in one or more reserved bit fields of the header.
Thus, according to embodiments, fast header parsing can be achieved in networks by marking frequent headers at the initiating network device, and template-parsing the frequent headers by the receiving network device.
Universal packet parsing, which parses ingress packets according to fields in the packet headers, may be complex and take a long time to execute. For example, practical universal parsing may be hierarchical, in the sense that the parsing of a first header field is needed to find an anchor to a second header field, and so on.
An example parsing technique is described in U.S. patent application Ser. No. 17/160,407, filed Jan. 28, 2021, which is assigned to the assignee of the present patent application and whose disclosure is incorporated herein by reference. The application describes a parsing apparatus including an interface, which is configured to receive packets belonging to a plurality of predefined packet types; a first parser, which is configured to identify any of the packet types; a second parser, which is configured to identify only a partial subset of the packet types; and a controller, which is configured to receive a packet via the interface, to attempt identifying a packet type of the received packet using the second parser, and in response to detecting that identifying the packet type using the second parser fails, to revert to identify the packet type of the received packet using the first parser.
Embodiments according to the present invention benefit from the observation that, in practical network systems, many of the packet headers use a limited number of packet header types. In embodiments, such frequent headers are marked by a source network device (“the initiating network device”), using reserved fields in the packet header. In an embodiment, the marking may include a code that identifies a header type. A destination network device then identifies the marked packet headers, parses the marked headers using a fast template-based parser, and the non-marked headers using a “conventional” parser (e.g., a hierarchical parser).
Network 100 comprises switches 106 that route communication packets through the network. Some (or all) of the switches may include circuitry to detect the marked packet headers, parse such headers using a fast template-based parser and parse other headers using an alternative, slower parser.
Receiving NICs 108 may receive communication packets from the network. At least some of the receiving NICs may include, similarly to switches 106, circuitry to detect the marked packet headers, parse such headers using a fast template-based parser and parse other headers using an alternative parser. Like initiator NIC 104, receiving NIC 108 may be replaced, in alternative embodiments, with any other suitable network-connected device that receives communication packets from the network; for example, a Host Channel Adapter (HCA) in InfiniBand™ network.
The example embodiment illustrated in
Serial-TSP 200 receives packets, including packet headers 202, (e.g., from a network port of a network device) and temporarily stores the headers in a rate-matching queue 204. A packet-type identification circuit 206 then reads the packet headers from queue 204, and detects, by inspecting predefined header fields, if the headers are marked. If the headers are marked, the packet-type identification circuit sends the packet header and a packet-type indication to a fast template parser 210, which parses the headers using preset templates. If the headers are not marked, the packet-type identification circuit sends the packet headers to a slower alternative parser 208, (e.g., a hierarchical parser). As most of the packet headers are expected to be marked, most of the header parsing will be done by the template-based parser, and the average parsing rate will be significantly increased. An example implementation of a fast template parser is shown in
According to the example embodiment illustrated in
The parallel-TBP receives packet headers 252 (e.g., from a network port that is not shown), and temporarily stores the packet headers in a rate-matching queue 254. Packet identifier 256 identifies marked and unmarked headers and sends the headers (with header-type codes) accordingly, to a template-based-parser in-queue (TB-P in-queue) 258, or to an Alternative Parser in-queue (ALT-P in-queue) 260. Unlike header-type identification circuit 206 (
A template parser 262 parses the marked headers according to predefined templates and sends the parsed headers to a template-based parser out-queue (TB-P out-queue) 264, whereas an alternative parser 266 parses the non-marked headers, (e.g., hierarchically), and sends the parsed headers to an alternate-parser out-queue (ALT-P out-queue) 268. Lastly, a selector 270 reads parsed headers from queues 264, 268, and sends the parsed headers for further processing.
Thus, according to the embodiments illustrated in
The embodiments illustrated in
As another example, the template parser (210 or 262) and the alternative parser (208 or 266) need not necessarily be implemented as two separate units. More generally, the disclosed techniques can be described as carried out by a parser having two different parsing schemes—(i) a template-based parsing scheme used for packets of types having corresponding parsing templates, and (ii) an alternative parsing scheme for packets that do not correspond to any of the stored parsing templates. The present disclosure refers to “two parsers” and to “a parser having two parsing schemes” interchangeably.
The parsed headers according to the example embodiment illustrated in
The template parser receives packet headers and packet type indications from header-type identification circuit 206 or 256 (
The selected parsing template is input to a field extractor 306 that extracts the header fields according to the templates, and outputs the parsed header (e.g., the separate header fields) for further processing, either directly (
The template comprises a Number-of-Header-Fields field 402, which defines the number of header fields in the corresponding header type, and an Anchor field 404 for each of the header fields. In some embodiments, anchor fields 404 also comprise an anchor-type field (not shown in the figure, for clarity).
The extraction of the header fields from the header may be done in parallel for all header fields; assuming the header fields are contiguous, header-field i is a bit-field in the header, starting at Anchor(i-1) and ending at Anchor(i)-1. In practice, however, since the network device receives the headers serially, the extraction of some of the fields may be deferred until the fields are input to the network device.
The structure of template 304 illustrated in
A SOP symbol is transmitted by the PHY layer to indicate start-of-frame. Typically, the SOP is fixed for a given protocol; however, in proprietary environments (e.g., a High-Performance Computing (HPC) architecture), other SOP symbols may be used.
EtherType is a 16-bit field in an Ethernet frame, used to indicate which protocol is encapsulated in the payload of the frame and for VLAN tagging. Some of the 216-bit combinations of the Ethertype field are not used (i.e., illegal, reserved, or undefined). (Ethertype is used in Ethernet; other fields may be used in other protocols.)
In the example embodiment illustrated in
In embodiments, detect circuit 508 and/or decoder 510 includes dedicated fast circuitry that extracts the SOP and Ethertype fields without fully parsing the header. For example, when SOP is used, SOP is always the first layer-1 byte and hence easily extracted. For another example, when Ethertype is used, as the Ethertype is always sent after the Virtual Local Area Network (VLAN) tagged packets, the fast circuitry will detect the number of VLAN tags.
The structure of marked packet header 502 illustrated in
The flowchart starts with a get-packet-header operation 602, wherein the TBP receives the header of the packet to be parsed (e.g., header 202,
If the SOP code in operation 604 or the Ethertype code in operation 606 are reserved codes, the TBP enters a Select Template operation 610, and selects a template that corresponds to the header type, which is determined responsively to The SOP and the Ethertype codes. The TBP will then, in a template-parse operation 612, use the selected template to rapidly parse the header.
After operations 608 and after operation 612, the TBP reenters operation 602, to parse the next header.
Thus, according to the example embodiment illustrated in
The structure of flowchart 600 illustrated in
In embodiments, template parsing 612 and alternative parsing 608 may be done in parallel (for different headers), as disclosed with reference to
The advantages of parsing using the disclosures described above will now be demonstrated, using a numerical example.
Let Tt and Ta denote the average heard parsing time (e.g., in nano-second/header), of template parser 210 and alternative parser 208, respectively (
Let Ft and Fa denote the relative frequencies of, respectively, marked and unmarked headers (Ft+Fa=1).
In an example application, we assume that the relative occurrence frequencies of header types are as follows:
Type 0—50%
Type 1—30%
Types 2 to 6—12% (total)
Other Types—8%
We further assume that the template parser is, on the average, four times faster than the alternative parser (that is—Ta/Tt=4).
If only types 0 and 1 (80% of the headers) are marked, we get parsing time of
0.8*Tt+0.2*Ta=0.4*Ta, or a 2.5 speed-up ratio (over an alternative parser only).
If all types 0 through 6 are marked (and, hence, template-parsed), we get a parsing time of:
0.92*Tt 0.08*Ta=0.31*Ta, or an approximately, 3.2 speed-up ratio.
It should be noted that, in addition to the speedup, silicon area (and, hence, power consumption) is saved. Alternatively, the saved silicon area/power consumption may be used to implement additional circuitry, e.g., for yet faster parsing.
We described below, with reference to
Network device 700 may receive from a host, through host interface 704, instructions to send a packet over the network. The instructions may include packet parameters such as a destination address, a source address, etc., and the packet payload. The network device temporarily stores the instructions in a queue 706.
According to the example embodiment illustrated in
When ready to build a next packet, the packet builder reads the instructions from queue 706, and builds a suitable packet header 710 (e.g., in a register). The packet header includes fields to store a source address, a destination address and other header fields, as indicated by the host instruction that the packet builder reads from the queue. If the header is a frequent header (and, hence, stored in the frequent-header table 707), the packet builder encodes the header types in L1 SOP field 718 and/or in L2 Ether type field 720.
When the header is ready, the network device sends the header, followed by the payload (from queue 706) through port 702, to the network. The packet builder may calculate a packet footer (e.g., Cyclic Redundancy Code-CRC), and append the footer to the payload.
Thus, a network device according to the example embodiment illustrated in
The structure of network device 700 is cited by way of example. Other structures may be used in alternative embodiments. For example, in some embodiments, a host prepares the entire packet, including the packet header. In another embodiment, the network device does not include a frequent-headers table and, instead, the host may add a frequent-header (and frequent header-type) indication to the headers that the host sends to the network device. In an embodiment, the packet footer is neither calculated and appended by the packet builder but, rather, calculated and appended by other circuits of the network element (e.g., by port 702).
In embodiments, packet building may be accelerated using a technique that is similar to the packet parsing techniques described above—the packet builder may include template headers, to be used for marked headers; if the host indicates that the header should be marked, the packet builder, rather than building the header, will only fill the various fields with data provided by the host.
The flowchart starts at a get list of frequent headers operation 801, wherein the packet builder gets a list of frequent header types from, the host and stores the list in table 707 (
Next, at a check-marked-header operation 806, the packet builder compares the header to the list of frequent headers, to check if the header should be marked. If so, the packet builder enters a build-header-with-encoded-fields operation 808, builds the packet-header and fills the various fields according to the header parameters that the host specifies; the packet builder also encodes the L1 SOP field and/or the L2 Ethertype field with reserved codes, responsively to the specified header type (all host parameters are typically received from a host, via host interface 704 and queue 706).
If, in operation 806, the header is not marked, the packet builder will enter a build-header-with-default-fields operation 810 and builds a header according to the header parameters and using non-reserved values for the SOP and Ethertype fields.
After either operation 808 or operation 810, the packet builder, in a send-header operation 812, sends the header, through port 702 (
After operation 816, the packet builder reenters operation 802, to send the next packet.
The structure of flowchart 800 illustrated in
The configuration of initiator NIC 104, switches 106, receiving NIC 108, serial-TBP 200, parallel-TBP 250, template parser 300, parsing template 304, marked-header decoding circuit 500, template-based parsing method 600, initiator network device 700 and templated packet generation method 800, are example configurations and methods that are shown purely by way of illustration. Any other suitable configurations and methods can be used in alternative embodiments.
In various embodiments, the template-based parsing circuitry of the receiving network device, and/or, the packet building circuitry of the initiating network device described hereinabove may be carried out by hardware, by software, or by a combination of hardware and software.
In various embodiments, the different elements of initiator NIC 104, switch 106 and receiving NIC 108, may be implemented using suitable hardware, such as one or more Application-Specific integrated Circuits (ASIC) or Field-Programmable Gate Arrays (FPGA), or a combination of ASIC and FPGA.
Each of initiator NIC 104, switch 106 and receiving NIC 108 may include one or more processors; the processors typically comprise one or more general-purpose processors, which are programmed in software to carry out at least part of the functions described hereinabove. The software may be downloaded to the processor in electronic form, over a network, for example, or it may, alternatively or additionally, be provided and/or stored on non-transitory tangible media, such as magnetic, optical, or electronic memory.
It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and sub-combinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art. Documents incorporated by reference in the present patent application are to be considered an integral part of the application except that to the extent any terms are defined in these incorporated documents is a manner that conflicts with the definitions made explicitly or implicitly in the present specification, only the definitions in the present specification should be considered.
Number | Name | Date | Kind |
---|---|---|---|
5394394 | Crowther et al. | Feb 1995 | A |
6308219 | Hughes | Oct 2001 | B1 |
6356951 | Gentry, Jr. | Mar 2002 | B1 |
6504842 | Bergenfeld | Jan 2003 | B1 |
6705533 | Venkatraman et al. | Apr 2004 | B2 |
6788680 | Perlman et al. | Sep 2004 | B1 |
6807172 | Levenson et al. | Oct 2004 | B1 |
7333484 | Henderson et al. | Feb 2008 | B2 |
7623468 | Panigrahy et al. | Nov 2009 | B2 |
7921046 | Parsons et al. | Apr 2011 | B2 |
8161468 | Todd | Apr 2012 | B2 |
8694448 | Mandelbaum et al. | Apr 2014 | B2 |
9209990 | Szabo et al. | Dec 2015 | B1 |
10015090 | Arad et al. | Jul 2018 | B2 |
10616111 | Immidi et al. | Apr 2020 | B1 |
10701190 | Kfir et al. | Jun 2020 | B2 |
10757230 | Kfir et al. | Aug 2020 | B2 |
20030043848 | Sonksen | Mar 2003 | A1 |
20030074388 | Pham et al. | Apr 2003 | A1 |
20040081101 | Bennett | Apr 2004 | A1 |
20040088430 | Busi et al. | May 2004 | A1 |
20040218528 | Shipman | Nov 2004 | A1 |
20050089016 | Zhang et al. | Apr 2005 | A1 |
20050108617 | Lappin | May 2005 | A1 |
20060062144 | Testa et al. | Mar 2006 | A1 |
20070140297 | Chung et al. | Jun 2007 | A1 |
20070276952 | Mitchell | Nov 2007 | A1 |
20080025308 | Morgan et al. | Jan 2008 | A1 |
20080095149 | Dai | Apr 2008 | A1 |
20090097418 | Castillo et al. | Apr 2009 | A1 |
20110149895 | Sala et al. | Jun 2011 | A1 |
20120159132 | Abel | Jun 2012 | A1 |
20130022128 | Symes | Jan 2013 | A1 |
20130128885 | Kardashov et al. | May 2013 | A1 |
20130215906 | Hidai | Aug 2013 | A1 |
20150081726 | Izenberg | Mar 2015 | A1 |
20150081833 | Pettit et al. | Mar 2015 | A1 |
20150156288 | Lu et al. | Jun 2015 | A1 |
20150172189 | Pitchai et al. | Jun 2015 | A1 |
20150256456 | Previdi et al. | Sep 2015 | A1 |
20150341265 | Basso et al. | Nov 2015 | A1 |
20160028860 | Rubin | Jan 2016 | A1 |
20160094667 | Jani | Mar 2016 | A1 |
20160139892 | Atreya et al. | May 2016 | A1 |
20160197852 | Hutchison et al. | Jul 2016 | A1 |
20160285759 | Ryan | Sep 2016 | A1 |
20170064047 | Bosshart | Mar 2017 | A1 |
20170168970 | Sajeepa et al. | Jun 2017 | A1 |
20170180253 | Koren et al. | Jun 2017 | A1 |
20170286292 | Levy et al. | Oct 2017 | A1 |
20180198809 | Kushwaha | Jul 2018 | A1 |
20180242218 | Muscariello et al. | Aug 2018 | A1 |
20180287941 | Louzoun et al. | Oct 2018 | A1 |
20180288197 | Izenberg | Oct 2018 | A1 |
20180288614 | Zaks et al. | Oct 2018 | A1 |
20190004701 | Guim Bernat et al. | Jan 2019 | A1 |
20190042489 | Nagabhushana | Feb 2019 | A1 |
20190073719 | Parsons et al. | Mar 2019 | A1 |
20190123984 | Nainar et al. | Apr 2019 | A1 |
20190132421 | Fedyak | May 2019 | A1 |
20190140979 | Levi et al. | May 2019 | A1 |
20190207860 | York | Jul 2019 | A1 |
20200293487 | Anderson | Sep 2020 | A1 |
20210176345 | Urman et al. | Jun 2021 | A1 |
20210306257 | Dutta | Sep 2021 | A1 |
20210328923 | Urman et al. | Oct 2021 | A1 |
Entry |
---|
Gibb et al., “Design Principles for Packet Parsers”, ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pp. 1-12, Oct. 2013. |
Deering et al., “Internet Protocol, Version 6 (IPv6) Specification”, Request for Comments: 2460, pp. 1-39, Dec. 1998. |
University of Southern California, “Darpa Internet Program”, Protocol Specification, Request for Comments 791, pp. 1-50, Sep. 1981. |
Cheng et al., “Telco Distributed DC with Transport Protocol Enhancement for 5G Mobile Networks,” working paper, Karlstad University, Faculty of Health, Science and Technology, pp. 1-58, Dec. 2017. |
Mula et al., U.S. Appl. No. 17/160,407, filed Jan. 28, 2021. |
Number | Date | Country | |
---|---|---|---|
20230127568 A1 | Apr 2023 | US |