This disclosure claims priority to China Patent Application No. 202010751254.2, filed on Jul. 30, 2020 in China National Intellectual Property Administration and entitled “FPGA-based FAST Protocol Decoding Method, Apparatus and Device”, the entire contents of which are incorporated herein by reference.
The present disclosure relates to the technical field of computers, in particular to a field-programmable gate array (FPGA)-based FAST protocol decoding method, apparatus and device, and a readable storage medium.
The domestic Shanghai and Shenzhen LEVEL-2 market is transmitted using the FIX Adapted For Streaming (FAST) protocol. FAST is a message data stream-oriented coding method with high compression rate and processing efficiency. FAST coding includes using an Extensible Markup Language (XML) information template for encoding, variable-length coded byte compression and the like.
In the prior art, a FAST protocol decoding scheme includes three parts: reading data, decoding fields, and outputting a decoding result, and intermediate results are buffered through a buffer. A data reading component is responsible for reading a coded value of a field from a FAST market data input stream buffer. A field decoding component is responsible for specific decoding according to rules of field operators. A result outputting component is responsible for outputting a decoded field value to a FIX message buffer. However, in each operation of its core FAST field decoding, the encoded value of at most one field is read, which has limited performance and low decoding efficiency. Moreover, in practical applications, an XML template may need to be modified dynamically. In this case, a control logic needs to be modified. This scheme cannot be applied to the modified XML template.
It can be seen that how to provide a decoding scheme of a FAST protocol to avoid the defect that only one field can be decoded at a time and a dynamically updated XML template is not supported is the problem that need to be solved by those skilled in the art.
The present disclosure aims to provide a FPGA-based FAST protocol decoding method, apparatus, and device, and a readable storage medium, so as to solve the following problems: the current FAST protocol decoding scheme can decode only one field at a time and does not support a dynamically updated XML template, so that the decoding efficiency is low, and the scenario applicability is low. The specific method is as follows:
In a first aspect, the present disclosure provides the FPGA-based FAST protocol decoding method, including:
In some embodiments, the existence condition of fields according to the structure of the XML template is the determined includes:
In some embodiments, according to the distribution of the fields in the XML template, the FAST protocol intermediate representation including the existence condition is generated includes:
In some embodiments, the byte data stream into field data streams is segmented, and the field data streams to the field shift register are buffered includes:
In some embodiments, corresponding fields in parallel from the field shift register according to the field matching state machine are read includes:
In some embodiments, the receiving a byte data stream of the FAST protocol by using the FPGA, segmenting the byte data stream into field data streams includes:
In some embodiments, before the field matching state machine according to the FAST protocol intermediate representation and preset decoding parameters is generated, the method further includes:
In a second aspect, the present disclosure provides an FPGA-based FAST protocol decoding apparatus, including:
In a third aspect, the present disclosure provides an FPGA-based FAST protocol decoding device, including:
In a fourth aspect, the present disclosure provides a readable storage medium, wherein the readable storage medium stores a computer program; and the computer program, when executed by a processor, implements the steps of the above FPGA-based FAST protocol decoding device.
The FPGA-based FAST protocol decoding method provided by the present disclosure includes: the XML template of a FAST protocol is acquired, the existence condition of fields according to the structure of the XML template is determined, and according to the distribution of the fields in the XML template, the FAST protocol intermediate representation including the existence conditions is generated; the field matching state machine according to the FAST protocol intermediate representation and preset decoding parameters is generated, wherein the decoding parameters include the maximum number of fields processed in each clock cycle; the field matching state machine is configured to describe fields read at each time in the decoding process; the number of fields read at the single time is less than or equal to the maximum number of fields, and the existence conditions of the fields read at the same time are independent of each other; the byte data stream of the FAST protocol by using an FPGA is received, the byte data stream into field data streams is segmented, and the field data streams to the field shift register are buffered; and corresponding fields in parallel from the field shift register according to the field matching state machine are read, and the corresponding fields read in parallel from the field shift register are decoded.
To sum up, for the problems of low supportability for a dynamically updated XML template and failure of adapting to different network bandwidths at the present, by means of an actual XML template in real time is acquired and the actual XML template is analyzed, the FAST protocol intermediate representation is generated, and according to preset decoding parameters, the maximum number of fields is determined which are read at a single time, so as to generate a field matching state machine, the method can support a dynamically updated XML template, and allows flexible setting of the maximum number of fields according to an actual network bandwidth. In a decoding process, the method realizes a function of decoding a plurality of fields at a time by means of the field shift register and the field matching state machine. In an embodiment, after an input byte stream is segmented into fields, the fields are stored in the field shift register. The fields with the number less than or equal to the maximum number of fields are read in parallel at each time through the field matching state machine and are decoded, which significantly improves the decoding efficiency.
In addition, the present disclosure further provides the FPGA-based FAST protocol decoding apparatus and device, and the readable storage medium, the technical effects of which correspond to the technical effects of the above method, and descriptions thereof are omitted here.
In order to describe the embodiments of the present disclosure or the methods in the prior art more clearly, drawings required to be used in the embodiments or the illustration of the existing art will be briefly introduced below. Obviously, the drawings in the illustration below are only some embodiments of the present disclosure. Those ordinarily skilled in the art also can acquire other drawings according to the provided drawings without creative work.
In order to make those skilled in the art better understand the solutions of the present disclosure, the present disclosure is further described in detail below with reference to the accompanying drawings and specific implementation modes. Apparently, the described embodiments are merely a part of the embodiments of the present disclosure and not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work all fall within the protection scope of the present disclosure.
An existing FAST protocol decoding scheme has the following defects:
First, in a decoding process, only one field can be read at a time. In a common network interface with a bandwidth of 10 G, an input FAST data stream often contains four fields at most. It can be seen that the existing FAST decoding scheme cannot give full play to the bandwidth advantage.
Second, there is a possibility of dynamic update of an XML template in the financial field. In the existing FAST decoding scheme, new decoding operators are manually mounted, and a control logic of a controller needs to be modified, so it takes a long time to modify and verify the control logic, and the update cannot be completed at the first time. It can be seen that the existing FAST decoding scheme does not support a dynamically updated XML template.
Third, for network interfaces with different kinds of bandwidths, including 1G/10G/25G/40G/100G, the existing FAST decoding scheme uses the same decoding method, which cannot flexibly adapt to different network bandwidths.
For the above problems, the present disclosure provides the FPGA-based FAST protocol decoding method, apparatus and device, and the readable storage medium. The method acquires an actual XML template in real time and analyzes the actual XML template, determines, according to preset decoding parameters, the maximum number of fields which are read at a single time, so as to generate a field matching state machine. Thus, the present disclosure can support a dynamically updated XML template, and allows flexible setting of the maximum number of fields according to an actual network bandwidth. Furthermore, after the input byte stream is segmented into the fields, the fields are stored in the field shift register. The fields with the number less than or equal to the maximum number of fields are read in parallel at each time through the field matching state machine and are decoded, which significantly improves the decoding efficiency.
Embodiment I of the FPGA-based FAST protocol decoding method provided by the present disclosure is described below. Referring to
At step S101, an XML template of a FAST protocol is acquired; an existence condition of fields is determined according to a structure of the XML template; and a FAST protocol intermediate representation including the existence condition is generated according to a distribution of the fields in the XML template.
At step S102, a field matching state machine is generated according to the FAST protocol intermediate representation and preset decoding parameters.
At step S103, a byte data stream of the FAST protocol is received by using an FPGA; the byte data stream is segmented into field data streams; and the field data streams are buffered to a field shift register.
At step S104, corresponding fields are read in parallel from the field shift register according to the field matching state machine, and the corresponding fields read in parallel from the field shift register are decoded.
The overall framework of this embodiment is as shown in
The first part is to generate the FAST protocol intermediate representation. The XML template of the FAST protocol is first read, and the structure of the XML template is analyzed to determine the existence condition of the various fields in the XML template; and the FAST protocol intermediate representation is generated according to the distribution of the fields in the XML template. In this embodiment, the FAST protocol intermediate representation contains the existence conditions of the fields.
The above existence conditions are used for describing a relationship between the existence of each FAST field and the previously received FAST field. In an embodiment, in practical applications, the existence of a field in the XML template may not be affected by other fields, or may be affected by an actual value of a field in front of this field. For these two cases, this embodiment describes the existence condition of each field.
The second part is to generate the field matching state machine of the FAST protocol. The field matching state machine is generated according to the FAST protocol intermediate representation and the preset decoding parameters. Wherein, the decoding parameters include the following information: the maximum number of fields processed in clock cycle. The field matching state machine is configured to describe fields read at each time in the decoding process, and satisfies the following conditions: the number of fields read at the single time is less than or equal to the maximum number of fields, and the existence conditions of the fields read at the same time are independent of each other.
Wherein, the decoding parameter refers to the maximum number of fields processed in each clock cycle. A specific value can be configured according to an actual application scenario, which can be 4, 8, 16, 32, 64 and other configurations. This embodiment does not limit the specific value.
Each existence conditions is independent of each other, so that whether a current field has this situation is not affected by other fields.
The main purpose of this part is to optimize FPGA parallelization and minimize an FPGA processing delay and processing logic. In an embodiment, the field matching state machine is configured to describe a certain number of fields that can be read in each reading process and have independent existence conditions, so as to achieve the purpose of reading multiple fields in parallel at each time and improve the decoding efficiency. As a preferred implementation, the field matching state machine can describe the maximum number of fields that can be read in each reading process and have independent existence conditions. It can be understood that the maximum number here also needs to satisfy the conditions of being less than or equal to the maximum number of fields mentioned above.
The third part is to segment and decode the byte data stream of the FAST protocol.
In an embodiment, a register transfer level (RTL) decoding code can be generated according to the foregoing field matching state machine, and then the whole decoding flow can be realized by running the RTL decoding code.
This embodiment provides the FPGA-based FAST protocol decoding method. The current problem of decoding at most one field at a time is solved by the field shift register and the field matching state machine. After the input byte stream is divided into the fields, the fields are stored in the field shift register, and the field matching state machine dynamically reads variable fields to achieve the processing ability of decoding multiple fields at a time, which significantly improves the decoding efficiency. For the problems that the current system has low supportability for dynamic template update and fails in adapting to different network bandwidths, in the process of generating the field matching state machine, the actual XML template is acquired in real time and the actual XML template is analyzed, and the maximum number of fields which are read at the single time is determined according to the preset decoding parameters, so as to generate a field matching state machine. Therefore, the method can support a dynamically updated XML template, and allows flexible setting of the maximum number of fields according to an actual network bandwidth.
To sum up, this embodiment at least includes the following advantages:
The following is a detailed description of Embodiment II of the FPGA-based FAST protocol decoding method provided in the present disclosure. Embodiment II is implemented on the basis of above Embodiment I, and has been expanded to a certain extent on the basis of Embodiment I.
In an embodiment, this embodiment describes in detail the process of determining the existence conditions of the fields. This embodiment also uses a parallel buffering manner in the buffering process, such as buffering with the current maximum number of fields. In addition, this embodiment can automatically set a corresponding decoding parameter according to a current network bandwidth.
Referring to
At step S301, the corresponding decoding parameter is determined according to a current network bandwidth, and an XML template of the FAST protocol is acquired in real time.
At step S302, existence conditions of fields are determined according to the structure of the XML template.
At step S303, decoding properties of the fields are determined according to the XML template; and a FAST protocol intermediate representation including the existence conditions and the decoding properties is generated according to the distribution of the fields in the XML template.
At step S304, the field matching state machine is generated according to the FAST protocol intermediate representation and preset decoding parameters.
At step S305, the RTL decoding code of the FAST protocol is generated according to the field matching state machine.
At step S306, the RTL decoding code is run on the FPGA, and the byte data stream of the FAST protocol is received; the byte data stream is segmented into field data streams according to the stop bit; and during buffering at each time, the fields with the maximum number of fields are buffered in parallel to the field shift register.
At step S307, field matching is performed according to the existence bitmap field; whether the various fields exist is determined; corresponding fields are read in parallel from the field shift register according to the field matching state machine, and the corresponding fields read in parallel from the field shift register are decoded.
A typical FAST protocol XML template UA3201 is shown in
After coding is performed according to the template, binary data received by an actual market is as shown in
During the generation of the intermediate representation, this embodiment first reads the XML template of the FAST protocol, analyzes the structure of the XML template, and finally generates the FAST protocol intermediate representation according to whether the structure contains an internal cycle, the existence conditions and decoding properties of the various fields, and the configured FAST decoding parameters. Therefore, as shown in
At step S601, when a target field is in a cycle structure in the XML template, it is determined that the target field has an existence relationship with a field, prior to the target field, in the XML template, which is taken as an existence condition of the target field; and
At step S602, when the target field is not in the cycle structure in the XML template, a target bit, corresponding to the target field, in an existence bitmap field is determined, which is taken as an existence condition of the target field.
For example, the FAST protocol intermediate representation corresponding to the XML template UA3201 is as follows, which describes the following information: The existence of each field is determined by a certain bit of the existence bitmap field:
A fragment of the XML template UA3202 of
For the relatively simple XML template of the FAST protocol, such as UA3201 shown in
Assuming that the maximum number of fields allowed by the current network bandwidth is four, the template UA3202 shown in
When the FPGA is used for decoding the FAST protocol, the input byte data stream is first segmented into field data streams according to the stop bit and buffered in the field shift register. The fields are dynamically matched according to the existing bitmap field of the XML template. The corresponding fields are read in parallel from the field shift register and are decoded to obtain FAST market data. The whole decoding flow is as shown in
It can be seen that this embodiment provides an FPGA-based FAST protocol decoding method, marked features of which include:
First, the FAST protocol intermediate representation is generated. First, the XML template of the FAST protocol is generated; a network structure of the FAST protocol is analyzed; and the FAST protocol intermediate representation format is generated according to whether the network structure contains an internal cycle, the existence and decoding properties of the various fields, and the configured FAST decoding parameters.
Second, the field matching state machine of the FAST protocol is generated. FPGA parallelization optimization is performed according to the FAST protocol intermediate representation and the FAST decoding parameters; the FPGA processing delay and processing logic are minimized, thus obtaining the field matching state machine of the FAST protocol.
Third, the RTL decoding code of the FAST protocol is generated, and the data stream of the FAST protocol is decoded by running the RTL decoding code.
In practical disclosures, the FAST market data may contain gear cycle data and commission cycle data in two directions of buying and selling. When decoding is performed according to the number of input fields, a jump relationship is too complex. In order to reduce the complexity of field matching, the present disclosure buffers the input fields to the field shift register, wherein the number of input fields in the shift register is the maximum number of fields in each clock cycle. For example, when four bytes are input in the case of 10 Gbits/s, the bytes correspond to at most four fields. An output of the shift register is the number of fields read by the field matching state machine. Both the throughput and design clock frequency are taken into consideration, and at most four fields are read. The field matching state machine simplifies the number of states of the matching state machine and increases the clock frequency by limiting the jump of a FAST decoding state. At the same time, the overall throughput is improved by obtaining the number of parallel fields.
The specific jump of the matching state machine jump is as shown in
Based on the above two features, the present disclosure reduces the number of cycles that FAST data needs to be processed, and increases the dominant frequency of the matching state machine, thus reducing the overall processing delay.
The following is a description of an FPGA-based FAST protocol decoding apparatus provided by an embodiment of the present disclosure. The FPGA-based FAST protocol decoding apparatus described below can refer to the FPGA-based FAST protocol decoding method described above.
As shown in
The FPGA-based FAST protocol decoding apparatus of this embodiment is configured to implement the aforementioned FPGA-based FAST protocol decoding method. Therefore, the specific implementation of this apparatus can refer to the embodiment of the FPGA-based FAST protocol decoding method mentioned above. For example, the template analysis module 111, the state machine generation module 112, the data stream segmentation module 113, and the decoding module 114 are respectively configured to implement steps S101, S102, S103 and S104 of the above FPGA-based FAST protocol decoding method. Therefore, the FPGA-based FAST protocol decoding apparatus's specific implementations can refer to the descriptions of the embodiments of the corresponding parts and will not be described here.
In addition, since the FPGA-based FAST protocol decoding device of this embodiment is configured to implement the aforementioned FPGA-based FAST protocol decoding method. Therefore, the function corresponds to that of the above method, which will not be repeated here.
In addition, the present disclosure further provides an FPGA-based FAST protocol decoding device, including:
Finally, the present disclosure provides the readable storage medium. The readable storage medium stores a computer program; and the computer program, when executed by a processor, implements the steps of the above FPGA-based FAST protocol decoding device.
All the embodiments in this specification are described in a progressive manner. Contents mainly described in each embodiment are different from those described in other embodiments. Same or similar parts of all the embodiments refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant part can be referred to the description of the method part.
The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the hardware and the software module. The software module can be placed in a random access memory (RAM), an internal memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a mobile disk, a CD-ROM, or any storage medium in other forms known to the technical field.
The above describes the solutions provided by the present disclosure in detail. Specific examples are used herein to illustrate the principle and implementation modes of the present disclosure. The descriptions of the above embodiments are only used to help understand the method and its key thoughts of the present disclosure. Moreover, for those of ordinary skill in the art, according to the ideas of the present disclosure, there will be changes in the specific implementation modes and the scope of disclosure. In summary, the content of this specification should not be construed as limiting the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
202010751254.2 | Jul 2020 | CN | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2021/076927 | 2/19/2021 | WO |