The present invention relates to a conversion device, a conversion method, and a conversion program.
To monitor a network and analyze traffic trends, there exists xFlow technology that performs packet sampling and calculates flow statistical information from the header information. Also, there exists xFlow technology that performs packet sampling and cuts out and forwards the header portion itself (header sample). Also, there exist technologies that interchangeably convert between the various existing xFlow formats.
Patent Literature 1: Japanese Patent Laid-Open No. 2019-097069
Non-Patent Literature 1: RFC 3954
Non-Patent Literature 2: RFC 5103
Non-Patent Literature 3: RFC 7011
Non-Patent Literature 4: RFC 7012
Non-Patent Literature 5: RFC 7013
Non-Patent Literature 6: RFC 7014
Non-Patent Literature 7: RFC 7015
Non-Patent Literature 8: “sFlow Version 5”, [retrieved Jan. 9, 2020], Internet <URL: https://sflow.org/sflow_version_5.txt>
Non-Patent Literature 9: RFC 7133
Non-Patent Literature 10: “pmcct”, [retrieved Jan. 9, 2020], Internet <URL: http://www.pmacct.net/>
Non-Patent Literature 11: “nProbe”, [retrieved Jan. 9, 2020], Internet <URL: https://www.ntop.org/products/netflow/nprobe/>
Non-Patent Literature 12: Yuhei HAYASHI, Hiroshi OSAWA, “Study of settings for inner-outer header mapping method using hashes”, IEICE Society Conference 2019, B-6-18
A network (NW) device applying xFlow technology of the related art measures flow information internally, and outputs various flow information attached to an xFlow packet. However, with respect to encapsulated packets, NW devices of the related art can only measure the outer flow information of the packets. In other words, with respect to encapsulated packets, NW devices of the related art cannot measure the inner flow information of the packets. Additionally, with the xFlow format conversion methods of the related art, header sampling format conversion cannot be performed on the inner packets of an encapsulated packet.
Consequently, the xFlow technology of the related art has a problem of being incapable of outputting packets in the xFlow format necessary for aggregation and analysis of flow information for the inner packets of encapsulated packets.
The present invention has been devised in light of the above, and an objective is to provide a conversion device, a conversion method, and a conversion program capable of generating an xFlow packet suitable for aggregation and analysis of inner flow information of an encapsulated packet.
To address the problems described above and achieve the objective, a conversion device of the present invention includes a first separation unit that separates an inputted encapsulated packet into flow information and sampled headers including outer headers and inner headers, a second separation unit that separates the outer headers from the sampled headers, and a generation unit that obtains statistics about the inner headers on the basis of the sampled headers separated from the outer headers, and generates an xFlow packet including at least statistical information indicating the statistics about the inner headers.
Also, a conversion method of the present invention is a conversion method executed by a conversion device, and includes separating an inputted encapsulated packet into flow information and sampled headers including outer headers and inner headers, separating the outer headers from the sampled headers, and obtaining statistics about the inner headers on a basis of the sampled headers separated from the outer headers, and generating an xFlow packet including at least statistical information indicating the statistics about the inner headers.
Also, a conversion program of the present invention causes a computer to execute a process including separating an inputted encapsulated packet into flow information and sampled headers including outer headers and inner headers, separating the outer headers from the sampled headers, and obtaining statistics about the inner headers on a basis of the sampled headers separated from the outer headers, and generating an xFlow packet including at least statistical information indicating the statistics about the inner headers.
According to the present invention, xFlow packets in a format suitable for aggregation and analysis can be generated.
Hereinafter, an embodiment of a conversion device, a conversion method, and a conversion program according to the present application will be described in detail on the basis of the drawings. Furthermore, the present invention is not limited by the embodiment described hereinafter.
[Embodiment]
First, the embodiment will be described. A conversion device according to the embodiment obtains statistics about the inner headers inside encapsulated packets inputted from each NW device, generates an xFlow packet including at least statistical information indicating the statistics about the inner headers, and outputs the generated xFlow packet to an external device that performs aggregation and analysis.
[Configuration of Communication System]
The NW devices 2 performs packet sampling in traffic to be monitored. For example, the NW devices 2 extract packet header samples from sampled packets and forward the extracted header samples encapsulated into an xFlow packet (encapsulated packet) to the conversion device 10. At this time, the NW devices 2 forward statistical information related to the flow of the number of packets and the like to the conversion device 10 by attaching the statistical information to the xFlow packet to be forwarded or transmitting the statistical information as a separate xFlow packet.
The conversion device 10 converts the xFlow packets inputted from the various NW devices 2 into xFlow packets in a format corresponding to the content of the processing performed by the external analysis device 3. Specifically, the conversion device 10 obtains statistics about the inner headers of the xFlow packets inputted from the various NW devices 2. Subsequently, the conversion device 10 generates an xFlow packet including at least statistical information indicating the obtained statistics about the inner headers, and outputs the generated xFlow packet to the external analysis device 3.
The analysis device 3 analyzes the traffic to be monitored and aggregates packets in the traffic to be monitored. The analysis device 3 performs analysis and aggregation by using the statistical information included in the xFlow packet converted by the conversion device 10.
[Conversion Device]
Next, the conversion device 10 will be described.
As illustrated in
The separation unit 11 separates an inputted xFlow packet into flow information and sampled headers including an outer header and an inner header. For example, the separation unit 11 separates an inputted xFlow packet P1 into xFlow information F1 and sampled headers H1 to H3 including an outer header and an inner header (see (1) in
The decapsulation unit 12 separates the outer headers from the sampled headers. The sampled headers separated from the outer headers contain an inner header and a payload. The decapsulation unit 12 includes a removal unit 121 that removes the outer headers from the sampled headers and a storage unit 122 that stores information indicating correspondence relationships between the outer headers and the inner headers in the correspondence relationship DB 14. The decapsulation unit 12 respectively removes outer headers Ho1 to Ho3 from the sampled headers H1 to H3 (see (2) in
The conversion unit 13 obtains statistics about the inner headers on the basis of the sampled headers separated from the outer headers. The conversion unit 13 generates an xFlow packet including at least statistical information indicating the obtained statistics about the inner headers. The conversion unit 13 generates the xFlow packet in a format corresponding to the content of the processing performed by the analysis device 3 that acts as the output destination of the generated xFlow packet.
The conversion unit 13 generates an xFlow packet including statistical information about the inner headers on the basis of the original xFlow information (out, in) and the inner header information of the sampled headers (see (3) in
At this point, the conversion unit 13 generates the xFlow packet in a format corresponding to the content of the processing performed by the analysis device 3. The format of the xFlow packet may be a format that includes only the statistical information (for example, the packet P5 in
The correspondence relationship DB 14 stores correspondence relationships between the outer headers and the inner headers of the inputted xFlow packet. For example, in the correspondence relationship DB 14, time information is registered in association with the 5-tuple of the inner header and the 5-tuple of the outer header.
In the conversion device 10, the separation process by the separation unit 11, the separation process by the decapsulation unit 12, and the conversion process by the conversion unit 13 are executed in parallel on a plurality of xFlow packets.
As illustrated in
Specifically, the function of the separation unit 11 is deployed to separation cores #1 to #n. The function of the decapsulation unit 12 is distributively deployed to decapsulation cores #1 to #n.
Sampled headers to be processed are assigned to the decapsulation cores #1 to #n according to outer information such as the 5-tuple. The sampled headers to be processed by the decapsulation core #1 all include an outer header “out 1”, while the sampled headers to be processed by the decapsulation core #n all include an outer header “out n”.
The function of the conversion unit 13 is distributively deployed to conversion cores #1 to #n. Inner headers to be processed are assigned to the conversion cores #1 to #n according to inner information such as the 5-tuple. The sampled headers separated from the outer headers to be processed by the conversion core #1 all include an inner header “in 1”, while the sampled headers separated from the outer headers to be processed by the conversion core #n all include an inner header “in n”.
In each of the separation cores #1 to #n, the separation unit 11 performs a process of separating an xFlow packet into xFlow information and sampled headers. Additionally, each of the separation cores #1 to #n uses outer information such as the 5-tuple in the sampled headers to assign each of the separated sampled headers to the decapsulation cores #1 to #n corresponding to the outer header information of each (see (1) in
In each of the conversion cores #1 to #n, the conversion unit 13 obtains statistics about the inner header of each assigned sampled header, and generates an xFlow packet including at least the statistical information.
In this way, in the conversion device 10, sampled headers are assigned to each core with consideration for the ordering of the flow. Furthermore, in the conversion device 10, by respectively deploying the function of the separation unit 11, the function of the decapsulation unit 12, and the function of the conversion unit 13 to a plurality of CPU cores in a distributed manner, the separation processing by the separation unit 11, the separation processing by the decapsulation unit 12, and the generation processing by the conversion unit 13 are executed in parallel on a plurality of packets. With this arrangement, the processing by the conversion device 10 can be sped up.
[Removal Unit]
Next, the processing by the removal unit 121 illustrated in
The removal unit 121 performs protocol stack analysis on a sampled header and specifies the outer header position in the sampled header. For example, the removal unit 121 may use the method described in Japanese Patent Laid-Open No. 2019-097069 to determine properties such as the header type and the outer header. The removal unit 121 determines a protocol stack pattern indicating the type and layout of each protocol header in the inputted sampled header according to determination rules. The protocol stack pattern is information indicating the type and layout of each protocol header.
Specifically, the removal unit 121 determines the protocol stack pattern of an inputted packet by using a determination tree for determining the protocol stack pattern created by successively inspecting packets with a known protocol stack pattern from the low-level header, a logical determination formula for determining the protocol stack pattern created on the basis of a specific bit sequence inside a packet with a known protocol stack pattern, or a protocol config file indicating standardized header information of each protocol. The determination rules may be pre-generated in another device or may be generated by performing training using protocol config files for inputted packets. Note that the removal unit 121 may also determine the header using another method.
[Storage Unit]
Next, the processing by the storage unit 122 illustrated in
For example, as illustrated in
The hash table 1222 includes fields for an address, an arrival flag indicating whether or not the 1st packet has arrived, and a timer. In the arrival flag, “0” indicates that the 1st packet has not yet arrived, and “1” indicates that the 1st packet has already arrived. The timer is a countdown timer used to perform a periodic entry refresh for reducing hash collisions. The default value of the arrival flag is “0”, and the default value of the timer is all “1”.
The hash function unit 1221 accepts a flow definition and 5-tuple information about the 5-tuple of the inner header and the 5-tuple of the outer header as input, and uses a hash function to calculate an information hash value concatenating the 5-tuple of the inner header and the 5-tuple of the outer header as an address. The storage unit 122 accesses the row of the hash table 1222 at the calculated address.
For example, with respect to a packet Pa, the storage unit 122 accesses the row of the calculated address “0x0003” in the hash table 1222. At this point, because the arrival flag is “0” in this row, the packet Pa is the initial packet of a sequential flow. The storage unit 122 changes the arrival flag from “0” to “1” in the row of the address “0x0003” (see (1) in
Also, with respect to a packet Pb, the storage unit 122 accesses the row of the calculated address “0x0007” in the hash table 1222. At this point, the arrival flag is “1” in this row (see (3) in
At this point, the storage unit 122 refreshes the entries at a predetermined timing on the basis of a distribution of the flow duration to initialize old entries and reduce collisions.
For example, the storage unit 122 obtains the flow duration x (sec) corresponding to the a percentile (0≤α≤1) from the distribution of flow duration, and uses the flow duration x (sec) to set the refresh timing. Additionally, in the case where the timer bits are 1 or greater, the storage unit 122 sets the refresh interval to “x/(timer bits^2)”, and decrements the timer every refresh interval. Also, by changing the arrival flag from “1” to “0” for an entry whose timer bits are all “0” and also changing the timer to “1111”, the storage unit 122 refreshes the entry. Also, in the case where the timer bits are other than 1, the storage unit 122 sets the refresh interval to “x”, and every refresh interval, changes all of the arrival flags to “0” and also changes the timer to a default value to refresh the entry.
Also, instead of the arrival flag field L1 and the timer field L2 in the hash table 1222, a timeout time field may be provided, and in the case where a 1st packet arrives, the storage unit 122 may change a default value in the timeout time field to a timeout time, and refresh the entry when timeout is reached.
Also, as illustrated in
For example, with respect to a packet Pa, the storage unit 122 accesses the row of the address “0x0003” in the hash table 1224. At this point, because the arrival flag is “0” in this row, the packet Pa is the initial packet of a sequential flow. The storage unit 122 changes the arrival flag from “0” to “1” in the row of the address “0x0003” (see (1) in
Also, with respect to a packet Pb, the storage unit 122 accesses the row of the address “0x0007” in the hash table 1224. At this point, the arrival flag in this row is “1” (see (4) in
Note that in the case where the arrival flag is “1” but the detection bits in the hash table 1244 do not match the calculated collision detection bits of the packet, the storage unit 122 detects a collision (hash collision), and may also sample the flow of the packet and store inner header information and outer header information. Also, the storage unit 122 refreshes the hash table 1224 using a method similar to the refresh method for the hash table 1222.
[Conversion Unit]
Next, the processing by the conversion unit 13 will be described. The conversion unit 13 obtains statistics about the inner headers on the basis of the sampled headers separated from the outer headers. In addition, the conversion unit 13 generates an xFlow packet in which statistical information indicating the obtained statistics about the inner headers is included in the xFlow information.
At this point, in the case of generating an xFlow packet in a format that includes only the statistical information or in a format that attaches an inner header sample to the statistical information, the conversion unit 13 totals the obtained statistical information about the inner headers and includes the totaled statistical information in the xFlow information.
The xFlow information F1 illustrated in
At this point, in the conversion device 10, the outer headers are separated from the sampled headers by the decapsulation unit 12, and statistics about the inner headers of the sampled headers P21, P22, and P23 separated from the outer headers are obtained in the conversion unit 13.
For example, the conversion unit 13 obtains statistics indicating that the inner headers of the sampled packets P21 and P23 is “in 1”, and the inner header of the sampled header P22 is “in 2”. In this case, because the inner headers of the sampled headers P21 and P23 are the same, the conversion unit 13 treats the sampled headers P21 and P23 as the same flow (see (2) in
The conversion unit 13 generates a packet P51 or a packet P41 in which inner header statistical information indicating that the number of packets with the inner header “in 1” is “2” and that the number of packets with the inner header “in 2” is “1” is included in the xFlow information.
In this way, the conversion unit 13 converts an encapsulated xFlow packet inputted into the conversion device 10 into an xFlow packet in which statistical information regarding the inner headers of the inner packets is included as flow information, and outputs the converted xFlow packet to the analysis device 3. As described in
Next, a function of combining statistical information in the conversion unit 13 will be described. The conversion unit 13 generates and outputs a single xFlow packet in which statistical information about a plurality of packets is totaled. Here, the packets for which the statistical information is totaled are the inner packets encapsulated inside the xFlow packet.
For example, the conversion unit 13 includes a function of outputting a single xFlow packet in which statistical information about a plurality of packets in the same flow is combined. In other words, if a plurality of packets belong to the same flow, the conversion unit 13 combines statistical information about the packets and outputs a single xFlow packet. Namely, the conversion unit 13 totals statistical information regarding the inner headers of a plurality of packets having the same inner header, and generates a single xFlow packet including the totaled statistical information.
Additionally, a maximum inactive communication time (flow-inactive-timeout) and a maximum active communication time (flow-active-timeout) may be set with respect to the conversion device 10, and a packet output condition may be set using the set maximum inactive communication time and the maximum active communication time. For example, the output condition stipulates that there is a flow for which the maximum inactive communication time has elapsed since the time when a packet was last received, or that there is a flow for which the maximum active communication time has elapsed since the time when a packet was first received.
Also, if a flow B is a flow for which the maximum active communication time has elapsed since the time when a packet was first received, the conversion unit 13 totals statistical information about the inner headers of the flow B, and outputs an xFlow packet including the totaled statistical information (see (3) in
In this way, because the conversion unit 13 totals statistical information about packets in the same flow and outputs xFlow information including the totaled statistical information, the number of packets outputted externally can be reduced (see (4) in
Additionally, the conversion unit 13 includes a function of outputting an xFlow packet with a plurality of header samples from different flows collectively attached, even if the headers are from different packets.
As illustrated in
The conversion unit 13 collects header sample portions without outputting flow information to the external analysis device 3 until the predetermined output condition is satisfied (see (2) in
Also,
[Processing Procedure of Conversion Process]
Next, a processing procedure of a packet conversion process executed by the conversion device 10 will be described.
As illustrated in
The conversion unit 13 performs the conversion process of obtaining statistics about the inner headers on the basis of the sampled headers separated from the outer headers, generating an xFlow packet including at least statistical information indicating the obtained statistics about the inner headers, and outputting the generated xFlow packet to the analysis device 3 (step S3).
[Processing Procedure of Conversion Process]
Next, a processing procedure of the conversion process (step S3) illustrated in
As illustrated in
In the case where the xFlow packet output condition is satisfied (step S12: Yes), the conversion unit 13 generates an xFlow packet in the set format (step S13). In this case, the conversion unit 13 includes statistical information indicating the obtained statistics about the inner headers in the xFlow information. Also, depending on the settings, the conversion unit 13 includes a totaled result of the statistical information about a plurality of packets in the same flow or a totaled result of the statistical information about a plurality of packets in different flows in the xFlow information. Subsequently, the conversion unit 13 outputs the generated xFlow packet to the external analysis device 3 (step S14).
[Effects of Embodiment]
At this point, an xFlow packet conversion process according to the related art will be described.
As an example,
As illustrated in
Consequently, according to the conversion device 10, it is possible to calculate statistical information about the inner part of an encapsulated packet, namely the inner headers, that could not be calculated in the related art (see (1) in
Additionally, the conversion unit 13 generates an xFlow packet including at least statistical information indicating statistics about the inner headers. At this time, the conversion unit 13 generates the xFlow packet in a format corresponding to the content of the processing in the external device.
For example, the conversion unit 13 selects the format of the xFlow packet to be generated from among a format that includes only the statistical information (for example, the packet P5 in
Additionally, the conversion device 10 adopts an architecture enabling parallelization of the function units with consideration for the flow ordering (see (3) in
Furthermore, in the conversion device 10, the conversion unit 13 includes a function of generating and outputting a single xFlow packet in which statistical information about a plurality of packets is totaled. In this way, because the conversion device 10 aggregates flows in the conversion unit 13 to generate and output a single xFlow packet in which statistical information about a plurality of packets is totaled, the number of packets outputted externally can be reduced (see (4) in
As above, according to the conversion device 10, an xFlow packet including statistical information about the inner flow information of an encapsulated packet can be generated, and furthermore, a speedup in performance of the device and a reduction in the number of packets outputted externally can be achieved.
[System Configuration of Embodiment]
Also, the structural elements of the conversion device 10 illustrated in
Also, all or any part of the processing performed in the conversion device 10 or 10B may be achieved by a CPU and a program that is interpreted and executed by the CPU. Moreover, the processing performed in the conversion device 10 may also be achieved as hardware through wired logic.
Additionally, it is also possible to perform manually all or part of the processes described as being performed automatically in the embodiment. Alternatively, it is possible to perform automatically, with known methods, all or part of the processes described as being performed manually. Otherwise, information including the processing sequences, control sequences, specific names, and various data or parameters described above and illustrated in the drawings may be modified appropriately except as noted.
[Program]
The memory 1010 includes ROM 1011 and RAM 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS), for example. The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disc is inserted into the disk drive 1100, for example. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.
The hard disk drive 1090 stores an operating system (OS) 1091, an application program 1092, program modules 1093, and program data 1094, for example. Namely, a program prescribing each process of the conversion device 10 is implemented as a program module 1093 stated in code executable by the computer 1000. The program modules 1093 are stored in the hard disk drive 1090, for example. For example, program modules 1093 for executing processes similar to the functional configuration of the conversion device 10 are stored in the hard disk drive 1090. Note that the hard disk drive 1090 may also be replaced by a solid state drive (SSD).
In addition, setting data used in the processes of the embodiment described above is stored in the memory 1010 or the hard disk drive 1090 for example as the program data 1094. Moreover, the CPU 1020 reads out the program modules 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes them.
Note that the program modules 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and may also be stored in a removable storage medium and read out by the CPU 1020 through the disk drive 1100 or the like, for example. Alternatively, the program modules 1093 and the program data 1094 may be stored in another computer connected over a network (such as a local area network (LAN) or a wide area network (WAN)). In addition, the program modules 1093 and the program data 1094 may also be read out by the CPU 1020 from another computer through the network interface 1070.
The above describes an embodiment applying the invention made by the inventor, but the present invention is not limited by the description and drawings which form a part of the disclosure of the present invention according to the embodiment. In other words, other embodiments, examples, practical technologies, and the like made by persons skilled in the art on the basis of the present embodiment are all included in the scope of the present invention.
1 Communication system
2 NW device
3 Analysis device
10 Conversion device
11 Separation unit
12 Decapsulation unit
13 Conversion unit
14 Correspondence relationship database (DB)
121 Removal unit
122 Storage unit
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2020/002526 | 1/24/2020 | WO |