Methods, systems, and computer readable media for processing network flow metadata at a network packet broker

Abstract

A method for network flow metadata processing at a network packet broker includes, receiving, as input at the network packet broker, network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in the network flow. The method further includes accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata. The method further includes processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule. The method further includes forwarding, by the network packet broker and based on results of the processing, egress network flow metadata to a network tool.

Description

TECHNICAL FIELD

The subject matter described herein relates to processing network flow metadata. More particularly, the subject matter described herein relates to processing network flow metadata at a network packet broker.

BACKGROUND

Network packet brokers are used to aggregate packets from multiple sources, deduplicate packets, since the same packet may be captured in different places, filter packets, redirect packets to final destination, possibly replicate the packets to multiple destinations, and filter to selectively send packets. However, the input to network packet brokers has been limited to packets, and the processing performed by network packet brokers has been limited to processing packets.

In addition to packet flows, another type of network data generated by network devices is network flow metadata, one example of which is NetFlow. NetFlow is a statistical, flow-by-flow summary of packet data at a higher level than raw packet data. If two network hosts communicate with each other, instead of generating a copy of every packet, NetFlow provides statistics about the overall conversation. NetFlow is a collection of conversations and reporting statistics about each conversation.

Many network devices can produce NetFlow statistics. Typically, a network operator configures a NetFlow collector as a destination, and sets the NetFlow collector as the destination for NetFlow data generated by a device that generates NetFlow records. This configuration is analogous to the configuration of a switch port analyzer (SPAN) port on a network device, such as a router. A SPAN port is typically configured with a destination for network packets copied by the router to the SPAN port.

Configuring a NetFlow generator or a SPAN port to forward packets to a single destination has a number of limitations. For example, it may be desirable to forward the same NetFlow statistics or packet copies to multiple destinations, which is not allowed by a device that allows only a single NetFlow or SPAN port destination to be configured. In another example, it may be desirable to feed two NetFlow collectors with NetFlow data from different switches. If the two switches see the same packet flow, NetFlow records will be duplicated, which may result in wasting of storage and processing resources of network tools that utilize data in NetFlow records. In another example, it may be desirable to redirect NetFlow records to a different destination when a primary NetFlow destination fails. Such redirection must be performed manually in a device that allows only a single configured NetFlow destination. These problems are magnified when NetFlow records are provided by multiple switches, requiring manual reconfiguration of the destination on each switch.

In light of these and other difficulties, there exists a need for methods, systems, and computer readable media for providing for network flow metadata processing at a network packet broker.

SUMMARY

A method for network flow metadata processing at a network packet broker includes receiving, as input at the network packet broker, network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in the network flow. The method further includes accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata. The method further includes processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule. The method further includes forwarding, by the network packet broker and based on results of the processing, egress network flow metadata to a network tool.

According to another aspect of the subject matter described herein, receiving the network flow metadata includes receiving at least two copies of the same network flow metadata record, identifying a network flow metadata processing rule includes identifying a network flow metadata deduplication processing rule to apply to the at least two copies of the same network flow metadata record, processing the network flow metadata includes performing deduplication processing of the at least two copies of the same network flow metadata record, and forwarding the egress network flow metadata includes forwarding a single copy of the same network flow metadata processing record to a single tool port.

According to another aspect of the subject matter described herein, receiving the network flow metadata includes receiving a single network flow metadata record, identifying a network flow metadata processing rule includes identifying a network flow metadata replication rule to apply to the single network flow metadata record, processing the network flow metadata record includes replicating the single network flow metadata record, and forwarding the egress network flow metadata includes forwarding at least two copies of the single network flow metadata record to at least two different network tools.

According to another aspect of the subject matter described herein, receiving the network flow metadata includes receiving an encrypted network flow metadata record, identifying the network flow metadata processing rule includes identifying a network flow metadata decryption, processing, and re-encryption rule to apply to the network flow metadata, processing the network flow metadata includes decrypting the network flow metadata record a decryption key obtained via a secure backchannel interface to produce a decrypted network metadata record, performing at least one operation on the decrypted network flow metadata record, re-encrypting the decrypted network flow metadata record to produce a re-encrypted network flow metadata record, and forwarding the egress network flow metadata includes forwarding the re-encrypted network metadata flow record to the network tool.

According to another aspect of the subject matter described herein, performing at least one operation on the decrypted network flow metadata record includes performing at least one of: network flow metadata deduplication, replication, aggregation, and format translation processing on the decrypted network flow metadata record.

According to another aspect of the subject matter described herein, receiving packet copies and network flow log data, wherein identifying a network flow metadata processing rule includes identifying a network flow metadata aggregation rule to apply to the network flow metadata, the packet copies, and the network flow log data, wherein processing the network flow metadata includes aggregating the network flow metadata with at least one of the packet copies and the network flow log data to create an aggregated network flow metadata record, and wherein forwarding the egress network flow metadata includes forwarding the aggregated network flow metadata record to the network tool.

According to another aspect of the subject matter described herein, receiving the network flow metadata includes receiving a network flow metadata record formatted according to a first network flow metadata format, identifying the network flow metadata processing rule includes identifying a network flow metadata format translation rule to apply to the network flow metadata record, processing the network flow metadata includes translating a format of the network flow metadata from the first network flow metadata format to a second network flow metadata format different from the first network flow metadata format to produce a translated network flow metadata record, and forwarding the egress network flow metadata includes forwarding the translated network flow metadata record to the network tool.

According to another aspect of the subject matter described herein, the first network flow metadata format comprises a first network flow metadata record format and the second network flow metadata format includes a second network flow metadata record format different from the first network flow metadata record format and wherein translating the format of the network flow metadata includes translating the format of the network flow metadata from first network flow metadata record format to the second network flow metadata record format.

According to another aspect of the subject matter described herein, receiving the network flow metadata from a network device includes receiving network flow metadata records from a network tap or a tap or span port on a router or switch.

According to yet another aspect of the subject matter described herein, forwarding the network flow metadata to a network tool includes forwarding the network flow metadata to a network flow metadata collector.

According to another aspect of the subject matter described herein, a system for network flow metadata processing includes a network packet broker including at least one processor and a memory. The system further includes a network flow metadata processing rules database stored in the memory and containing rules for processing network flow metadata. The system further includes a network flow metadata processing engine implemented by the at least one processor for receiving network flow metadata as input, the network flow metadata including a network flow statistic generated by a network device regarding packets in the network flow, for accessing the network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata, for processing the network flow metadata using the network flow metadata processing rule, and for forwarding, based on results of the processing, egress network flow metadata to a network tool.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive at least two copies of the same network flow metadata record, identify a network flow metadata deduplication processing rule to apply to the at least two copies of the same network flow metadata record, perform deduplication processing of the at least two copies of the same network flow metadata record, and forward a single copy of the same network flow metadata processing record to a single tool port.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive a single network flow metadata record, identify a network flow metadata replication rule to apply to the single network flow metadata record, replicate the single network flow metadata record, and forward at least two copies of the single network flow metadata record to at least two different network tools.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive an encrypted network flow metadata record, identify a network flow metadata decryption, processing, and re-encryption rule to apply to the network flow metadata, decrypt the network flow metadata record a decryption key obtained via a secure backchannel interface to produce a decrypted network metadata record, perform at least one operation on the decrypted network flow metadata record, re-encrypt the decrypted network flow metadata record to produce a re-encrypted network flow metadata record, and forward the re-encrypted network metadata flow record to the network tool.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to perform at least one of: network flow metadata deduplication, replication, aggregation, enrichment, and format translation processing on the decrypted network flow metadata record.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive packet copies and network flow log data, identify a network flow metadata aggregation rule to apply to the network flow metadata, the packet copies, and the network flow log data, aggregate the network flow metadata with at least one of the packet copies and the network flow log data to create an aggregated network flow metadata record, and forward the aggregated network flow metadata record to the network tool.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive a network flow metadata record formatted according to a first network flow metadata format, identify a network flow metadata format translation rule to apply to the network flow metadata record, translate a format of the network flow metadata from the first network flow metadata format to a second network flow metadata format different from the first network flow metadata format to produce a translated network flow metadata record, and forward the translated network flow metadata record to the network tool.

According to another aspect of the subject matter described herein, the first network flow metadata format comprises a first network flow metadata record format and the second network flow metadata format includes a second network flow metadata record format different from the first network flow metadata record, and the network flow metadata processing engine is configured to translate the network flow metadata from the first network flow metadata record format to the second network flow metadata record format.

According to another aspect of the subject matter described herein, the network flow metadata processing engine is configured to receive NetFlow records from a network tap or a tap or SPAN port on a router or switch and to forward the network flow metadata to a NetFlow collector.

According to another aspect of the subject matter described herein, a non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps is provided. The steps include receiving, as input at the network packet broker, network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in the network flow. The steps further include accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata. The steps further include processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule. The steps further include forwarding, by the network packet broker and based on results of the processing, egress network flow metadata to a network tool.

The subject matter described herein can be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein can be implemented in software executed by a processor. In one exemplary implementation, the subject matter described herein can be implemented using a non-transitory computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a network diagram illustrating conventional NetFlow collection;

FIG. 2 is a block diagram illustrating an exemplary NetFlow packet structure;

FIG. 3 is a diagram illustrating exemplary parameters in a NetFlow/IP flow information export (IPFIX) packet header;

FIG. 4 is a block diagram illustrating a network packet broker with network flow metadata processing capabilities where the network packet broker performs deduplication of network flow metadata records;

FIG. 5 is a block diagram illustrating the network packet broker with network flow metadata processing capabilities where the network packet broker replicates network flow metadata to different destinations;

FIG. 6 is a block diagram illustrating a network packet broker with network flow metadata processing capabilities where the network packet broker performs decryption and re-encryption of network flow network flow metadata;

FIG. 7 is a block diagram of a network packet broker for with network flow metadata processing capabilities where the network packet broker aggregates network flow metadata with packet copies and network flow log file data;

FIG. 8 is a block diagram illustrating a network packet broker with network flow metadata processing capabilities where the network packet broker performs normalization or translation of different network flow metadata record formats

FIG. 9 is a flow chart illustrating an exemplary process for processing network flow metadata at a network packet broker.

DETAILED DESCRIPTION

Some network devices, such as switches, routers, and network taps, include a client/process that generates and exports NetFlow records to a single destination. These records are sent to a NetFlow collector, which then provides the records to analyzers and users (e.g., via a front-end user interface (UI), etc.).

FIG. 1 is a block diagram illustrating a NetFlow data collection architecture. In FIG. 1, a NetFlow exporter 100 is a device, such as a network tap, that generates NetFlow records from packets flowing through various computer networks 102, 104, 106, and 108. NetFlow exporter 100 exports the NetFlow packets to a NetFlow collector 110, which stores the records in storage 112. An analysis console 114 analyzes the NetFlow records stored in flow storage 112 to make network planning and other operational decisions.

One problem with the architecture illustrated in FIG. 1 is that it is not flexible. NetFlow collector 110 is configured as the sole destination for NetFlow exporter 100. It may be desirable to export NetFlow records to more than one NetFlow collector. In another example, it may be desirable to perform the duplication or other processing on NetFlow or other network metadata records. The architecture illustrated in FIG. 1 does not provide these and other capabilities.

Export of NetFlow Records

A network device, such as a router, will output a NetFlow record when the router determines that the flow is finished. The router does this by flow aging: when the router sees new traffic for an existing flow it resets the aging counter. Also, transmission control protocol (TCP) session termination in a TCP flow causes the router to expire the flow. Routers can also be configured to output a flow record at a fixed interval even if the flow is still ongoing.

NetFlow Packet Transport Protocol

NetFlow records are traditionally exported using user datagram protocol (UDP) and collected using a NetFlow collector. The IP address of the NetFlow collector and the destination UDP port must be configured on the sending router. A common value is UDP port 2055, but other values like 9555 or 9995, 9025, 9026 etc. can also be used. As stated above, however, most routers only allow a single NetFlow destination to be configured, and the configuration is static, requiring manual reconfiguration to change the destination for NetFlow records.

NetFlow records are carried in export packets (which are carried in UDP datagrams). FIG. 2 is a block diagram illustrating an exemplary format for a NetFlow export packet. More particularly, the top portion of FIG. 2 illustrates the overall configuration of a NetFlow export packet and the bottom portion of FIG. 2 illustrates the fields in the packet header of the NetFlow export packet. It should be noted that the NetFlow export packet includes one or more template flow sets and data flow sets which contain statistics generated by the NetFlow exporter regarding network packet flows. A flow set is a generic name for a collection of records that follow the packet header in the export packet. A template flow set is a set of template records that define the format for data flow records that follow the template flow set in the export packet. A data flow set is a collection of one or more data records that contain statistics about network packet flows.

The NetFlow packet header includes a version identifier that identifies the NetFlow version, a system uptime field that indicates how long the NetFlow exporter has been running, a Unix seconds field that indicates the number of seconds that have elapse since the start of coordinated universal time, a package sequence field stores a sequence number for the NetFlow packet, and a source ID field, which stores an identifier of the source of the NetFlow export packet.

FIG. 3 illustrates exemplary values for the fields that may appear in a NetFlow or IP flow information export (IPFIX) packet header. The fields include the NetFlow version, a packet count, system uptime, timestamp, flow sequence number, and sequence ID.

For efficiency reasons, the router traditionally does not keep track of flow records already exported, so if a NetFlow packet is dropped due to network congestion or packet corruption, all contained records are lost forever. The UDP protocol does not inform the router of the loss so it can send the packets again. This can be a real problem, especially with NetFlow v8 or v9 that can aggregate a lot of packets or flows into a single record. A single UDP packet loss can cause a huge impact on the statistics of some flows. That is why some modern implementations of NetFlow use the stream control transmission protocol (SCTP) to export packets so as to provide some protection against packet loss, and make sure that NetFlow v9 templates are received before any related record is exported. Note that TCP would not be suitable for NetFlow because a strict ordering of packets would cause excessive buffering and delays.

The problem with SCTP is that it requires interaction between each NetFlow collector and each router exporting NetFlow. There may be performance limitations if a router has to deal with many NetFlow collectors, and a NetFlow collector has to deal with lots of routers, especially when some of them are unavailable due to failure or maintenance.

SCTP may not be efficient if NetFlow must be exported toward several independent collectors, some of which may be test servers that can go down at any moment. UDP allows simple replication of NetFlow packets using Network taps or L2 or L3 Mirroring. Simple stateless equipment can also filter or change the destination address of NetFlow UDP packets if necessary. Since NetFlow export almost only use network backbone links, packet loss will often be negligible. If it happens, it will mostly be on the link between the network and the NetFlow collectors.

NetFlow Record

A NetFlow record can contain a wide variety of information about the traffic in a given flow.

NetFlow version 5 (one of the most commonly used versions, followed by version 9) contains the following:

• • Input interface index used by simple network management protocol (SNMP) (if Index in management information base (IF-MIB)).
  • Output interface index or zero if the packet is dropped.
  • Timestamps for the flow start and finish time, in milliseconds since the last boot.
  • Number of bytes and packets observed in the flow
  • Layer 3 headers:
    • Source and destination IP addresses
    • Internet control message protocol (ICMP) Type and Code.
    • IP protocol
    • Type of Service (ToS) value
  • Source and destination port numbers for TCP, UDP, SCTP
  • For TCP flows, the union of all TCP flags observed over the life of the flow.
  • Layer 3 routing information:
    • IP address of the immediate next-hop (not the border gateway protocol (BGP) next hop) along the route to the destination
    • Source and destination IP masks (prefix lengths in the classless inter-domain routing (CIDR) notation)

NetFlow version 9 can include all of these fields and can optionally include additional information such as multiprotocol label switching (MPLS) labels and IPv6 addresses and ports.

By analyzing flow data, a picture of traffic flow and traffic volume in a network can be built.

The exemplary nomenclature may be used in describing NetFlow records:

Version

• • The version of NetFlow records exported in this packet; for Version 9, this value is 0x0009

Count

• • Number of FlowSet records (both template and data) contained within this packet

System Uptime

• • Time in milliseconds since this device was first booted

UNIX Seconds

• • Seconds since 0000 Coordinated Universal Time (UTC) 1970

Sequence Number

• • Incremental sequence counter of all export packets sent by this export device; this value is cumulative, and it can be used to identify whether any export packets have been missed
    • **Note: This is a change from the NetFlow Version 5 and Version 8 headers, where this number represented “total flows.”

Source ID

• • The Source ID field is a 32-bit value that is used to guarantee uniqueness for all flows exported from a particular device. (The Source ID field is the equivalent of the engine type and engine ID fields found in the NetFlow Version 5 and Version 8 headers). The format of this field is vendor specific. In the Cisco implementation, the first two bytes are reserved for future expansion, and will always be zero. Byte 3 provides uniqueness with respect to the routing engine on the exporting device. Byte 4 provides uniqueness with respect to the particular line card or Versatile Interface Processor on the exporting device. Collector devices should use the combination of the source IP address plus the Source ID field to associate an incoming NetFlow export packet with a unique instance of NetFlow on a particular device.

All NetFlow packets begin with version-dependent header that contains at least the following fields:

• • Version number (v1, v5, v7, v8, v9)
  • Sequence number to detect loss and duplication
  • Timestamps at the moment of export, as system uptime or absolute time.
  • Number of records (v5 or v8) or list of templates and records (v9)

The subject matter described herein includes network packet broker (NPB) that provides network packet broker features related to the processing of typical user plane packet traffic (e.g., mobile IP packet traffic transiting a cellular network, GTP packet traffic, etc.)—aggregate, deduplicate, replicate to multiple network tool destinations, on-the-fly changes to different network tool destinations, filter what each network tool destination gets, etc. The network packet broker described herein provides processing on NetFlow and other network flow metadata records. In one embodiment, the network packet broker described herein may emulate a NetFlow collector element.

The network packet broker may receive copies of NetFlow export records that are obtained from network taps, probes or SPAN ports in a monitored network. The network packet broker, in one processing example, may examine the received NetFlow records and determine whether a received NetFlow record is a duplicate of a NetFlow record previously received at the network packet broker. If the NetFlow record is determined to be a duplicate, then the duplicate NetFlow record is filtered out/dropped, as shown in FIG. 4.

More particularly, FIG. 4 illustrates an example of a network packet broker 400 that receives NetFlow records as input and performs deduplication processing on the NetFlow records. Network packet broker 400 illustrated in FIG. 4 includes at least one processor 402 and memory 404. A network flow metadata processing engine 406 may reside in memory 404 and the executed by processor 402 to perform various processing on network flow metadata and other network flow data received by network packet broker 400. In one example, network flow metadata processing engine 406 may access a network metadata processing rules database 408 to identify network metadata processing rules to apply to received network metadata. The result of the processing may be used to configure one or more ingress network metadata filters 410 and 412, one or more egress metadata filters 414 and 416, or to perform other processing on received NetFlow records.

In the illustrated example, duplicate copies 418 and 420 of the same NetFlow record are received on different network ports 422 and 424 of network packet broker 400. NetFlow records 418 and 420 may be generated by network taps or other network devices located in different locations but that produce the same NetFlow record. Network flow metadata processing engine 406 receives NetFlow records 418 and 420 and accesses database 408 to determine that deduplication processing is required. Network flow metadata processing engine 406 performs the deduplication processing and identifies that NetFlow records 418 and 420 are duplicates of the same record. Accordingly, network flow metadata processing engine 406 configures egress filters 414 and 416 such that only a single egress NetFlow record 426 is output on tool ports 428 or 430 of network packet broker 400. Egress NetFlow record 426 may be forwarded to a network tool, such as a NetFlow collector.

FIG. 5 illustrates another example of network metadata processing that may be performed by network packet broker 400. In FIG. 5, network packet broker 400 receives a single NetFlow record 418 on network port 422. Network flow metadata processing engine 406 performs a lookup in network metadata processing rules database 408 and determines that replication processing is to be performed for the NetFlow record 418. Accordingly, network flow metadata processing engine 406 replicates NetFlow record 418 and forwards the NetFlow record 418 and the duplicate 432 of the NetFlow record 418 to network devices via egress filters 414 and 416 and tool ports 428 and 430. In one example, tool ports 428 and 430 may provide NetFlow feeds to different NetFlow collectors or other network tools.

Network packet broker 400 may implement and enforce various filtering criteria that are applied to received NetFlow records. Exemplary filtering criteria may include, but are not limited to, NetFlow version, count, system uptime, timestamp, sequence number, etc. These criteria may be provisioned by a human operator or may be dynamically provisioned via a machine-to-machine interface, where a network tool sends NetFlow filter criteria to network packet broker 400, and network packet broker 400 installs and implements the new NetFlow record filter criteria on the fly without human intervention.

Network packet broker 400 may receive multiple NetFlow records and to aggregate these NetFlow records and or consolidate or “stitch” NetFlow records with other NetFlow records and/or other network data. Examples of such aggregation will be described below with regard to FIG. 7.

Network packet broker 400 may receive a NetFlow record that is addressed to a first NetFlow collector and redirect the NetFlow record to a second NetFlow collector different from the first NetFlow collector. Such redirection may be done for the purposes of load balancing, failover, or recovery after a failover. For example, network packet broker 400 replicates NetFlow records to a single destination, as illustrated in FIG. 1, and that destination fails, network packet broker 400 may be configured with a secondary destination for the NetFlow records, and, upon detecting failure of the first NetFlow destination, may cease forwarding NetFlow records to the failed destination and begin forwarding NetFlow records to the new destination. Such redirection may occur automatically without the need for a human to provision the new destination for the NetFlow records. When the network packet broker 400 detects recovery of the primary NetFlow destination, network packet broker 400 may either automatically switch back to forwarding NetFlow records to the primary destination or may load balance NetFlow records between the primary and secondary destinations.

Network packet broker 400 may include the ability to receive and temporarily decrypt NetFlow records that are, for instance, being communicated securely via secure sockets layer/transport layer security (SSL/TLS) or other encryption protocols (FIG. 6). The decrypted NetFlow records may be processed as described previously in this disclosure, and subsequently re-encrypted prior to egress.

FIG. 6 illustrates an example where network packet broker 400 receives secure or encrypted NetFlow records 600 from a NetFlow producer. Network flow metadata processing engine 406 performs a lookup in network metadata processing rules database 408 and determines that decryption and re-encryption processing as well as replication may be required for the received secure NetFlow record. In one example, network flow metadata processing engine 406 may access one or more encryption and decryption keys stored in encryption and decryption key store 602. The encryption and decryption keys stored in encryption and decryption key store 602 may include decryption keys obtained via a secure back channel with an encrypted NetFlow record producer. The decrypted NetFlow record may be re-encrypted using different encryption keys specific to different network tool destinations. In the illustrated example, secure NetFlow records 604 and 606 may be sent to different network tools that have different security relationships with network packet broker 400.

It will be appreciated that although most of the examples described herein are based on the use of the NetFlow protocol, network packet broker 400 may receive as input and process other network flow metadata record formats, including jFlow (produced by Juniper Networks-compatible devices), rFlow (produced by Ericsson-compatible devices), sampled flow (sFlow), IPFIX, NetStream, etc. jFlow and rFlow are proprietary network metadata record formats generated by network devices of Juniper Networks, and Ericsson, respectively. sFlow is an industry standard for exporting packets. sFlow is standardized by the sFlow consortium at www.sFlow.org.

In FIG. 7, network packet broker 400 includes the ability to receive any combination of packet copies, flow log data (e.g., virtual private cloud (VPC) flow log data, Zeek/Bro log data, etc.) and flow records (e.g., NetFlow, sFlow, IPFIX, IxFlow (an Ixia/Keysight network flow metadata record format), etc.) and perform consolidation processing operations that involve some or all of these ingress feeds. As used herein the term consolidation processing is intended to include processing operations that aggregate, enrich or combine/merge the information contain in multiple ingress feeds into a single network flow metadata format of the user's choosing (e.g., IxFlow, NetFlow, sFlow, etc.). For example, in FIG. 7, one ingress feed includes VPC flow log data 700, while another ingress feed includes NetFlow records 418, and yet another ingress feed includes unprocessed or raw packet copies 704 obtained from a monitored network. Network flow metadata processing engine 406 may access rules in network metadata processing rules database 408 for performing the desired aggregation, enrichment, stitching, etc. operations. The resulting flow record (e.g., IxFlow) that contains the consolidated and possibly enriched flow information is transmitted to a network tool via an egress port of network packet broker 400.

In general, network packet broker 400 may receive any flow data in any format (e.g., VPC flow logs), and merge or combine two or more different sources of data. Such combining can be referred to as flow enrichment. Flow enrichment can be in the form of combining two or more NetFlow streams, one NetFlow stream and other source of network metadata (e.g., Zeek/Bro logs). Network packet broker 400 may take packets/packet copies and network flow metadata as input, or VPC flow logs and an extended Berkeley packet filter (eBPF) agent metadata stream and output flow records (e.g., IxFlow, NetFlow, etc.).

In the architecture illustrated in FIG. 7, network packet broker 400 receives a NetFlow record 418 via network port 422. Network packet broker 400 also receives flow log data 700 via network port 702. Network packet broker 400 further receives packet copies 704 via network port 706. Network flow metadata processing engine 406 performs a lookup in network metadata processing rules database 408 and identifies a network metadata aggregation, enrichment, consolidation, and/or stitching rule to apply to the network flow metadata, the packet copies, and the flow log data. In the illustrated example, network flow metadata processing engine 406 produces an aggregated output flow record 708 that includes network metadata from NetFlow record 418, flow log data from flow log data 700, and/or packet copies from packet copy 704. Network flow metadata processing engine 406 may also configure one or more of ingress filters 410, 412, and 710 to apply ingress filter rules to input network metadata, packets, and/or flow log data. Network flow metadata processing engine 406 may also configure one or more egress filters 414 to filter egress network metadata, packets, and/or log data.

FIG. 8 illustrates an example in which network packet broker 400 is configured to perform network metadata format translation or normalization. Network metadata format translation refers to translating network metadata from one format, such as NetFlow, to another format, such as IxFlow or sFlow. Network metadata format normalization involves translating network metadata formats from different input formats to a single normalized output format. Referring to FIG. 8, network port 422 receives a NetFlow record 418. Network port 800 receives an sFlow formatted record 802. Network metadata processing engine 406 accesses network metadata processing rules database 408 and determines that normalization or translation processing is to be applied to network flow records 418 and 802. In the illustrated example, records 418 and 802 are converted to a common output network flow metadata format. In FIG. 8, the common output network metadata format is IxFlow. Accordingly, network packet broker 400 outputs IxFlow records 804 and 806 to network tools via tool ports 428 and 430.

In one example, when translating from NetFlow to sFlow, network packet broker 400 may extract the network statistics in the NetFlow packets and insert the network statistics into sFlow datagrams.

FIG. 9 is a flow chart illustrating an exemplary process for performing network flow metadata processing at a network packet broker. Referring to FIG. 9, in step 900, the process includes receiving, as input at a network packet broker, a network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow. For example, network packet broker 400 may receive NetFlow, IxFlow, sFlow, or other network metadata at one of its network ports. The network device that produces the network flow metadata may be a network tap, router, or switch configured to produce such records.

In step 902, the process includes, accessing, by the network packet broker, a network metadata processing rules database and identifying a network flow metadata processing rule to apply to network flow metadata. For example, network flow metadata processing engine 406 may perform a lookup in network metadata processing rules database 408 to identify one or more network metadata processing rules to apply to ingress network flow metadata. The lookup may be performed based on one or more parameters in the received network metadata record. For example, the source address and destination address may be used to extract a network metadata processing rule for a particular packet flow.

In step 904, the process includes processing, by the network packet broker, network flow metadata using the NetFlow flow metadata processing rules. For example, network flow metadata processing engine 406 may perform one or more of deduplication, replication, aggregation, enrichment, decryption and encryption, or other processing on ingress NetFlow data based on the rule or rules accessed in database 408.

In step 906, the process includes, forwarding, by the network packet broker and based on the results of the processing, egress network flow metadata to a network tool. For example, a network packet broker 400 may output deduplicated, replicated, aggregated, decrypted and re-encrypted, and/or enriched network metadata records to one or more network tools.

By placing network metadata processing functionality at a network packet broker, the inflexibility of the conventional NetFlow distribution architecture illustrated in FIG. 1 is avoided. For example, it is possible to redirect network flow metadata streams to different network tools when one network tool becomes unavailable without having to reconfigure the network metadata generators. It is also possible to apply network packet broker functions, such as deduplication, replication, and other functions to network metadata streams.

It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.

Claims

1. A method for processing network flow metadata at a network packet broker, the method comprising: receiving, as input at a network port of the network packet broker, the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata;wherein the processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata; andforwarding, by the network packet broker, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the receiving the network flow metadata includes further receiving at least two network flow metadata records, the identifying the network flow metadata processing rule includes further identifying a network flow metadata deduplication processing rule to apply to the at least two network flow metadata records, the processing network flow metadata includes performing deduplication processing of the at least two network flow metadata records, and the forwarding the egress network flow metadata includes further forwarding one of the at least two network flow metadata records to a single tool port.

2. A method for processing network flow metadata at a network packet broker, the method comprising: receiving, as input at a network port of the network packet broker, the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata;wherein the processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata; andforwarding, by the network packet broker, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the receiving the network flow metadata includes further receiving a single network flow metadata record, the identifying the network flow metadata processing rule includes further identifying a network flow metadata replication rule to apply to the single network flow metadata record, the processing the network flow metadata includes replicating the single network flow metadata record, and the forwarding the egress network flow metadata includes further forwarding at least two copies of the single network flow metadata record to at least two different network tools.

3. A method for processing network flow metadata at a network packet broker, the method comprising: receiving, as input at a network port of the network packet broker, the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata;wherein the processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata; andforwarding, by the network packet broker, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the receiving the network flow metadata includes further receiving an encrypted network flow metadata record, the identifying the network flow metadata processing rule includes further identifying a network flow metadata decryption, processing, and re-encryption rule to apply to the network flow metadata, the processing the network flow metadata includes decrypting the network flow metadata record using a decryption key obtained via a secure backchannel interface to produce a decrypted network metadata record, performing at least one operation on the decrypted network flow metadata record, re-encrypting the decrypted network flow metadata record to produce a re-encrypted network flow metadata record, and the forwarding the egress network flow metadata includes further forwarding the re-encrypted network metadata flow record to the network tool.

4. The method of claim 3 wherein the performing at least one operation on the decrypted network flow metadata record includes further performing at least one of: network flow metadata deduplication, replication, aggregation, and format translation processing on the decrypted network flow metadata record.

5. A method for processing network flow metadata at a network packet broker, the method comprising: receiving, as input at a network port of the network packet broker, network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata;wherein the processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata; andforwarding, by the network packet broker, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the method further comprises receiving packet copies and network flow log data,wherein the identifying the network flow metadata processing rule includes further identifying a network flow metadata aggregation rule to apply to the network flow metadata, the packet copies, and the network flow log data,wherein the processing the network flow metadata includes aggregating the network flow metadata with at least one of the packet copies and the network flow log data to create an aggregated network flow metadata record, andwherein the forwarding the egress network flow metadata includes further forwarding the aggregated network flow metadata record to the network tool.

6. The method of claim 1 wherein the receiving the network flow metadata includes further receiving a network flow metadata record formatted according to a first network flow metadata format, the identifying the network flow metadata processing rule includes further identifying a network flow metadata format translation rule to apply to the network flow metadata record, the processing the network flow metadata includes translating a format of the network flow metadata from the first network flow metadata format to a second network flow metadata format different from the first network flow metadata format to produce a translated network flow metadata record, and the forwarding the egress network flow metadata includes further forwarding the translated network flow metadata record to the network tool.

7. The method of claim 6wherein the first network flow metadata format comprises a first network flow metadata record format,wherein the second network flow metadata format includes a second network flow metadata record format different from the first network flow metadata record format, andwherein the translating the format of the network flow metadata includes further translating the format of the network flow metadata from the first network flow metadata record format to the second network flow metadata record format.

8. A system for processing network flow metadata, the system comprising: a network packet broker including a network port, at least one processor and a memory, the network port for receiving the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein receiving the network flow metadata generated by the network device includes receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;a network flow metadata processing rules database stored in the memory and containing rules for processing the network flow metadata; anda network flow metadata processing engine implemented by the at least one processor for accessing the network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata, for the processing the network flow metadata using the network flow metadata processing rule to generate or identify egress network flow metadata, and for forwarding, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the network flow metadata processing engine is configured to receive at least two network flow metadata records, identify a network flow metadata deduplication processing rule to apply to the at least two network flow metadata records, perform deduplication processing of the at least two network flow metadata records, and forward one of the at least two network flow metadata records to a single tool port.

9. A system for processing network flow metadata the system comprising: a network packet broker including a network port, at least one processor and a memory, the network port for receiving the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch a span port of the router, or a span port of the switch;a network flow metadata processing rules database stored in the memory and containing rules for processing the network flow metadata; anda network flow metadata processing engine implemented by the at least one processor for accessing the network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata, for the processing the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata, and for forwarding the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to a network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the network flow metadata processing engine is configured to receive a single network flow metadata record, identify a network flow metadata replication rule to apply to the single network flow metadata record, replicate the single network flow metadata record, and forward at least two copies of the single network flow metadata record to at least two different network tools.

10. A system for processing network flow metadata, the system comprising: a network packet broker including a network port, at least one processor and a memory, the network port for receiving the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;a network flow metadata processing rules database stored in the memory and containing rules for the processing the network flow metadata; anda network flow metadata processing engine implemented by the at least one processor for accessing the network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata, for the processing the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata, and for forwarding, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to a network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the network flow metadata processing engine is configured to receive an encrypted network flow metadata record, identify a network flow metadata decryption, processing, and re-encryption rule to apply to the network flow metadata, decrypt the network flow metadata record using a decryption key obtained via a secure backchannel interface to produce a decrypted network metadata record, perform at least one operation on the decrypted network flow metadata record, re-encrypt the decrypted network flow metadata record to produce a re-encrypted network flow metadata record, and forward the re-encrypted network metadata flow record to the network tool.

11. The system of claim 10 wherein the network flow metadata processing engine is configured to perform at least one of: network flow metadata deduplication, replication, aggregation, enrichment, and format translation processing on the decrypted network flow metadata record.

12. A system for processing network flow metadata, the system comprising: a network packet broker including a network port, at least one processor and a memory, the network port for receiving the network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch, a span port of the router, or a span port of the switch;a network flow metadata processing rules database stored in the memory and containing rules for the processing the network flow metadata; anda network flow metadata processing engine implemented by the at least one processor for accessing the network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata, for the processing the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata, and for forwarding, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the network flow metadata processing engine is configured to receive packet copies and network flow log data, identify a network flow metadata aggregation rule to apply to the network flow metadata, the packet copies, and the network flow log data, aggregate the network flow metadata with at least one of the packet copies and the network flow log data to create an aggregated network flow metadata record, and forward the aggregated network flow metadata record to the network tool.

13. The system of claim 8 wherein the network flow metadata processing engine is configured to receive a network flow metadata record formatted according to a first network flow metadata format, identify a network flow metadata format translation rule to apply to the network flow metadata record, translate a format of the network flow metadata from the first network flow metadata format to a second network flow metadata format different from the first network flow metadata format to produce a translated network flow metadata record, and forward the translated network flow metadata record to the network tool.

14. The system of claim 13wherein the first network flow metadata format comprises a first network flow metadata record format,wherein the second network flow metadata format includes a second network flow metadata record format, andwherein the network flow metadata processing engine is configured to translate the network flow metadata from the first network flow metadata record format to the second network flow metadata record format.

15. A non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps comprising: receiving, as input at a network port of a network packet broker, network flow metadata, the network flow metadata including a network flow statistic generated by a network device regarding packets in a network flow,wherein the receiving the network flow metadata generated by the network device includes further receiving network flow metadata records from one of: a network tap, a tap port of a router, a tap port of a switch a span port of the router, or a span port of the switch;accessing, by the network packet broker, a network flow metadata processing rules database and identifying a network flow metadata processing rule to apply to the network flow metadata;processing, by the network packet broker, the network flow metadata using the network flow metadata processing rule to produce or identify egress network flow metadata; andforwarding, by the network packet broker, the egress network flow metadata to a network tool,wherein the forwarding the egress network flow metadata to the network tool includes further forwarding the egress network flow metadata to a network flow metadata collector,wherein the receiving the network flow metadata includes further receiving at least two network flow metadata records, the identifying a network flow metadata processing rule includes further identifying a network flow metadata deduplication processing rule to apply to the at least two network flow metadata records, the processing the network flow metadata includes performing deduplication processing of the at least two network flow metadata records, and the forwarding the egress network flow metadata includes further forwarding one of the at least two network flow metadata records to a single tool port.