Prioritizing network traffic

Abstract

Methods, systems and apparatus, including computer programs encoded on a computer storage medium, for receiving, at a global server system, from each of a plurality of local network devices, network data specifying network communication activity at the local network device, wherein the plurality of local network devices collectively provide backbone communications facilities for multiple networks; aggregating, at the global server system, the network data from each of the local network devices; analyzing, at the global server system, the aggregated network data to identify network activities; generating, at the global server system, update data based on the analysis of the aggregated network data, the update data including instructions for the local network devices for processing network communications to or from the local network devices; and transmitting from the global server system the update data to the local network devices.

Description

TECHNICAL FIELD

This document relates generally to systems and methods for processing communications and more particularly to systems and methods for prioritizing network traffic.

BACKGROUND

Internet connectivity has become central to many daily activities. For example, millions of people worldwide use the internet for various bill pay and banking functionalities. Countless more people use the internet for shopping, entertainment, to obtain news, and for myriad other purposes. Moreover, many businesses rely on the internet for communicating with suppliers and customers, as well as providing a resource library for their employees. Further, many governmental agencies use the internet for responding to various emergencies.

However, because of the large volume of malicious and unsolicited internet traffic, network resources and bandwidth are heavily burdened thus potentially preventing or delaying important and legitimate traffic from timely reaching its destination.

SUMMARY

In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving, at a global server system, from each of a plurality of local network devices, network data specifying network communication activity at the local network device, wherein the plurality of local network devices collectively provide backbone communications facilities for multiple networks; aggregating, at the global server system, the network data from each of the local network devices; analyzing, at the global server system, the aggregated network data to identify network activities; generating, at the global server system, update data based on the analysis of the aggregated network data, the update data including instructions for the local network devices for processing network communications to or from the local network devices; and transmitting from the global server system the update data to the local network devices.

Other embodiments of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. Managing network traffic across multiple networks based on real-time feedback from a distributed set of local network devices, in the multiple networks, reporting network activities (e.g., network volume and concentration) to reduce or mitigate network security risks in real-time. Using the reported network activities to identify local network devices that pose a security risk or may be exposed to a security risk and tactically managing the network communication channels used and/or controlled by those identified local network devices to reduce or mitigate the security risk without unnecessary impacting network traffic through local network devices that are not involved with the security risk.

The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

DESCRIPTION OF DRAWINGS

FIG. 1A is a block diagram illustrating an example network topology including reputation based routing systems.

FIG. 1B is a block diagram illustrating an example network topology for distribution of reputation information.

FIG. 2 is a block diagram illustrating an example reputation based routing system receiving reputation information from a reputation system.

FIG. 3 is a block diagram illustrating an example of a reputation based routing system including a local cache of reputation information.

FIG. 4 is a block diagram illustrating another example of a reputation based routing system including a delay module.

FIG. 5 is a block diagram illustrating another example of a reputation based routing system including a classification module.

FIG. 6 is a block diagram illustrating another example of a reputation based routing system including classification retrieval.

FIG. 7 is a flow diagram illustrating an example reputation based prioritization of network traffic.

FIG. 8 is a flow diagram illustrating an example prioritization of network traffic based upon reputation and classification information.

DETAILED DESCRIPTION

Reputation based prioritization of network traffic can include providing reputation based policy to routing devices (e.g., routers). Routers typically inspect packets to extract destinations associated with the data packets and retrieve routing information associated with the destinations before communicating the data packets to the recipient (or to another router). During the retrieval of routing information, reputation information associated with an originating entity and/or a destination entity can be retrieved. The reputation information can provide an indication of whether the traffic associated with the data packets is non-reputable (e.g., malicious, unsolicited, etc.). The reputation based prioritization system can then prioritize the traffic based upon reputation information associated with the device.

FIG. 1 is a block diagram illustrating an example network topology including reputation based routing systems 100a-c. The reputation based routing systems 100a-c can be modules of network 110. The reputation based routing systems can communicate with a reputation system 120, which can be operable to distribute reputation information from a reputation store 130. The reputation based routing systems 100a-c can provide backbone communications facilities for the network 110 to communicate data packets between entities 140a-o.

In various implementations, the entities 140a-o can include any of internet protocol addresses, domain names, universal resource locators, devices (e.g., as identified by a media access control (MAC) address), or user identity, company identity, among many others. Thus, many different entities can be associated with a single device. For example, a device can perform as a web server for many different URLs and/or domain names, or the device might have several different users resulting in several different user identities. Moreover, the device might be dynamically addressed resulting in the use of several different IP addresses. Thus, in various implementations, the manifestations of a device can be tracked separately from each other (or in combination).

The entities 140a-o can access the network 110 in a variety of different manners. In some examples, the entities 140a-o can be any type of local area networks (LANs) or wide area networks (WANs). In other examples, the networks can be networks operated by a company, or a school or university to enable workers/students to access the internet for research, communications, acquisition, etc. In still further examples, some entities 140c, 140e, 140h, 140k, 140o can be internet service providers providing internet service to still more entities (not shown).

The reputation based routing systems 100a-c can include route processing information facilitating routing a communications from one entity to another. For example, Entity A 140a can communicate with Entity I 140i by sending data packets to an associated router (e.g., reputation based routing system 100a). The reputation based routing system 100a can parse the data packets to identify a destination associated with the data packets. The reputation based routing system can identify a second router (e.g., reputation based routing system 100c) based upon routing tables associated with the reputation based routing system 100a. The reputation based routing system 100a can communicate the data packets to the other reputation based routing system 100c based on the reputation associated with the originating entity or destination entity associated with the data packets.

The reputation of the originating and/or receiving entities can be retrieved from a reputation system 120. In some implementations the reputation system 120 can include a reputation server that can serve reputation information to other reputation based devices (e.g., reputation based routing systems 100a-c). In other implementations, the reputation system 120 can include a distributed reputation system. For example, a distributed reputation system can include a global reputation server and a number of local reputation devices. In various implementations, the reputation server can periodically push reputation updates to other reputation based devices (e.g., reputation based routing systems 100a-c).

In some implementations, communication of updated reputation information can be relayed from one reputation based routing system (e.g., reputation based routing system 100a) to another reputation based routing system (e.g., reputation based routing system 100c) where there is no direct connection between the reputation system 120 and the reputation based routing system 100c. In such implementations, the updated reputation information can be communicated securely to the reputation based routing system 100c to prevent tampering. In other implementations, updated reputation information can include credentials authenticating the reputation update. For example, the reputation system 120 can generate a CRC checksum of the reputation update which must match a CRC checksum of the reputation update generated by the receiving reputation device before a reputation update is applied.

In some implementations, the reputation of various entities that are tracked can be derived based upon activities in which those entities take part. For example, if an entity consistently originates spam communications, the entity can be classified with a reputation as a spammer. Alternatively, if the entity consistently originates reputable communications, the entity can be classified with a reputation as a reputable sender.

In additional implementations, the reputation of the originating and/or receiving devices can be derived based upon relationships derived between the entities. The relationship can be derived based upon any of communications between the entities, traffic patterns (e.g., similar increases and/or decreases in traffic volume) associated with the entities, similar communications originating from the entities independently, sporadic communication patterns, or use of commonly spoofed address (e.g., IP, MAC, URL, domain, etc.), among many others. For example, a first entity that has an indeterminate classification might be identified as communicating consistently with a second entity that has a reputation for originating botnet traffic (e.g., a network of malware infected computers that surreptitiously originate, e.g., spam traffic). Thus, while the reputation of the first entity might be indeterminate, a portion of the reputation of the second entity can be applied to the first entity based upon a relationship identified between the first and second entities. Alternatively, if a first entity with an indeterminate reputation consistently communicates with a second entity having a reputation for originating/receiving reputable traffic the reputation of the first entity can be biased towards classification as a reputable entity.

In some implementations, the reputation for certain activities can be time or location based. For example, an entity associated with a business might consistently show activity between a period of 6:00 AM and 7:00 PM. Thus, if the entity shows uncommon activity outside of that time period the reputation of the entity might be classified differently during business hours than it is overnight. Similarly, an entity might show consistent origination of traffic from a given geolocation (e.g., based upon a registered location or a first router that receives communications from the entity). Communications received from a different geolocation that claim to be associated with the same entity can be treated as suspect and/or the reputation of an entity can be identified as non-reputable based upon the geolocation associated with the entity. In other implementations, the fact that an entity is being used for non-reputable activities can lead to the determination that the entity is not being properly secured and/or policed by an owner. In such implementations, the reputation of the entity can be biased towards a non-reputable category, even if an owner of the entity acts reputably with regard to the entity.

In further implementations, the reputation can be based upon multiple entity attributes. For example, a domain might have a reputation for phishing when the domain is associated with a particular IP address. Thus, the correlation of the domain and the IP address can be assigned a reputation for spoofing while the domain separate from the IP address might retain a reputation for reputable traffic. In other implementations, the fact that an entity is being used for non-reputable (e.g., phishing) activities can lead to the determination that an otherwise reputable entity (e.g., the reputable domain) is not being properly secured and/or policed. In such implementations, the reputation of the domain can be biased towards a non-reputable category based upon such activity, even if an otherwise reputable entity takes no part in the non-reputable activity exhibited by someone disguising themselves with the entity.

A complete description of the reputation derivation processes can be found, for example, in U.S. patent application Ser. No. 11/142,943, entitled “Systems and Methods for Classification of Messaging Entities,” filed on Jun. 2, 2005, which application is hereby incorporated by reference in its entirety. Other descriptions of reputation systems can be found in: U.S. patent application Ser. No. 11/626,462, entitled “Correlation and Analysis of Entity Attributes,” filed on Jan. 24, 2007, which application is hereby incorporated by reference in its entirety; U.S. patent application Ser. No. 11/626,470, entitled “Web Reputation Scoring,” filed on Jan. 24, 2007, which application is hereby incorporated by reference in its entirety; U.S. patent application Ser. No. 11/626,479, entitled “Aggregation of Reputation Data,” filed on Jan. 24, 2007, which application is hereby incorporated by reference in its entirety; U.S. patent application Ser. No. 11/626,603, entitled “Multi-Dimensional Reputation Scoring,” filed on Jan. 24, 2007, which application is hereby incorporated by reference in its entirety; and, U.S. patent application Ser. No. 12/020,370, entitled “Reputation based Message Processing,” filed on Jan. 25, 2008, which application is hereby incorporated by reference in its entirety. The reputation retrieval module 220, in some examples, can retrieve reputation information provided by a TrustedSource™ database, available from Secure Computing Corporation of San Jose, Calif.

In some implementations, the analysis of the activities in which an entity participates can take place separately from the reputation based routing system(s) 100a-c. Such separate analysis of the activities associated with the entities can help to facilitate efficient routing of communications by the routing systems 100a-c. The reputation information derived thereby can be pushed to the reputation based device (e.g., including reputation based routing systems 100a-c).

In other implementations, analysis of the activities in which an entity participates can be provided by the reputation based routing systems 100a-c, or can be distributed to other reputation devices based upon processor utilization by a respective reputation based routing system 100a-c.

FIG. 1B is a block diagram illustrating an example network topology for distribution of reputation information. The network topology of FIG. 1B illustrates a larger network of reputation based routing systems 100d-n than depicted in FIG. 1A, along with a distributed reputation system 120a-d. In the example of FIG. 1B, the reputation based routing systems 100d-n provide communications paths for network entities (not shown). In some examples, communication between two entities might include several hops (e.g., handling by multiple routers between an originating entity and a destination entity).

In some implementations, when more than one hop is defined in the path of a communication from originating entity to destination entity, a reputation determination might occur only once between source and destination. Reputation based routing systems 100d-n can notify subsequent reputation based routing systems 100d-n that policy has already been applied to the data packet. In such implementations, a secure notification can be used to communicate the previous application of policy to other reputation based routing systems 100d-n in a path from originating entity to destination entity. In further implementations, notification of the application of policy to a stream can include a temporal limitation. For example, if a new policy or updated reputation is received after a notification that policy has already been applied to the communication, the application of policy to the communication stream is no longer current. Thus, the new policy and/or reputation can be used to determine whether the data is to be communicated to a next hop or destination entity or the data is to be dropped entirely or merely delayed. Such implementations as described above can facilitate the efficient handling of data such that a particular communication is not queried multiple times in the path from originating entity to destination entity.

In other implementations, when more than one hop is defined in the path of a communication from originating entity to destination entity, each reputation based routing system 100d-n in the path from originating entity to destination entity can retrieve reputation information associated with the originating and/or destination entities and apply policy to the communication. Such implementations can reduce the amount of analysis the reputation based routing systems 100d-n perform on the data to determine whether to apply policy and avoid problems with fraudulent generation of notification of previous application of reputation based policy to the data.

In some implementations, a distributed reputation system 120a-d can be used to distribute reputation information to reputation based routing systems 100d-n. A distributed reputation system 120a-d can reduce propagation delays in applying reputation updates to reputation based routing systems 100d-n, especially where it eliminates multiple hops between the reputation system 120a-d and the reputation based routing systems 100d-e.

Local reputation servers 120b-d can be placed throughout the network to provide reputation updates to reputation based routing systems 100d-n. As described previously, the reputation updates can be securely communicated to the reputation based routing systems 100d-n, or provided with a CRC checksum to be independently verified prior to application of the reputation update by the reputation based routing system 100d-n. In those instances when a potentially fraudulent reputation update is received, a notification of the failed reputation update can be communicated to a central reputation server (e.g., global reputation server 120a). In some implementations, the global reputation server 120a can provide a fresh reputation updated to a notifying reputation based routing system 100d-n (e.g., securely, along with credentials, etc.).

In some implementations, a global reputation server 120a can also provide certain reputation based routing systems (e.g., reputation based routing systems 100a, 100h, 100k) with reputation updates. In some examples, the reputation updates provided by the global reputation system can be provided to nearby reputation based routing systems 100a, 100h, 100k. In other examples, the global reputation server 120a can provide reputation updates to logically important (e.g., high volume) reputation based routing devices.

The global reputation server 120a can aggregate reputation information received from local reputation servers 120b-d. Aggregation of reputation information is described in detail in U.S. patent application Ser. No. 11/626,479, entitled “Aggregation of Reputation Data,” filed on Jan. 24, 2107, incorporated by reference above.

Distributed reputation systems 120a-d can provide for more frequent updates of reputation information. Moreover, because local reputation servers 120b-d update reputation information based upon data observed by the local reputation server 120b-d, the update is likely to be more relevant to the particular data being routed by the reputation based routing system 100d-n. For example, a local reputation server 120b is more likely to see data from entities that communicate often over the reputation based routing systems 100e, 100f, 100i. This is because the reputation based routing systems 100e, 100f, 100i to whom the local reputation server 120b provides reputation updates also provide the local reputation server 120b with data being communicated across the network. Moreover, the local reputation servers 120b can be distributed in a similar logical space or nearby physical space to the reputation based routing systems 100e, 100f, 100i that they serve.

FIG. 2 is a block diagram illustrating an example reputation based routing system 200 receiving reputation information from a reputation system 120. The reputation based routing system 200 can receive incoming communications from an originating entity 140a, e.g., directly from the originating entity 140a or indirectly through another reputation based routing system or through another device (e.g., gateway, internet service provider, legacy router, etc.).

The reputation based routing system 200 can include route processing 210, reputation retrieval 220 and a prioritization module 230. The route processing module 210 can parse incoming data to identify an originating entity associated with the data and a destination entity associated with the data. In some implementations, the route processing module 210 can provide basic functionalities traditionally associated with a router device. The route processing module 210 can also receive a prioritization signal from the prioritization module 230. The prioritization signal can facilitate the prioritization of routing of certain data packets (e.g., those with specified reputation(s)) over other data packets (e.g., those data packets with other reputation(s)).

In some implementations, the reputation retrieval module 220 can retrieve reputation information from reputation system 120. As discussed above, the reputation system 120, in various implementations, can be provided centrally from a single server or distributed across numerous servers. Reputation can be derived based upon attributes (e.g., observed actions, relationships, etc.) associated with an entity. Actions that occur in recognizable patterns can be abstracted into behaviors. A specified set of behaviors can be associated with reputation classifications. The attributes, behaviors and classifications associated with the various entities can be stored in a reputation store 130 by the reputation system 120. The reputation system 120 can retrieve the reputation information associated with a specified entity from the reputation store 130. In some implementations, the reputation system 120 can provide the reputation information to a reputation retrieval module 220 upon receiving a retrieval request from the reputation retrieval module 220.

The reputation retrieval module 220, upon receiving reputation information associate with the originating entity and/or receiving entity can forward the reputation information to a prioritization module 230. The prioritization module 230 can prioritize the transmission of data by the route processing module 210 through a prioritization signal provided to the route processing module 210.

In some implementations, prioritization of the data can be based upon a prioritization policy provided by an administrator 240. The prioritization policy provided by the administrator 240 can specify that data originating from specified classes of reputations are to be transmitted, for example, with low priority (e.g., after other traffic), dropped, quarantined for further testing or information gathering, etc., and/or that specified classes of reputations are to be transmitted, for example, with high priority (e.g., before other traffic). In some implementations, if a network is bandwidth limited, a connection for traffic with low priority can be dropped in order to provide a connection for traffic with high priority.

In some implementations, a special entity can be generated that can be recognized by the reputation based routing system and can route traffic associated with the special entity prior to routing other traffic. For example, in states of emergency internet traffic often drastically increases in volume leading to a bandwidth limited situation. Such a rise in traffic can often lead to slower throughput for all traffic. Alternatively, when a network is being clogged by a distributed denial of service attack it can be difficult for an administrator to get the bandwidth necessary in such a bandwidth limited situation to shut the attack down remotely. In such examples, it can often be difficult for those individuals with the means to solve the problem to adequately communicate the solution (e.g., a system administrator might have difficulty remotely communicating with a server/firewall to shut down a distributed denial of service attack because network routers are jammed with denial of service requests). Thus, as described above, a special entity can be generated to provide unimpeded access to the network by those special entities, whereby other users will be dropped in order to provide any requested bandwidth to the special entity.

In some implementations, the route processing module 210 can operate in parallel to the reputation processing, thereby increasing the efficiency of the reputation lookup and prioritization decision.

FIG. 3 is a block diagram illustrating an example of a reputation based routing system 300 including a local cache 310 of reputation information. In the example of FIG. 3, a reputation based routing system 200 can use a local reputation store 310 to locally cache reputation information from the reputation system 120. Such caching with a local reputation store 310 can reduce delays associated with retrieving reputation information from remotely located reputation systems, and provide reputation information locally to reputation based routing systems.

Routers often have limited resources for additional processing. Thus, the resources within the router can be conserved by limiting the amount of reputation information locally cached at the local reputation store 310 by the reputation based routing system 300. In some implementations, the local cache can include a least recently used (LRU) algorithm operable to push a least recently used reputation information entry out of the cache upon receipt of a new reputation information entry. In some examples, entries that are retrieved from the LRU stack can be re-entered at the top of the stack, thereby preserving their existence in the stack until the stack has been cycled without receipt of data specifying the reputation information associated with the entry. Thus, data which is most commonly requested by the retrieval module remains in the local cache the longest, while data which is not regularly requested by the retrieval module is not stored in the local cache 310. Other stacking algorithms, e.g., including least frequently used stacking algorithms, can be implemented.

In other implementations, the local reputation store 310 can comprise at least a partial mirror of the reputation data store 130. In those implementations in which only a portion of the reputation data store 130 is mirrored at the local reputation store 310 it can be difficult to accurately determine which portion of the reputation data store 130 should be mirrored by the local reputation store 310.

In some implementations, the reputation system 120 can use a Bloom filter to provide a probabilistic determination of the particular reputation information which is to be included in the local reputation store 310. Use of a Bloom filter on the reputation dataset can reduce the size of the dataset stored on the reputation based routing system 300 and reduce access time for retrieving the data.

In some implementations, the reputation system 120 can identify the particular reputation information which is most likely to be used by the reputation based routing system 300. The reputation system 120 can also allow the reputation retrieval module 210 to query the reputation system 120 if a communication associated with an entity not in the local reputation store 310 is received. For example, if reputation information for entities A, C, E, F, and G are stored in the local reputation store 310, and the reputation based routing system receives data originating from entity D, the reputation retrieval module can query the reputation system 120 for reputation information associated with entity D.

In some implementations, updates to the local reputation store 310 can be performed periodically. Reputation information migrates over time based upon additional data collected by the reputation system 120. Thus, the reputation information stored by the local reputation store 310 can become stale. In some implementations, the reputation system 120 can keep track of the reputation information stored by the local reputation store 310 and can compare the version of the reputation information stored by the local reputation store 310 to the current version and provide a reputation update that includes only reputation information that has changed since a previous update.

In some implementations, the reputation system 120 can push reputation updates to the local reputation store 310, e.g., during periods of forecasted low activity. The forecasted low activity can be based upon historical usage of the network. In other implementations, the reputation based routing system 300 can signal periods of low activity to the reputation system 120. The reputation system 120 can handle such signals as requests to apply a reputation update. Other reputation update procedures can be used.

In additional implementations, the reputation system 120 can receive feedback from the reputation retrieval module (e.g., reputation retrieval module 210a). The feedback can indicate how often reputation for various entities is being retrieved. Such feedback can be used to modify the reputation system to provide reputation updates for the most often requested entities. In some implementations, the feedback can be generalized by physical proximity (e.g., region, location, etc.) of the reputation based routing systems. For example, if feedback from a reputation based routing system indicates that entity A is being requested often, the reputation system can provide the reputation for entity A to all reputation based routing systems in the same region or location. In other implementations, the feedback can be generalized by logical proximity of the reputation based routing systems. For example, a reputation based routing system serving a certain type of traffic might identify that a reputation for entity B is being requested frequently. The reputation based routing system can provide a reputation update including entity B to all other reputation based routing systems routing the same type of traffic. In additional implementations, the reputation system can receive information from external sources indicating a rise in activity by specified entities. In still further implementations, the reputation system can analyze the feedback to identify a temporal component/dependency to the activity of certain entities. The reputation system can provide reputation updates that account for the temporal component to the activity of certain entities by providing reputation updates that include those entities only between certain hours of the day, based upon the temporal component associated with the entities' activities.

FIG. 4 is a block diagram illustrating another example of a reputation based routing system 400 including a delay module 410. In some implementations, the reputation based routing system 400 can include a delay module 410 to delay routing of communications based upon a reputation of one or more entities 140a, 140b associated with the communications. The routing of communications can be delayed based upon application of a prioritization policy associated with a reputation based routing system 400 to the reputation of an entity 140a, 140b associated with the communications.

In some implementations the prioritization policy can delay routing of communications based on an indeterminate reputation associated with one or more of the entities 140a, 140b associated with the communications. When a reputation is identified as indeterminate by the reputation retrieval module 210a, a prioritization module 220a can apply a prioritization policy to the packet based upon the reputation. In some examples, the prioritization policy can specify that a packet with an indeterminate reputation is sent to a delay module 410.

The delay module 410 can hold the packet for a period of time before resubmitting the packet to the reputation retrieval module 210a. In some implementations, routing of the packet can be delayed by the prioritization module 220a in conjunction with the delay module 410 until a reputation is determinate. In other implementations, communications can be dropped after a predefined period or number of cycles during which the reputation of one or more entities 140a, 140b associated with the communications remain indeterminate. In still further implementations, communications that are associated with an entity 140a, 140b with a reputation that remains indeterminate after a predefined period of time or number of cycles is communicated to a destination entity 140b.

FIG. 5 is a block diagram illustrating another example of a reputation based routing system 500 including a classification module 510. In some implementations, the reputation based routing system 500 can include route processing 200, reputation retrieval 520, a classification module 510 and a prioritization module 530. The route processing module 200 can receive incoming communications from an originating entity 140a or some other entity (e.g., including another reputation based route processing system of any of the implementations described herein). The route processing module can extract originating entity 140a and destination entity 140b information associated with the incoming communications and process a route associated with the communications based upon the application of a routing table to the destination entity 140b.

The route processing module can also forward the packet and identification of the extracted originating entity 140a and destination entity 140b information to a reputation retrieval module 520. The reputation retrieval module 520 can identify reputation information associated with the originating entity 140a and or destination entity 140b.

In some implementations, if the reputation of an entity 140a, 140b associated with the communications is indeterminate, the reputation retrieval module can notify the prioritization module 530 and send the communications to a classification module 510. The classification module can perform a variety of tests on the communications to identify a class associated with the communication. In various implementations, the classification module can extract features from the communications to derive feature vectors and compare the feature vectors to respective linear classifiers that use those feature vectors to determine whether the feature vector derived from the communications shares features that define the communication as being classified with a classification associated with the respective feature vector. Examples of feature vector classification are described in U.S. patent application Ser. No. 12/020,253, entitled “Granular Support Vector Machine with Random Granularity,” filed on Jan. 25, 2008, which is hereby incorporated by reference in its entirety. Additional classification processes and system are described in detail by: U.S. patent application Ser. No. 11/173,941, entitled “Message Profiling Systems and Methods,” filed on Jul. 1, 2005, which is hereby incorporated by reference in its entirety; and, U.S. patent application Ser. No. 11/383,347, entitled “Content-based Policy Compliance Systems and Methods, filed on May 15, 2006, which is hereby incorporated by reference in its entirety. The classification module 510, in some implementations, can query by a TrustedSource™ database, available from Secure Computing Corporation of San Jose, Calif., which can operate to provide classification definitions against which communications can be compared for classification. Other machine learning classification systems (including other Support Vector Machine (SVM) or Random Forest processes) can be used to classify messages.

The classification module 510 can communicate the derived classification to the prioritization module 530. The prioritization module 530 can apply a prioritization policy received from an administrator 230 to the reputation and/or classification associated with the communications to identify a priority to provide to the communications. In further implementations, the prioritization policy can instruct the prioritization module 530 to drop communications based upon the classification associated with the communications and/or the reputation of one or more entities associated with the communications.

The prioritization module 530 can communicate the prioritization of the communications to the route processing module 200. The route processing module 200 can process the communications based on the received prioritization.

FIG. 6 is a block diagram illustrating another example of a reputation based routing system 600 including classification retrieval 610. In some implementations, the reputation based routing system 500 can include route processing 200, reputation retrieval 210, classification retrieval 610, a prioritization module 620 and an undelivered communications module 630. The route processing module 200 can receive incoming communications from an originating entity 140a or some other entity (e.g., including another reputation based route processing system of any of the implementations described herein). The route processing module can extract originating entity 140a and destination entity 140b information associated with the incoming communications and process a route associated with the communications based upon the application of a routing table to the destination entity 140b.

The route processing module can also forward the packet and identification of the extracted originating entity 140a and destination entity 140b information to a reputation retrieval module 210. The reputation retrieval module 210 can identify reputation information associated with the originating entity 140a and or destination entity 140b, for example, based upon retrieval of the reputation from a reputation system 120. In other examples, the retrieval of the reputation information can be based upon retrieval of reputation information from a local reputation store (e.g., local reputation store 310 of FIG. 3) providing at least a partial mirror of the reputation data store 130.

The prioritization module 530 can send the communications to the prioritization module 620 along with reputation information for one or more of the entities associated with the communication. The prioritization module 620 can apply a prioritization policy to the communication based upon the reputation information received from the reputation retrieval module 210.

In some implementations, application of the prioritization policy can determine that the communication(s) should be sent to a classification retrieval module 610. The classification retrieval module 610 can forward the communications to a classification system 640. The classification system 640 can perform a variety of tests on the communications to identify a class associated with the communication. In various implementations, the classification module can extract features from the communications to derive feature vectors and compare the feature vectors to respective linear classifiers that use those feature vectors to determine whether the feature vector derived from the communications shares features that define the communication as being classified with a classification associated with the respective feature vector. Other classification systems and processes can be used to classify messages.

The classification system 640 can return the identified classification associated with the communication(s) to the classification retrieval module 610. The classification retrieval module 610 can communicate the derived classification to the prioritization module 620. The prioritization module 620 can apply a prioritization policy received from an administrator 230 to the reputation and/or classification associated with the communications to identify a priority for the communications. In further implementations, the prioritization policy can instruct the prioritization module 620 to send the communications to an undelivered communications module 630.

The prioritization module 620 can communicate the prioritization of the communications to the route processing module 200. The route processing module 200 can process the communications based on the received prioritization.

FIG. 7 is a flow diagram illustrating an example reputation based prioritization of network traffic. At stage 700, communications can be received. The communications can be received, for example, by a route processing module (e.g., route processing 210 of FIG. 2). The communication can include one or more data packets, and each of the one or more data packets can identify a communication stream it belongs to as well as source and destination address for routing purposes.

In some implementations, the receipt of communications can cause a reputation based routing system to determine whether the routing system is in a bandwidth limited situation. In a bandwidth limited situation, the reputation based routing system can route the communications based upon reputation associated with the communications.

At stage 710, an originating entity and destination entity of the communications can be identified. The originating entity and destination entity can be identified, for example, by a route processing module (e.g., route processing 200 of FIG. 2). In various implementations, data packets associated with the communication can be parsed to identify an originating entity and a destination entity addresses from the data packet headers. The data packet headers can also identify a data stream to which the data packet belongs. In various implementations, the route processing module can use the originating entity and destination entity addresses to identify a routing of the data packets.

At stage 720, reputation of source entity and destination entity can be retrieved. The source entity and destination entity reputation can be retrieved, for example, by a reputation retrieval module (e.g., reputation retrieval 210 of FIG. 2) in conjunction with a local reputation store (e.g., local reputation store 310 of FIG. 3) and/or a reputation system (e.g., reputation system 120 of FIG. 2). The reputation can be derived remotely from a reputation based routing system using the reputation information. In various implementations, the derived reputation information can be pushed to the reputation based routing system by a reputation system or retrieved from the reputation system directly and locally cached. In those implementations where the reputation information is pushed to the reputation based routing system, a Bloom filter can be used to select the particular dataset of reputation information which is to be pushed to a local reputation store.

At stage 730 a prioritization policy can be applied. The prioritization policy can be applied, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2). In some implementations, the prioritization policy is applied to all communications. In such implementations, the prioritization policy can be based on identifying a bandwidth limited situation and based upon reputation of the entities associated with the communication. In other implementations, the prioritization policy can be applied to communications when route processing has determined that the network is in a bandwidth limited situation. In further implementations, the prioritization policy can be applied to communications when the communications exceed a threshold usage associated with the reputation based routing system.

At stage 740 routing of communication is be prioritized based on reputation. The routing of the communication can be prioritized, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2). In some implementations, the prioritization module can be provided with prioritization policy from an administrator (e.g., admin 240 of FIG. 2). The prioritization policy can define the handling of communications based upon the reputation of one or more of the entities associated with the communications.

FIG. 8 is a flow diagram illustrating an example prioritization of network traffic based upon reputation and classification information. At stage 800 network communications are received. The communications can be received, for example, by a route processing module (e.g., route processing 210 of FIG. 2). The communication can include one or more data packets, and each of the one or more data packets can identify a communication stream it belongs to as well as source and destination address for routing purposes. In some implementations, receipt of communications can cause a reputation based routing system to determine whether the route processing is in a bandwidth limited situation.

At stage 810, the network communications can be parsed to identify an originating entity and destination entity. The originating entity and destination entity can be parsed, for example, by a route processing module (e.g., route processing 200 of FIG. 2). In various implementations, data packets associated with the communication can be parsed to identify an originating entity and a destination entity addresses from the data packet headers. The data packet headers can also identify a data stream to which the data packet belongs. In various implementations, the route processing module can use the originating entity and destination entity addresses to identify a routing of the data packets.

At stage 820, reputation of source entity and destination entity can be retrieved. The source entity and destination entity reputation can be retrieved, for example, by a reputation retrieval module (e.g., reputation retrieval 210 of FIG. 2) in conjunction with a local reputation store (e.g., local reputation store 310 of FIG. 3) and/or a reputation system (e.g., reputation system 120 of FIG. 2). The reputation can be derived remotely from a reputation based routing system using the reputation information. In various implementations, the derived reputation information can be pushed to the reputation based routing system by a reputation system or retrieved from the reputation system directly and locally cached. In those implementations where the reputation information is pushed to the reputation based routing system, a Bloom filter can be used to select the particular dataset of reputation information which is to be pushed to a local reputation store.

At stage 830 it is determined whether the entities are reputable. The determination of whether the entities are reputable can be made, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2).

If the entities are reputable, a prioritization policy can be applied to the communications at stage 840. The prioritization policy can be applied, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2). In some implementations, the prioritization module can be provided with prioritization policy from an administrator (e.g., admin 240 of FIG. 2). The prioritization policy can define the handling of communications based upon the reputation of one or more of the entities associated with the communications.

At stage 850, the data packets can be routed based on priority. The routing of the communication can be routed, for example, by a route processing module (e.g., route processing 200 of FIG. 2). In some implementations, the route processing module can retrieve a routing table and identify routing based on the routing table. In further implementations, the route processing module can prioritize routing of communications with higher priority over those with lower priority. For example, if a communication with high priority is identified, a connection associated with a low priority communication can be dropped. In other examples, communications with lower priorities can be delayed until higher priority communications have been routed.

Returning to the reputable entity determination stage (830), if it is determined that the communication is associated with a non-reputable entity, classification of the communication can be retrieved at stage 860. Classification of communications can be retrieved, for example, by a classification retrieval module (e.g., classification retrieval module 610 of FIG. 6). In some implementations, the classification retrieval module can retrieve classification information based upon querying a classification system. In other implementations, the classification retrieval module can retrieve classification definitions (e.g., SVM linear classification vectors), derive feature vectors from the communication, and compare the feature vector to the linear classification vector to determine whether the communication belongs to a classification associated with the linear classification vector. Other classification methods can be used.

At stage 870 it is determined whether the communication is legitimate. The determination of whether the communication is legitimate can be made, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2).

If the communication is legitimate, a prioritization policy can be applied to the communications at stage 880. The prioritization policy can be applied, for example, by a prioritization module (e.g., prioritization module 230 of FIG. 2). In some implementations, the prioritization module can be provided with prioritization policy from an administrator (e.g., admin 240 of FIG. 2). The prioritization policy can define the handling of communications based upon the classification of the communication in lieu of the reputation of the entities associated with the communications.

At stage 850, the data packets can be routed based on priority. The routing of the communication can be routed, for example, by a route processing module (e.g., route processing 200 of FIG. 2). In some implementations, the route processing module can retrieve a routing table and identify routing based on the routing table. In further implementations, the route processing module can prioritize routing of communications with higher priority over those with lower priority. For example, if a communication with high priority is identified, a connection associated with a low priority communication can be dropped. In other examples, communications with lower priorities can be delayed until higher priority communications have been routed.

Returning to the legitimate communication determination stage (870), if the communication is determined not to be legitimate, the communication can be dropped, quarantined, delayed, etc. at stage 890. The communication can be dropped, quarantined, delayed, etc., for example, by an undelivered message module (e.g., undelivered message module 630 of FIG. 6). In some implementations, the particular handling (e.g., drop, quarantine, delay, etc.) can be specified by the prioritization policy applied to the communication. Other communication handling mechanisms can be specified based upon the prioritization policy.

Use of reputation in prioritization of network traffic as it relates to network routing is also disclosed in U.S. patent application Ser. No. 11/937,274, entitled “Prioritizing Network Traffic,” filed on Nov. 8, 2007, which is hereby incorporated by reference in its entirety.

The systems and methods disclosed herein may use data signals conveyed using networks (e.g., local area network, wide area network, internet, etc.), fiber optic medium, carrier waves, wireless networks (e.g., wireless local area networks, wireless metropolitan area networks, cellular networks, etc.), etc. for communication with one or more data processing devices (e.g., mobile devices). The data signals can carry any or all of the data disclosed herein that is provided to or from a device.

The methods and systems described herein may be implemented on many different types of processing devices by program code comprising program instructions that are executable by one or more processors. The software program instructions may include source code, object code, machine code, or any other stored data that is operable to cause a processing system to perform methods described herein.

The systems and methods may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.) that contain instructions for use in execution by a processor to perform the methods' operations and implement the systems described herein.

The computer components, software modules, functions and data structures described herein may be connected directly or indirectly to each other in order to allow the flow of data needed for their operations. It is also noted that software instructions or a module can be implemented for example as a subroutine unit of code, or as a software function unit of code, or as an object (as in an object-oriented paradigm), or as an applet, or in a computer script language, or as another type of computer code or firmware. The software components and/or functionality may be located on a single device or distributed across multiple devices depending upon the situation at hand.

This written description sets forth the best mode of the invention and provides examples to describe the invention and to enable a person of ordinary skill in the art to make and use the invention. This written description does not limit the invention to the precise terms set forth. Thus, while the invention has been described in detail with reference to the examples set forth above, those of ordinary skill in the art may effect alterations, modifications and variations to the examples without departing from the scope of the invention.

As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise. Finally, as used in the description herein and throughout the claims that follow, the meanings of “and” and “or” include both the conjunctive and disjunctive and may be used interchangeably unless the context clearly dictates otherwise.

Ranges may be expressed herein as from “about” one particular value, and/or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.

These and other implementations are within the scope of the following claims.

Claims

1. A computer-implemented method, comprising: receiving, at a global server system, from each of a plurality of local network devices, network data specifying network communication activity at the local network device, wherein the plurality of local network devices collectively provide backbone communications facilities for multiple networks;aggregating, at the global server system, the network data from each of the local network devices;analyzing, at the global server system, the aggregated network data to identify network activities;generating, at the global server system, update data based on the analysis of the aggregated network data, the update data including instructions for the local network devices for processing network communications to or from the local network devices, wherein the instructions specify a prioritization schema for network communications based at least in part on reputations associated with network devices originating the network communications or associated with network devices to which the network communications are destined, the instructions operable to cause one or more local network devices to drop or delay particular network communications based at least in part on the prioritization schema; andtransmitting from the global server system the update data to the local network devices, the transmitting of the update data being one of a response to queries received from the local network devices or a push from the global server system to the local network devices.

2. The method of claim 1, wherein the instructions cause a local network device to prioritize a transmission of a network communication prior to transmitting the network communication to a next local network device in a communication path of the network communication.

3. The method of claim 1, wherein: a network communication has a communication path defining a route from a local network device originating the network communication through one or more intermediate local network device hops to a local network device to which the network communication is destined; andthe instructions cause a local network device in the communication path to change or remove at least one of the intermediate local network device hops in the communication path.

4. The method of claim 1, wherein the network communication activity for a local network device specifies network communication traffic patterns or malicious activity at the local network device.

5. The method of claim 1, wherein the plurality of local network devices comprise local network devices operating on multiple, different service provider networks.

6. The method of claim 1, wherein network data are generated by the local network devices in real time with respect to network communication activity occurring at the local network devices.

7. The method of claim 1, wherein aggregating the network data, analyzing the aggregated network data, generating the update data and transmitting the update data are performed in real time.

8. The method of claim 1, wherein the plurality of local network devices comprises routers and gateways.

9. The method of claim 1, wherein transmitting from the global server system the update data to the local network devices comprises: determining a frequency with which entities originate or receive network communications based on the network data; andtransmitting update data to local network devices processing network communications from entities originating or receiving network communications at a first frequency more frequently than to local network devices processing network communications from entities originating or receiving network communications at a second frequency lower than the first frequency.

10. The method of claim 1, further comprising: iteratively receiving, at the global server system, additional network data from the plurality of local network devices specifying additional network communication activity at the local network devices; andafter each iteration of the received additional network data:aggregating, at the global server system, the additional network data from each of the local network devices;analyzing, at the global server system, the aggregated additional network data to identify network activities;generating, at the global server system, additional update data based on the analysis of the aggregated additional network data, the additional update data including further instructions for the local network devices for processing network communications to or from the local network devices; andtransmitting from the global server system the additional update data to the local network devices, the transmitting of the additional update data being one of a response to queries received from the local network devices or a push from the global server system to the local network devices.

11. A system, comprising: a global server system operable to:receive from each of a plurality of local network devices, network data specifying network communication activity at the local network device, wherein the plurality of local network devices collectively provide backbone communications facilities for multiple networks;aggregate the network data from each of the local network devices;analyze the aggregated network data to identify network activities;generate update data based on the analysis of the aggregated network data, the update data including instructions for the local network devices for processing network communications to or from the local network devices, wherein the instructions specify a prioritization schema for network communications based at least in part on reputations associated with network devices originating the network communications or associated with network devices to which the network communications are destined, the instructions operable to cause one or more local network devices to drop or delay particular network communications based at least in part on the prioritization schema; andtransmit the update data to the local network devices, the transmitting of the update data being one of a response to queries received from the local network devices or a push from the global server system to the local network devices.

12. The system of claim 11, wherein each of the plurality of network devices is operable to transmit the network data to the global server system.

13. The system of claim 11, wherein the instructions cause a local network device to prioritize a transmission of a network communication prior to transmitting the network communication to a next local network device in a communication path of the network communication.

14. The system of claim 11, wherein the global server system is further operable to: iteratively receive additional network data from the plurality of local network devices specifying additional network communication activity at the local network devices; andafter each iteration of the received additional network data:aggregate the additional network data from each of the local network devices;analyze the aggregated additional network data to identify network activities;generate additional update data based on the analysis of the aggregated additional network data, the additional update data including further instructions for the local network devices for processing network communications to or from the local network devices; andtransmit the additional update data to the local network devices, the transmitting of the additional update data being one of a response to queries received from the local network devices or a push from the global server system to the local network devices.

15. The system of claim 11, further comprising: the plurality of local network devices, wherein each of the plurality of local network devices is operable to transmit network data to the global server system.

16. The system of claim 11, wherein: a network communication has a communication path defining a route from a local network device originating the network communication through one or more intermediate local network device hops to a local network device to which the network communication is destined; andthe instructions cause a local network device in the communication path to change or remove at least one of the intermediate local network device hops in the communication path.

17. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations, comprising: receiving, at a global server system, from each of a plurality of local network devices, network data specifying network communication activity at the local network device, wherein the plurality of local network devices collectively provide backbone communications facilities for multiple networks;aggregating, at the global server system, the network data from each of the local network devices;analyzing, at the global server system, the aggregated network data to identify network activities;generating, at the global server system, update data based on the analysis of the aggregated network data, the update data including instructions for the local network devices for processing network communications to or from the local network devices, wherein the instructions specify a prioritization schema for network communications based at least in part on reputations associated with network devices originating the network communications or associated with network devices to which the network communications are destined, the instructions operable to cause one or more local network devices to drop or delay particular network communications based at least in part on the prioritization schema; andtransmitting from the global server system the update data to the local network devices, the transmitting of the update data being one of a response to queries received from the local network devices or a push from the global server system to the local network devices.