Level of hierarchy in MST for traffic localization and load balancing

Information

  • Patent Grant
  • 10212074
  • Patent Number
    10,212,074
  • Date Filed
    Friday, July 28, 2017
    7 years ago
  • Date Issued
    Tuesday, February 19, 2019
    5 years ago
Abstract
In one embodiment, a multiple spanning tree (MST) region is defined in a network, where the MST region includes a plurality of network nodes interconnected by links. A MST cluster is defined within the MST region, where the MST cluster includes a plurality of network nodes selected from the plurality of network nodes of the MST region. A network node of the MST cluster generates one or more MST bridge protocol data units (BPDUs) that present the MST cluster as a single logical entity to network nodes of the MST region that are not included in the MST cluster, yet enables per-multiple spanning tree instance (per-MSTI) load balancing of traffic across inter-cluster links that connect network nodes included in the MST cluster and network nodes of the MST region that are not included in the MST cluster.
Description
TECHNICAL FIELD

The present disclosure relates generally to computer networks, and, more particularly, to multiple spanning tree protocols.


BACKGROUND

IEEE Std. 802.1Q-2005 multiple spanning tree protocol (MSTP) is widely used in computer networks. With IEEE Std. 802.1Q-2005 MSTP, a bridged network may be organized into multiple spanning tree (MST) regions. Frames originating at one network node inside of a MST region, and destined for another network node of the same MST region, are contained inside the MST region. From the point of view of nodes outside of the MST region, MST regions appear as a single logical entity (specifically, a single virtual bridge). As a result of this, load balancing of traffic is typically not available across the boundaries of MST regions. Even if there are multiple links that interconnect a MST is region with external nodes, only a single link generally may be utilized. This limitation of MST regions has proved problematic in certain network configurations. Sometimes it may be desirable to contain traffic among a particular set of network nodes and present them a single logical entity to external nodes, and also enable load balancing between the group of network nodes and external nodes. However, this is currently not possible using IEEE Std. 802.1Q-2005 MSTP.





BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments described herein may be better understood by referring to the accompanying drawings in which like reference numerals indicate identically or functionally similar elements, of which:



FIG. 1 is a schematic block diagram of an example computer network comprising a plurality of network nodes interconnected by links;



FIG. 2 is a schematic block diagram of an example computer network that includes a plurality of network nodes, some of which are members of a master-satellite switching configuration;



FIG. 3 is a schematic block diagram of the example computer network of FIG. 1 in which first and second MST clusters have been established with a MST region;



FIG. 4 is a schematic block diagram of an example network node, which is representative of the network nodes shown in FIGS. 1-3, and whose components may be used with the techniques for implementing MST clusters described herein;



FIG. 5 is a schematic block diagram of an intra-cluster MST BPDU that may be propagated between nodes within the same MST cluster to implement certain of the techniques described herein; and



FIG. 6 is a flow diagram of an example sequence of steps for implementing MST clusters within a network.





DESCRIPTION OF EXAMPLE EMBODIMENTS
Overview

According to embodiments of the disclosure, an additional level of hierarchy introduced to a multiple spanning tree (MST) environment. A MST region is defined in the network, where the MST region includes a plurality of network nodes interconnected by links. A MST cluster is defined within the MST region, where the MST cluster includes a plurality of network nodes selected from the plurality of network nodes of the MST region. A network node of the MST cluster generates one or more MST bridge protocol data units (BPDUs) that present the MST cluster as a single logical entity to network nodes of the MST region that are not included in the MST cluster, yet enable per-multiple spanning tree instance (per-MSTI) load balancing of traffic across inter-cluster links.


Example Embodiments

A computer network is a geographically distributed collection of nodes, such as bridges and switches, interconnected by communication links used to transport data between the nodes. Many types of computer networks are available, with the types ranging from local area networks (LANs) to wide area networks (WANs). The nodes typically communicate by exchanging discrete packets or messages of data according to pre-defined protocols. In this context, a protocol consists of a set of rules defining how the nodes interact with each other.


Most computer networks include redundant links so that a failure of any given link does not isolate any portion of the network. The existence of redundant links, however, may cause the formation of circuitous paths or “loops.” Loops are highly undesirable because data frames may traverse the loops indefinitely.


To avoid the formation of loops, most bridges and switches execute a spanning tree protocol which allows them to calculate an active network topology that is loop-free (i.e., a tree) and yet connects every node within the network (i.e., the tree is spanning). A basic spanning tree protocol, whose operation is illustrative of the class or protocols is IEEE Std. 802.1D-1998 spanning tree protocol (STP). In general, by executing IEEE Std. 802.1D-1998 STP, nodes elect a single node within the network to be the “Root.” For each LAN coupled to any node, exactly one port (the “Designated Port”) on one node (the “Designated Bridge”) is elected. The Designated Bridge is typically the one closest to the Root. All ports on the Root are Designated Ports.


Each non-Root also selects one port from among its non-Designated Ports (its “Root Port”) which gives the lowest cost path to the Root. The Root Ports and Designated Ports are selected for inclusion in the active topology and are placed in a forwarding state so that data frames may be forwarded to and from these ports. Ports not included within the active topology are placed in a blocking state. When a port is in the blocking state, data frames will not be forwarded to, or received from, the port. A network administrator may also exclude a port from the spanning tree by placing it in a disabled state.


To obtain the information necessary to run IEEE Std. 802.1D-1998 STP, nodes exchange special messages called bridge protocol data unit (BPDU) messages or simply BPDUs. BPDUs carry information, such as assumed root and lowest root path cost, used in computing the active topology.


In addition to IEEE Std. 802.1D-1998 STP, a variety of more advanced spanning tree protocols have been developed, that implement various refinements, optimizations, and extensions to the operation of IEEE Std. 802.1D-1998 STP. One such protocol is IEEE Std. 802.1Q-2005 multiple spanning tree protocol (MSTP). IEEE Std. 802.1Q-2005 MSTP organizes a bridged network into multiple spanning tree (MST) regions. Within each MST region, an internal spanning tree (IST) is established which provides connectivity to all nodes within the respective region, and to the ISTs established within other regions. The IST established within each MST region also provides connectivity to one common spanning tree (CST) established outside of the MST regions. The IST of a given MST region receives and sends BPDUs to the CST. Accordingly, all bridges of the bridged network are connected by a single Common and Internal Spanning Tree (CIST). From the point of view of outside devices, each MST region appears as a single logical entity, specifically, a single virtual bridge.


Within each MST region, IEEE Std. 802.1Q-2005 MSTP establishes a plurality of active topologies, each of which is called a multiple spanning tree instance (MSTI). Virtual LANs (VLANs) are assigned or mapped to the MSTIs. Because VLANs may be assigned to different MSTIs, frames associated with different VLANs can take different paths through a MST region. Each MSTI is basically a spanning tree that exists only inside the respective MST region, and the MSTIs do not interact outside of the MST region. Frames originating at one network node inside of a MST region and destined for another network node of the same MST region, follow a corresponding MSTI to reach their destination. Such internal traffic is contained inside the MST region, and is not allowed to, for example, leave the MST region, take external links, and then renter the MST region, as this may lead to the formation of loops.


IEEE Std. 802.1Q-2005 MSTP uses BPDUs to establish the ISTs and MSTIs, as well as to define the boundaries of the different MST regions. Unlike some other protocols, the bridges do not send separate BPDUs for each MSTI. Instead, every BPDU carries the information needed to compute the active topology for all of the MSTIs defined within the respective MST region.


As discussed above, from the point of view of external node, each MST region appears as a single logical entity (specifically, a single virtual bridge). As a result of this, load balancing of traffic is typically not available across boundaries of MST regions. Even if there are multiple links that interconnect a MST region with external nodes, only a single link generally may be utilized. This limitation of MST regions has proved problematic in certain network configurations. Sometimes it may be desirable to contain traffic among a particular set of network nodes and present them a single logical entity to external nodes, and also enable load balancing between the group of network nodes and external nodes. However, this is currently not possible using IEEE Std. 802.1Q-2005 MSTP.



FIG. 1 is a schematic block diagram of an example computer network 100 comprising a plurality of network nodes 110-155 (e.g., bridges or switches) interconnected by links, which illustrate some of the above discussed issues. The nodes 110-155 are interconnected by a plurality of links, referred to herein by reference to the names of the nodes at the opposing ends of the links (e.g., link sw0-sw1 being the link connecting network node sw0110 to network node sw1115, link sw0-sw2 being the link connecting network node sw0100 to network node sw2120, etc.). Suppose it is desired for nodes sw4130, sw5135 and sw6140 to present themselves to external nodes as a single logical entity, and to contain internal traffic among themselves. Further, suppose it is also desired that nodes sw7145, sw8150 and sw9155 present themselves as a single logical entity and contain internal traffic among themselves, and, additionally, that nodes sw2120 and sw3125 present themselves as a single logical entity and contain internal traffic among themselves. One could establish three MST regions: Region 1160 encompassing nodes sw2120 and sw3125; Region 2170 encompassing nodes sw4130, sw5135, and sw6140; and Region 3180 encompassing nodes sw7145, sw8150, and sw9155. The establishment of such MST regions may effectively contain internal traffic. However, load balancing may not be enabled across the boundaries of the three MST regions, as each of the MST regions would appear as single virtual bridge on the CST. For example, all traffic between Region 1160 and Region 2170 would be allowed only to travel on a single link (e.g., link sw2-sw4) selected for inclusion in the CST, while other redundant links (e.g., link sw2-sw6) would not be used. Accordingly, available network bandwidth may not be efficiently utilized.


Network configurations similar to that shown in FIG. 1, with their corresponding issues, may arise in a variety of settings. For example, the setting may involve a master-satellite switching configuration. FIG. 2 is a schematic block diagram of an example computer network 200 that includes a plurality of network nodes 210-280, some of which (i.e. nodes sw2220, sw3230 and sw4240) are members of a master-satellite switching configuration. Master node sw2220 is connected to satellite nodes sw3230 and sw4240. To present Master node sw2220 and satellite nodes sw3230 and sw4240 as a single logical entity, and contain internal traffic among them, one may place such nodes within a MST region, i.e., Region 290. Use of such a MST region, for example, may prevent traffic received from host 1250 and destined for host 2280, from being directed along a path from satellite node sw3230, to node sw5260, to node sw6270, and then back to node sw4240, to reach host 2280, rather than utilize the sw3-sw4 link. However, use of such a MST region may also prevent load balancing across links to external nodes, for example, across links sw3-sw5 and sw4-sw6, due to the MST region appearing to external nodes as a single virtual bridge.


Similarly, the above discussed issues may arise in the context of a multi-chassis virtual switching system (VSS) using MST in conjunction with virtual switch links (VSLs), or in the context of a data center, using MST among backbone or “top-of-the rack” devices. In these example scenarios as well, it may be desirable to both contain traffic among a particular set of nodes, and cause the group of nodes to externally present as a single logical entity, while also enabling load balancing between the group of nodes and external nodes. Accordingly, there is a need for a technique that can achieve these formerly mutually exclusive goals within a MST environment.


MST Clusters


According to embodiments of the present disclosure, an additional level of hierarchy, referred to herein as a “MST cluster”, is defined within a MST environment. A MST cluster includes a set of nodes selected from the nodes of a MST region. The group of nodes selected for inclusion in a MST cluster may be a group of nodes for which traffic containment is desired. The cluster is presented within BPDUs as a single logical entity. Internal traffic that is sourced and destined for network nodes of the MST cluster is contained within the MST cluster, and passed over “intra-cluster links”, i.e. links that couple nodes located within the MST cluster.


Unlike a MST region, load balancing is enabled across the boundary of a MST cluster. Since nodes within the MST cluster and the rest of the MST region may have consistent MST configurations (i.e., share the same MST region name, revision number and MST configuration table), per-MSTI load balancing may occur across the boundary of the MST cluster. In this manner, external traffic that is sourced from, or destined for, nodes of the MST region located external to MST cluster may be load balanced across “inter-cluster links”, i.e. links that couple nodes located within the MST cluster with nodes located external to the MST cluster that are within the MST region.



FIG. 3 is a schematic block diagram of the example computer network of FIG. 1 in which first and second MST clusters 310, 320 have been established within a MST region 330. Network nodes sw2 to sw9120-155 are arranged into a MST region 330. Within the MST region 330, a plurality of MSTIs may be established. Nodes sw4130, sw5135 and sw6140, in this example, are arranged into a first MST cluster 310, with links sw4-sw6, sw4-sw5 and sw5-sw6 serving as intra-cluster links. Links sw0-sw4, sw2-sw4, sw2-sw6, sw4-sw7 and sw6-sw8 serve as inter-cluster links, and couple the first MST cluster 310 to external nodes. The first MST cluster 310 is presented to external nodes as a single logical entity, and internal traffic sourced and destined for network nodes within the first MST cluster 310 is contained therein.


Similarly, nodes sw7145, sw8150 and sw9155, in this example, are arranged into a second MST cluster 320, with links sw7-sw8, sw7-sw9 and sw8-sw9 serving as intra-cluster links. Links sw1-sw7, sw3-sw7, sw3-sw8, sw4-sw7 and sw6-sw8 serve as inter-cluster links, and couple the second MST cluster 320 to external nodes. Like the first MST cluster 310, the second MST cluster 330 is presented to external nodes as a single logical entity, and internal traffic sourced and destined for network nodes within the second MST cluster 320 is contained therein.


Since nodes within the first and second MST clusters 310, 320, and the rest of the MST region 330, have consistent MST configurations (i.e., share the same MST region name, revision number and MST configuration table) per-MSTI load balancing may occur across the boundaries of the MST clusters. That is, differing MSTIs of the MST region 330 may utilize different ones of the inter-cluster links, and frames associated with different MSTIs, may travel over different ones of the inter-cluster links. For example, per-MSTI load balancing may permit load balancing between inter-cluster links sw2-sw4 and sw2-sw6, through association of some VLANs with an MSTI that utilizes inter-cluster link sw2-sw4 and blocks inter-cluster link sw2-sw6, and other VLANs with an MSTI that utilizes inter-cluster link sw2-sw6 and blocks link inter-cluster sw2-sw4. In this manner, load balancing may be permitted across cluster boundaries, while MST clusters still present an appearance to external devices of a single logical entity.



FIG. 4 is a schematic block diagram of an example network node (e.g., a bridge or switch) 400, which is representative of the network nodes 110-155 and 210-280 shown in FIGS. 1-3, and whose components may be used with the techniques for implementing MST clusters described herein. The example network node 400 includes a plurality of network interfaces 410, processor(s) 420, and a memory 430 interconnected by a system bus 450. The network interfaces 410 contain the mechanical, electrical, and signaling circuitry for communicating over links. The memory 430 includes a plurality of storage locations for storing software and data structures, including software and data structures. The processor(s) 420 include logic configured to execute the software and manipulate data from the data structures. An operating system 432, portions of which are resident in memory 430 and executed by the processor(s) 420, functionally organizes the network node 400. A MSTP process 434 may interact with the operating system 432 to implement, for example, aspects of IEEE Std. 802.1Q-2005 MSTP, as well as the novel techniques for adding an additional level of hierarchy (i.e. MST clusters) to MST regions discussed herein.



FIG. 5 is a schematic block diagram of an intra-cluster MST BPDU 500 that may be propagated between nodes within the same MST cluster to implement certain of the techniques described herein. An intra-cluster MST BPDU 500 is intended to be propagated only within a MST cluster, and should not knowingly be forwarded across a cluster boundary. However, as discussed below, certain properties of the intra-cluster MST BPDU may permit interoperability with nodes running legacy MST implementations, which may not be aware of MST clusters or their respective boundaries.


A number of the fields within the intra-cluster MST BPDU 500 may be identical, or otherwise similar, to those within an IEEE Std. 802.1Q-2005 MSTP BPDU, and accordingly will be familiar to those skilled in the art. For example, a Protocol Identifier field 502, a Protocol Version Identifier field 504, a BPDU Type field 506, a CIST Flags field 508, a CIST Root Identifier field 510, a CIST External Path Cost field 512, a CIST Regional Root Identifier field 514, a CIST Port Identifier filed 516, a Message Age field 518, a Max Age field 520, a Hello Time field 522, a Forward Delay filed 524, a Version 1 Length filed 526, a Version 3 Length field 528, a MST Configuration Identifier field 530, a CIST Internal Root Path Cost field 532, a CIST Remaining Hops field 536, and one or more MSTI configuration Messages 532, may be identical, or otherwise similar, to those within a IEEE Std. 802.1Q-2005 MSTP BPDU. In the interest of brevity, the requirements and functions of such fields are not described in detail herein, and only fields that have been redefined, or newly introduced, as compared to an IEEE 802.1Q-2005 MSTP BPDU, are described in detail below.


Following a CIST Internal Root Path Cost field 532, an IEEE 802.1Q-2005 MSTP BPDU typically would include a CIST Bridge Identifier field to indicate the bridge identifier of the node that transmitted the BPDU. In order to present an MST cluster as a single logical entity, such field may be redefined to operate as a CIST Cluster Root Identifier field 534, to store the bridge identifier of a node of the cluster (i.e. a CIST cluster root node) that has the lowest cost path to the CIST regional root node. For example, referring to FIG. 3, node sw4130 may serve as the CIST cluster root node. In which case, nodes sw4130, sw5135 and sw6140 would each generate intra-cluster MST BPDUs 500 that include the bridge identifier of node sw4130 in their CIST Cluster Root Identifier field 534.


Typically, an IEEE 802.1Q-2005 MSTP BPDU would terminate after one or more MSTI Configuration Messages 538. To implement the techniques described herein, a new CIST Cluster Configuration Message 570 may be inserted after the one or more MSTI Configuration Messages 538. Since the CIST Cluster Configuration Message 570 is located after the MSTI Configuration Messages 538, it will be beyond the length typically indicated in the Version 3 Length field 528. This may advantageously permit interoperation with nodes that are running certain legacy MSTP implementations, which would typically ignore information beyond the length indicated in the Version 3 length field.


The CIST Cluster Configuration Message 570 may include a CIST Intra Cluster Root Path Cost field 544, a CIST Cluster Bridge Identifier filed 546, and a CIST Cluster Remaining Hops field 548.


The CIST Intra Cluster Root Path Cost field 544 stores a cost within the cluster to reach the CIST cluster root node. For example, referring to FIG. 3, if node sw6140 generated the Intra Cluster MST BPDU 500, and node sw4130 is the CIST cluster root node, the CIST Intra Cluster Root Path Cost field 544 would store a cost between node sw4130 and node sw6140.


The CIST Cluster Bridge Identifier field 546 stores the bridge identifier of the node that generated the Intra Cluster MST BPDU 500. For example, referring to FIG. 3, if node sw6140 generated the Intra Cluster MST BPDU 500, its bridge identifier would be included in the CIST Cluster Bridge Identifier field 546.


The CIST Cluster Remaining Hops field 548 stores a number of hops within the MST cluster which limits the propagation and longevity of received spanning tree information for the CIST.


Following the CIST Cluster Configuration Message 570, one or more MSTI Cluster Configuration Messages 580, corresponding to the cluster described in the CIST Cluster Configuration Message 570, may be included to provide information about respective MSTIs supported by the MST cluster. As with the CIST Cluster Configuration Message 570, the MSTI Cluster Configuration Messages 580 are beyond the length typically indicated in the Version 3 Length field 528 and thus will typically be ignored by legacy MSTP implementations.


Each such message 570 may be similar to a MSTI Configuration Message 538, but instead contain MST cluster specific information. A MSTI Cluster Configuration Message may include a MSTI Cluster Root Identifier field 550, a MSTI Intra Cluster Root Path Cost field 552, and a MSTI Cluster Remaining Hops field 554.


The MSTI Cluster Root Identifier field 550 stores the bridge identifier of a node of the cluster serving as a root node of the cluster for the respective MSTI. For example, referring to FIG. 3, node sw6140 may serve as the root node of the cluster for one MSTI, and node sw4140 may serve as the root node of the cluster for another MSTI. In each such case, the respective node's bridge identifier may be stored in the MSTI Cluster Root Identifier field 550.


The MSTI Intra Cluster Root Path Cost field 552 stores a cost within the cluster to reach the cluster root node for the respective MSTI. For example, referring to FIG. 3, if node sw4130 generated the Intra Cluster MST BPDU 500, and node sw4130 is the cluster root node for the respective MSTI, the MSTI Intra Cluster Root Path Cost field 552 would store a cost between node sw6140 and node sw4130.


The MSTI Cluster Remaining Hops field 554 stores a number of hops within the MST cluster which limits the propagation and longevity of received spanning tree information for the respective MSTI.


The above discussed intra-cluster MST BPDU 500 may be limited in size due to Ethernet frame size limits, such that, a intra-cluster MST BPDU 500 may consume no more than about 1500 bytes. As such, a single intra-cluster MST BPDU 500 may be limited to contain information for at most 32 different MSTIs, in addition to the CIST. If more MSTIs are configured, multiple intra-cluster MST BPDUs may be utilized to describe the network configuration.


The information contained in BPDUs, and that is used to select root nodes and shortest paths by a spanning tree protocol, is typically referred to a “spanning tree priority vector.” Spanning tree priority vectors provide a basis for a concise specification of a protocol's computation of the active topology. In IEEE 802.1Q-2005 MSTP, the CIST priority vector consists of a) root identifier (RootID) (i.e., the bridge identifier of the CIST root node), b) external root path cost (ExtRootPathCost) (i.e., the path cost between MST Regions from the transmitting Bridge to the CIST Root), c) regional root identifier (RRootID) (i.e., the bridge identifier of the CIST regional root node), d) internal root path cost (IntRootPathCost) (i.e., the path cost to the CIST regional root node), e) designated bridge identifier (DesignatedBridgeID) (i.e., the bridge identifier for the transmitting node for the CIST), f) Designated Port Identifier (DesignatedPortID) (i.e., the Port Identifier for the transmitting port for the CIST) and g) the BPDU-receiving port ID (RcvPortID) which is not conveyed in BPDUs, but used as a tie-breaker between otherwise equal priority vectors within a receiving node. To implement the additional level of hierarchy as described herein, the IEEE 802.1Q-2005 MSTP CIST priority vector may be modified. A cluster root identifier (CRootID) (i.e., the bridge identifier of the CIST cluster root node) and an intra-cluster root path cost (IntraRootPathCost) (i.e., the path cost to the CIST cluster root node) may be added to the priority vector. Accordingly, a complete CIST priority vector that supports MST clusters may be defined as: root identifier (RootID), b) external root path cost (ExtRootPathCost), c) regional root identifier (RRootID), d) internal root path cost (IntRootPathCost), e) cluster root identifier (CRootID), f) intra-cluster root path cost (IntraRootPathCost), g) designated bridge identifier (DesignatedBridgeID), h) Designated Port Identifier (DesignatedPortID) and i) the BPDU-receiving port ID (RcvPortID). It should be understood that external root path cost (ExtRootPathCost) is updated across MST region boundaries, internal root path cost (IntRootPathCost) is updated across MST cluster boundaries, but not within a MST cluster, and intra-cluster root path cost (IntraRootPathCost) is updated within a MST cluster.


Similarly, in IEEE 802.1Q-2005 MSTP, the MST priority vector consists of a) regional root identifier (RRootID) (i.e., the bridge identifier of the MSTI regional root for this particular MSTI in this MST region), b) internal root path cost (IntRootPathCost) (i.e., the path cost to the MSTI Regional Root for this particular MSTI in this MST region), c) designated bridge identifier (DesignatedBridgeID) (i.e., the bridge identifier for the transmitting node for this MSTI), d) Designated Port Identifier (DesignatedPortID) (i.e., the port identifier for the transmitting port for this MSTI) and e) receiving port identifier (RcvPortID), which is not conveyed in BPDUs, but used as tie-breaker between otherwise equal priority vectors within a receiving node. To implement the additional level of hierarchy as described herein, the IEEE 802.1Q-2005 MST priority vector may be modified. A cluster root identifier (CRootID) (i.e., the bridge identifier of the cluster root node for this MSTI) and an intra-cluster root path cost (IntraRootPathCost) (i.e., the path cost to the cluster root node for this MSTI) are added to the priority vector. Accordingly, a complete MST priority vector that supports MST clusters may be defined as: a) regional root identifier (RRootID), b) internal root path cost (IntRootPathCost), c) cluster root identifier (CRootID), d) intra-cluster root path cost (IntraRootPathCost), e) designated bridge identifier (DesignatedBridgeID), f) Designated Port Identifier (Designated-PortID) and f) receiving port identifier (RcvPortID).



FIG. 6 is a flow diagram of an example sequence of steps 600 for implementing MST clusters within a network. Such sequence of steps may be implemented, at least in part, by MSTP process 434, working in conjunction with the operating system 432, when their instructions are executed on processor(s) 220. At step 610 a multiple spanning tree (MST) region is defined that includes a plurality of network nodes interconnected by links. At step 620, a MST cluster is defined within the MST region, where the MST cluster includes a plurality of network nodes selected from the plurality of network nodes of the MST region. At step 630, MST BPDUs are generated that present the MST cluster as a single logical entity to network nodes of the MST region that are not included in the MST cluster, yet enables per-MSTI load balancing of traffic across inter-cluster links that connect network nodes included in the MST cluster and network nodes of the MST region that are not included in the MST cluster. Specifically, the BPDUs may include a CIST Cluster Root Identifier field 534, that stores the bridge identifier of a node of the cluster that has the lowest cost path to CIST regional root node; may include one or more MSTI Configuration Messages 570, that indicate a CIST intra cluster root path cost, a CIST cluster bridge identifier, and a CIST cluster remaining hops count; and may include one or more corresponding MSTI Cluster Configuration Messages 580, that indicate a MSTI cluster root identifier, MSTI intra cluster root path cost, and a MSTI cluster remaining hops count. Further, by specification of consistent MST configurations (i.e., the same MST region name, revision number and MST configuration table), per-MSTI load balancing may be provided across inter-cluster links.


In summary, the present disclosure provides an additional level of hierarchy, (MST clusters) within a MST environment. A MST cluster includes a set of nodes selected from the nodes of a MST region. A MST cluster is presented within BPDUs as a single logical entity. Internal traffic that is sourced and destined for network nodes of the MST cluster is contained within the MST cluster, and passed over intra-cluster links. Since nodes within the MST cluster and the rest of the MST region may have consistent MST configurations, per-MSTI load balancing may occur across the boundary of the MST cluster. In this manner, external traffic that is sourced from, or destined for, nodes of the MST region located external to MST cluster may be load balanced across inter-cluster links.


It should be understood that various adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, at least some of the above-described embodiments may be implemented in software, in hardware, or a combination thereof. A software implementation may include computer-executable instructions stored in a non-transitory computer-readable medium, such as a volatile or persistent memory, a hard-disk, a compact disk (CD), or other tangible medium. A hardware implementation may include configured processors, logic circuits, application specific integrated circuits, and/or other types of hardware components. Further, a combined software/hardware implementation may include both computer-executable instructions stored in a non-transitory computer-readable medium, as well as one or more hardware components, for example, processors, memories, etc. Accordingly, it should be understood that the above descriptions are meant to be taken only by way of example. It is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the embodiments herein.

Claims
  • 1. A method comprising: defining a multiple spanning tree (MST) region in a network, the MST region comprising a plurality of network nodes interconnected by links;defining a plurality of non-overlapping MST clusters within the MST region, each of the MST clusters being a subset of at least two of the plurality of network nodes within the MST region, each of the MST clusters presenting itself as a single logical entity and permitting traffic on a plurality of inter-cluster links associated with the subset of the plurality of network nodes; andenabling load balancing of traffic across the plurality of inter-cluster links, each link of the plurality of inter-cluster links connecting a respective network node in one of the MST clusters to a respective network node of the MST region that is outside of the one of the MST clusters.
  • 2. The method of claim 1, further comprising: including a bridge identifier of a network node of the one of the MST clusters that has a lowest cost path to a Common and Internal Spanning Tree (CIST) regional root node in one or more MST bridge protocol data units (BPDUs).
  • 3. The method of claim 2, wherein the bridge identifier of the network node of the one of the MST clusters that has the lowest cost path to the CIST regional root node is stored in a CIST Cluster Root Identifier field of the one or more MST BPDUs.
  • 4. The method of claim 1, further comprising: including a cost within the one of the MST clusters to reach a Common and Internal Spanning Tree (CIST) cluster root node in one or more MST bridge protocol data units (BPDUs).
  • 5. The method of claim 4, wherein the cost within the one of the MST clusters to reach the CIST cluster root node is stored in a CIST Intra Cluster Root Path Cost field of a CIST Cluster Configuration Message in the one or more MST BPDUs.
  • 6. The method of claim 1, further comprising: generating, by one or more processors, one or more MST bridge protocol data units (BPDUs) that present the one of the MST clusters as a single logical entity to network nodes of the MST region that are not included in the MST cluster; andincluding a respective bridge identifier of a particular network node that generated the MST BPDUs in the one or more MST BPDUs.
  • 7. The method of claim 6, wherein the particular network node that generated the MST BPDUs is stored in a MSTI Cluster Root Identifier field of a MSTI Cluster Configuration Message in the one or more MST BPDUs.
  • 8. The method of claim 1, further comprising: including a number of hops within the one of the MST clusters for a Common and Internal Spanning Tree (CIST) in one or more MST bridge protocol data units (BPDUs).
  • 9. The method of claim 8, wherein the number of hops within the one of the MST clusters for the CIST is stored in a CIST Cluster Remaining Hops field of a CIST Cluster Configuration Message in the one or more MST BPDUs.
  • 10. The method of claim 1, further comprising: including a respective bridge identifier of a particular node of the one of the MST clusters serving as a root node of the one of the MST clusters for a respective MSTI in one or more MST bridge protocol data units (BPDUs).
  • 11. The method of claim 10, wherein a cost within the one of the MST clusters to reach the root node associated with the respective MSTI cluster is stored in a MSTI Cluster Root Identifier field of a MSTI Cluster Configuration Message in the one or more MST BPDUs.
  • 12. The method of claim 1, further comprising: including a cost within the one of the MST clusters to reach a cluster root node for a respective MSTI in one or more MST bridge protocol data units (BPDUs).
  • 13. The method of claim 12, wherein the cost within the one of the MST clusters to reach the cluster root node for the respective MSTI is stored in a MSTI Intra Cluster Root Path Cost field of a MSTI Cluster Configuration Message in the one or more MST BPDUs.
  • 14. The method of claim 1, further comprising: including a number of hops within the one of the MST clusters for a respective MSTI in one or more MST bridge protocol data units (BPDUs).
  • 15. The method of claim 14, wherein the number of hops within the one of the MST clusters for the respective MSTI is stored in a MSTI Cluster Remaining Hops field of a MSTI Cluster Configuration Message in the one or more MST BPDUs.
  • 16. The method of claim 14, wherein the one or more MST BPDUs are intra cluster MST BPDUs.
  • 17. An apparatus comprising: a network interface configured to couple the apparatus to one or more network nodes of a network;a processor coupled to the network interface and configured to execute one or more processes; anda memory configured to store a multiple spanning tree protocol (MSTP) process executable by the processor, the MSTP process when executed operable to: determine a multiple spanning tree (MST) region has been defined in the network, wherein the MST region comprises a plurality of network nodes interconnected by links;determine a plurality of non-overlapping MST clusters within the MST region, each of the MST clusters being a subset of at least two of the plurality of network nodes within the MST region, wherein each of the MST clusters presenting itself as a single logical entity and permits traffic on a plurality of inter-cluster links associated with the subset of the plurality of network nodes; andenable load balancing of traffic across the plurality of inter-cluster links, each link of the plurality of inter-cluster links connecting a respective network node in one of the MST clusters to a respective network node of the MST region that is outside of the one of the MST clusters.
  • 18. The apparatus of claim 17, wherein a bridge identifier is stored in a Common and Internal Spanning Tree (CIST) Cluster Root Identifier field of one or more MST bridge protocol data units (BPDUs), wherein a CIST Cluster Configuration Message in the one or more MST BPDUs comprises a CIST Intra Cluster Root Path Cost field containing a value representing a cost within the one of the MST clusters to reach a Common and Internal Spanning Tree (CIST) cluster root node in the one or more MST BPDUs.
  • 19. The apparatus of claim 17, wherein one or more MST bridge protocol data units (BPDUs) comprise a number of hops within the one of the MST clusters for a Common and Internal Spanning Tree (CIST) regional root node.
  • 20. A non-transitory computer-readable storage medium comprising: computer-executable instructions stored thereon which, when executed by one or more processors, cause the one or more processors to: define a multiple spanning tree (MST) region in a network, the MST region comprising a plurality of network nodes interconnected by links;define a plurality of non-overlapping MST clusters within the MST region, each of the MST clusters being a subset of at least two of the plurality of network nodes within the MST region, each of the MST clusters presenting itself as a single logical entity and permitting traffic on a plurality of inter-cluster links associated with a subset of the plurality of network nodes; andenable load balancing of traffic across the plurality of inter-cluster links, each link of the plurality of inter-cluster links connecting a respective network node in one of the MST clusters to a respective network node of the MST region that is outside of the one o the MST clusters.
RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No. 13/168,571 filed on Jun. 24, 2011, the contents of which is incorporated by reference in its entirety.

US Referenced Citations (448)
Number Name Date Kind
5812773 Norin Sep 1998 A
5889896 Meshinsky et al. Mar 1999 A
6049797 Guha Apr 2000 A
6108782 Fletcher et al. Aug 2000 A
6178453 Mattaway et al. Jan 2001 B1
6298153 Oishi Oct 2001 B1
6343290 Cossins et al. Jan 2002 B1
6643260 Kloth et al. Nov 2003 B1
6683873 Kwok et al. Jan 2004 B1
6721804 Rubin et al. Apr 2004 B1
6733449 Krishnamurthy et al. May 2004 B1
6735631 Oehrke et al. May 2004 B1
6813250 Fine et al. Nov 2004 B1
6996615 McGuire Feb 2006 B1
7054930 Cheriton May 2006 B1
7058706 Lyer et al. Jun 2006 B1
7062571 Dale et al. Jun 2006 B1
7111177 Chauvel et al. Sep 2006 B1
7212490 Kao et al. May 2007 B1
7277948 Igarashi et al. Oct 2007 B2
7292581 Finn Nov 2007 B2
7313667 Pullela et al. Dec 2007 B1
7379846 Williams et al. May 2008 B1
7480258 Shuen et al. Jan 2009 B1
7480672 Hahn et al. Jan 2009 B2
7496043 Leong et al. Feb 2009 B1
7536476 Alleyne May 2009 B1
7567504 Darling et al. Jul 2009 B2
7583665 Duncan et al. Sep 2009 B1
7606147 Luft et al. Oct 2009 B2
7644437 Volpano Jan 2010 B2
7647594 Togawa Jan 2010 B2
7706364 Smith et al. Apr 2010 B2
7729296 Choudhary Jun 2010 B1
7760668 Zinjuvadia Jul 2010 B1
7773510 Back et al. Aug 2010 B2
7808897 Mehta et al. Oct 2010 B1
7821972 Finn et al. Oct 2010 B1
7835306 Tallet Nov 2010 B2
7881957 Cohen et al. Feb 2011 B1
7889681 Finn Feb 2011 B2
7917647 Cooper et al. Mar 2011 B2
8010598 Tanimoto Aug 2011 B2
8028071 Mahalingam et al. Sep 2011 B1
8041714 Aymeloglu et al. Oct 2011 B2
8121117 Amdahl et al. Feb 2012 B1
8171415 Appleyard et al. May 2012 B2
8234377 Cohn Jul 2012 B2
8244559 Horvitz et al. Aug 2012 B2
8250215 Stienhans et al. Aug 2012 B2
8280880 Aymeloglu et al. Oct 2012 B1
8284664 Aybay et al. Oct 2012 B1
8301746 Head et al. Oct 2012 B2
8345692 Smith Jan 2013 B2
8406141 Couturier et al. Mar 2013 B1
8407413 Yucel et al. Mar 2013 B1
8448171 Donnellan et al. May 2013 B2
8477610 Zuo et al. Jul 2013 B2
8495356 Ashok et al. Jul 2013 B2
8495725 Ahn Jul 2013 B2
8510469 Portolani Aug 2013 B2
8514868 Hill Aug 2013 B2
8532108 Li et al. Sep 2013 B2
8533687 Greifeneder et al. Sep 2013 B1
8547974 Guruswamy et al. Oct 2013 B1
8560639 Murphy et al. Oct 2013 B2
8560663 Baucke et al. Oct 2013 B2
8589543 Dutta et al. Nov 2013 B2
8590050 Nagpal et al. Nov 2013 B2
8611356 Yu et al. Dec 2013 B2
8612625 Andries et al. Dec 2013 B2
8630291 Shaffer et al. Jan 2014 B2
8639787 Lagergren et al. Jan 2014 B2
8656024 Krishnan et al. Feb 2014 B2
8660129 Brendel et al. Feb 2014 B1
8719804 Jain May 2014 B2
8775576 Hebert et al. Jul 2014 B2
8797867 Chen et al. Aug 2014 B1
8805951 Faibish et al. Aug 2014 B1
8850002 Dickinson et al. Sep 2014 B1
8850182 Fritz et al. Sep 2014 B1
8856339 Mestery et al. Oct 2014 B2
8909928 Ahmad et al. Dec 2014 B2
8918510 Gmach et al. Dec 2014 B2
8924720 Raghuram et al. Dec 2014 B2
8930747 Levijarvi et al. Jan 2015 B2
8938775 Roth et al. Jan 2015 B1
8959526 Kansal et al. Feb 2015 B2
8977754 Curry, Jr. et al. Mar 2015 B2
9009697 Breiter et al. Apr 2015 B2
9015324 Jackson Apr 2015 B2
9043439 Bicket et al. May 2015 B2
9049115 Rajendran et al. Jun 2015 B2
9063789 Beaty et al. Jun 2015 B2
9065727 Liu et al. Jun 2015 B1
9075649 Bushman et al. Jul 2015 B1
9130846 Szabo et al. Sep 2015 B1
9164795 Vincent Oct 2015 B1
9167050 Durazzo et al. Oct 2015 B2
9201701 Boldyrev et al. Dec 2015 B2
9201704 Chang et al. Dec 2015 B2
9203784 Chang et al. Dec 2015 B2
9223634 Chang et al. Dec 2015 B2
9244776 Koza et al. Jan 2016 B2
9251114 Ancin et al. Feb 2016 B1
9264478 Hon et al. Feb 2016 B2
9294408 Dickinson et al. Mar 2016 B1
9313048 Chang et al. Apr 2016 B2
9361192 Smith et al. Jun 2016 B2
9379982 Krishna et al. Jun 2016 B1
9380075 He et al. Jun 2016 B2
9432245 Sorenson, III et al. Aug 2016 B1
9432294 Sharma et al. Aug 2016 B1
9444744 Sharma et al. Sep 2016 B1
9473365 Melander et al. Oct 2016 B2
9503530 Niedzielski Nov 2016 B1
9558078 Farlee et al. Jan 2017 B2
9571570 Mutnuru Feb 2017 B1
9613078 Vermeulen et al. Apr 2017 B2
9628471 Sundaram et al. Apr 2017 B1
9658876 Chang et al. May 2017 B2
9692802 Bicket et al. Jun 2017 B2
9736065 Tsai et al. Aug 2017 B2
9755858 Bagepalli et al. Sep 2017 B2
20010055303 Horton et al. Dec 2001 A1
20020073337 Ioele et al. Jun 2002 A1
20020143928 Maltz et al. Oct 2002 A1
20020166117 Abrams et al. Nov 2002 A1
20020174216 Shorey et al. Nov 2002 A1
20030018591 Komisky Jan 2003 A1
20030056001 Mate et al. Mar 2003 A1
20030228585 Inoko et al. Dec 2003 A1
20040004941 Malan et al. Jan 2004 A1
20040034702 He Feb 2004 A1
20040081171 Finn Apr 2004 A1
20040088542 Daude et al. May 2004 A1
20040095237 Chen et al. May 2004 A1
20040131059 Ayyakad et al. Jul 2004 A1
20040197079 Latvala et al. Oct 2004 A1
20040264481 Darling et al. Dec 2004 A1
20050060418 Sorokopud Mar 2005 A1
20050125424 Herriott et al. Jun 2005 A1
20050259597 Benedetto et al. Nov 2005 A1
20060007939 Elangovan Jan 2006 A1
20060062187 Rune Mar 2006 A1
20060104286 Cheriton May 2006 A1
20060126665 Ward et al. Jun 2006 A1
20060146825 Hofstaedter et al. Jul 2006 A1
20060155875 Cheriton Jul 2006 A1
20060168338 Bruegl et al. Jul 2006 A1
20060233106 Achlioptas et al. Oct 2006 A1
20070064628 Tallet Mar 2007 A1
20070140147 Touve Jun 2007 A1
20070174663 Crawford et al. Jul 2007 A1
20070223487 Kajekar et al. Sep 2007 A1
20070242830 Conrado et al. Oct 2007 A1
20070258390 Khan et al. Nov 2007 A1
20070263640 Finn Nov 2007 A1
20080005293 Bhargava et al. Jan 2008 A1
20080080524 Tsushima et al. Apr 2008 A1
20080084880 Dharwadkar Apr 2008 A1
20080165704 Marchetti Jul 2008 A1
20080165778 Ertemalp Jul 2008 A1
20080198752 Fan et al. Aug 2008 A1
20080198858 Townsley et al. Aug 2008 A1
20080201711 Amir Husain Aug 2008 A1
20080235755 Blaisdell et al. Sep 2008 A1
20090006527 Gingell, Jr. et al. Jan 2009 A1
20090019367 Cavagnari et al. Jan 2009 A1
20090031312 Mausolf et al. Jan 2009 A1
20090059800 Mohan Mar 2009 A1
20090083183 Rao et al. Mar 2009 A1
20090138763 Arnold May 2009 A1
20090177775 Radia et al. Jul 2009 A1
20090178058 Stillwell, III et al. Jul 2009 A1
20090182874 Morford et al. Jul 2009 A1
20090265468 Annambhotla et al. Oct 2009 A1
20090265753 Anderson et al. Oct 2009 A1
20090293056 Ferris Nov 2009 A1
20090300608 Ferris et al. Dec 2009 A1
20090313562 Appleyard et al. Dec 2009 A1
20090323706 Germain et al. Dec 2009 A1
20090328031 Pouyadou et al. Dec 2009 A1
20100036903 Ahmad et al. Feb 2010 A1
20100040070 Suh Feb 2010 A1
20100042720 Stienhans et al. Feb 2010 A1
20100061250 Nugent Mar 2010 A1
20100115341 Baker et al. May 2010 A1
20100118740 Takacs May 2010 A1
20100131765 Bromley et al. May 2010 A1
20100149966 Achlioptas et al. Jun 2010 A1
20100191783 Mason et al. Jul 2010 A1
20100192157 Jackson et al. Jul 2010 A1
20100205601 Abbas et al. Aug 2010 A1
20100211782 Auradkar et al. Aug 2010 A1
20100293270 Augenstein et al. Nov 2010 A1
20100318609 Lahiri et al. Dec 2010 A1
20100325199 Park et al. Dec 2010 A1
20100325441 Laurie et al. Dec 2010 A1
20100333116 Prahlad et al. Dec 2010 A1
20110016214 Jackson Jan 2011 A1
20110035754 Srinivasan Feb 2011 A1
20110055396 Dehaan Mar 2011 A1
20110055398 Dehaan et al. Mar 2011 A1
20110055470 Portolani Mar 2011 A1
20110072489 Parann-Nissany Mar 2011 A1
20110075667 Li et al. Mar 2011 A1
20110097001 Labbi Apr 2011 A1
20110110382 Jabr et al. May 2011 A1
20110116443 Yu et al. May 2011 A1
20110126099 Anderson et al. May 2011 A1
20110128892 Tallet Jun 2011 A1
20110131302 Bulusu Jun 2011 A1
20110138055 Daly et al. Jun 2011 A1
20110145413 Dawson et al. Jun 2011 A1
20110145657 Bishop et al. Jun 2011 A1
20110173303 Rider Jul 2011 A1
20110185063 Head et al. Jul 2011 A1
20110185065 Stanisic et al. Jul 2011 A1
20110206052 Tan et al. Aug 2011 A1
20110213966 Fu et al. Sep 2011 A1
20110219434 Betz et al. Sep 2011 A1
20110231715 Kunii et al. Sep 2011 A1
20110231899 Pulier et al. Sep 2011 A1
20110239039 Dieffenbach et al. Sep 2011 A1
20110252327 Awasthi et al. Oct 2011 A1
20110261811 Battestilli et al. Oct 2011 A1
20110261828 Smith Oct 2011 A1
20110276675 Singh et al. Nov 2011 A1
20110276951 Jain Nov 2011 A1
20110283013 Grosser et al. Nov 2011 A1
20110295998 Ferris et al. Dec 2011 A1
20110305149 Scott et al. Dec 2011 A1
20110307531 Gaponenko et al. Dec 2011 A1
20110320870 Kenigsberg et al. Dec 2011 A1
20120005724 Lee Jan 2012 A1
20120036234 Staats et al. Feb 2012 A1
20120054367 Ramakrishnan et al. Mar 2012 A1
20120072318 Akiyama et al. Mar 2012 A1
20120072578 Alam Mar 2012 A1
20120072581 Tung et al. Mar 2012 A1
20120072985 Davne et al. Mar 2012 A1
20120072992 Arasaratnam et al. Mar 2012 A1
20120084445 Brock et al. Apr 2012 A1
20120084782 Chou et al. Apr 2012 A1
20120096134 Suit Apr 2012 A1
20120102193 Rathore et al. Apr 2012 A1
20120102199 Hopmann et al. Apr 2012 A1
20120131174 Ferris et al. May 2012 A1
20120137215 Kawara May 2012 A1
20120158967 Sedayao et al. Jun 2012 A1
20120159097 Jennas, II et al. Jun 2012 A1
20120167094 Suit Jun 2012 A1
20120173710 Rodriguez Jul 2012 A1
20120179909 Sagi et al. Jul 2012 A1
20120180044 Donnellan et al. Jul 2012 A1
20120182891 Lee et al. Jul 2012 A1
20120185913 Martinez et al. Jul 2012 A1
20120192016 Gotesdyner et al. Jul 2012 A1
20120192075 Ebtekar et al. Jul 2012 A1
20120201135 Ding et al. Aug 2012 A1
20120214506 Skaaksrud et al. Aug 2012 A1
20120222106 Kuehl Aug 2012 A1
20120236716 Anbazhagan et al. Sep 2012 A1
20120240113 Hur Sep 2012 A1
20120265976 Spiers et al. Oct 2012 A1
20120272025 Park et al. Oct 2012 A1
20120281706 Agarwal et al. Nov 2012 A1
20120281708 Chauhan et al. Nov 2012 A1
20120290647 Ellison et al. Nov 2012 A1
20120297238 Watson et al. Nov 2012 A1
20120311106 Morgan Dec 2012 A1
20120311568 Jansen Dec 2012 A1
20120324092 Brown et al. Dec 2012 A1
20120324114 Dutta et al. Dec 2012 A1
20130003567 Gallant et al. Jan 2013 A1
20130013248 Brugler et al. Jan 2013 A1
20130036213 Hasan et al. Feb 2013 A1
20130044636 Koponen et al. Feb 2013 A1
20130066940 Shao Mar 2013 A1
20130080509 Wang Mar 2013 A1
20130080624 Nagai et al. Mar 2013 A1
20130086403 Jenne Apr 2013 A1
20130091557 Gurrapu Apr 2013 A1
20130097601 Podvratnik et al. Apr 2013 A1
20130104140 Meng et al. Apr 2013 A1
20130111540 Sabin May 2013 A1
20130117337 Dunham May 2013 A1
20130124712 Parker May 2013 A1
20130125124 Kempf et al. May 2013 A1
20130138816 Kuo et al. May 2013 A1
20130144978 Jain et al. Jun 2013 A1
20130152076 Patel Jun 2013 A1
20130152175 Hromoko et al. Jun 2013 A1
20130159097 Schory et al. Jun 2013 A1
20130159496 Hamilton et al. Jun 2013 A1
20130160008 Cawlfield et al. Jun 2013 A1
20130162753 Hendrickson et al. Jun 2013 A1
20130169666 Pacheco et al. Jul 2013 A1
20130179941 McGloin et al. Jul 2013 A1
20130182712 Aguayo et al. Jul 2013 A1
20130185433 Zhu et al. Jul 2013 A1
20130191106 Kephart et al. Jul 2013 A1
20130198374 Zalmanovitch et al. Aug 2013 A1
20130201989 Hu et al. Aug 2013 A1
20130204849 Chacko Aug 2013 A1
20130232491 Radhakrishnan et al. Sep 2013 A1
20130246588 Borowicz et al. Sep 2013 A1
20130250770 Zou et al. Sep 2013 A1
20130254356 Eastlake, III Sep 2013 A1
20130254415 Fullen et al. Sep 2013 A1
20130262347 Dodson Oct 2013 A1
20130283364 Chang et al. Oct 2013 A1
20130297769 Chang et al. Nov 2013 A1
20130318240 Hebert et al. Nov 2013 A1
20130318546 Kothuri et al. Nov 2013 A1
20130339949 Spiers et al. Dec 2013 A1
20140006481 Frey et al. Jan 2014 A1
20140006535 Reddy Jan 2014 A1
20140006585 Dunbar et al. Jan 2014 A1
20140040473 Ho et al. Feb 2014 A1
20140040883 Tompkins Feb 2014 A1
20140052877 Mao Feb 2014 A1
20140056146 Hu et al. Feb 2014 A1
20140059310 Du et al. Feb 2014 A1
20140074850 Noel et al. Mar 2014 A1
20140075048 Yuksel et al. Mar 2014 A1
20140075108 Dong et al. Mar 2014 A1
20140075357 Flores et al. Mar 2014 A1
20140075501 Srinivasan et al. Mar 2014 A1
20140089727 Cherkasova et al. Mar 2014 A1
20140098762 Ghai et al. Apr 2014 A1
20140108985 Scott et al. Apr 2014 A1
20140122560 Ramey et al. May 2014 A1
20140136779 Guha et al. May 2014 A1
20140140211 Chandrasekaran et al. May 2014 A1
20140141720 Princen et al. May 2014 A1
20140156557 Zeng et al. Jun 2014 A1
20140164486 Ravichandran et al. Jun 2014 A1
20140188825 Muthukkaruppan et al. Jul 2014 A1
20140189095 Lindberg et al. Jul 2014 A1
20140189125 Amies et al. Jul 2014 A1
20140215471 Cherkasova Jul 2014 A1
20140222953 Karve et al. Aug 2014 A1
20140244851 Lee Aug 2014 A1
20140245298 Zhou et al. Aug 2014 A1
20140281173 Im et al. Sep 2014 A1
20140282536 Dave et al. Sep 2014 A1
20140282611 Campbell et al. Sep 2014 A1
20140282889 Ishaya et al. Sep 2014 A1
20140289200 Kato Sep 2014 A1
20140295831 Karra et al. Oct 2014 A1
20140297569 Clark et al. Oct 2014 A1
20140297835 Buys Oct 2014 A1
20140310391 Sorenson, III et al. Oct 2014 A1
20140310417 Sorenson, III et al. Oct 2014 A1
20140310418 Sorenson, III et al. Oct 2014 A1
20140314078 Jilani Oct 2014 A1
20140317261 Shatzkamer et al. Oct 2014 A1
20140321278 Cafarelli et al. Oct 2014 A1
20140330976 van Bemmel Nov 2014 A1
20140330977 van Bemmel Nov 2014 A1
20140334488 Guichard et al. Nov 2014 A1
20140362682 Guichard et al. Dec 2014 A1
20140365680 van Bemmel Dec 2014 A1
20140366155 Chang et al. Dec 2014 A1
20140369204 Anand et al. Dec 2014 A1
20140372567 Ganesh et al. Dec 2014 A1
20140379938 Bosch et al. Dec 2014 A1
20150033086 Sasturkar et al. Jan 2015 A1
20150043576 Dixon et al. Feb 2015 A1
20150052247 Threefoot et al. Feb 2015 A1
20150052517 Raghu et al. Feb 2015 A1
20150058382 St. Laurent et al. Feb 2015 A1
20150058459 Amendjian et al. Feb 2015 A1
20150071285 Kumar et al. Mar 2015 A1
20150085870 Narasimha et al. Mar 2015 A1
20150089082 Patwardhan et al. Mar 2015 A1
20150100471 Curry, Jr. et al. Apr 2015 A1
20150103827 Quinn et al. Apr 2015 A1
20150106802 Ivanov et al. Apr 2015 A1
20150106805 Melander et al. Apr 2015 A1
20150117199 Chinnaiah Sankaran et al. Apr 2015 A1
20150117458 Gurkan et al. Apr 2015 A1
20150120914 Wada et al. Apr 2015 A1
20150124622 Kovvali et al. May 2015 A1
20150138973 Kumar et al. May 2015 A1
20150178133 Phelan et al. Jun 2015 A1
20150189009 van Bemmel Jul 2015 A1
20150215819 Bosch et al. Jul 2015 A1
20150227405 Jan et al. Aug 2015 A1
20150242204 Hassine et al. Aug 2015 A1
20150249709 Teng et al. Sep 2015 A1
20150263901 Kumar et al. Sep 2015 A1
20150280980 Bitar Oct 2015 A1
20150281067 Wu Oct 2015 A1
20150281113 Siciliano et al. Oct 2015 A1
20150309908 Pearson et al. Oct 2015 A1
20150319063 Zourzouvillys et al. Nov 2015 A1
20150326524 Tankala et al. Nov 2015 A1
20150339210 Kopp et al. Nov 2015 A1
20150358850 La Roche, Jr. et al. Dec 2015 A1
20150365324 Kumar et al. Dec 2015 A1
20150373108 Fleming et al. Dec 2015 A1
20160011925 Kulkarni et al. Jan 2016 A1
20160013990 Kulkarni et al. Jan 2016 A1
20160026684 Mukherjee et al. Jan 2016 A1
20160062786 Meng et al. Mar 2016 A1
20160094389 Jain et al. Mar 2016 A1
20160094398 Choudhury et al. Mar 2016 A1
20160094453 Jain et al. Mar 2016 A1
20160094454 Jain et al. Mar 2016 A1
20160094455 Jain et al. Mar 2016 A1
20160094456 Jain et al. Mar 2016 A1
20160094480 Kulkarni et al. Mar 2016 A1
20160094643 Jain et al. Mar 2016 A1
20160099847 Melander et al. Apr 2016 A1
20160099853 Nedeltchev et al. Apr 2016 A1
20160099864 Akiya et al. Apr 2016 A1
20160105393 Thakkar et al. Apr 2016 A1
20160127184 Bursell May 2016 A1
20160134557 Steinder et al. May 2016 A1
20160156708 Jalan et al. Jun 2016 A1
20160164780 Timmons et al. Jun 2016 A1
20160164914 Madhav et al. Jun 2016 A1
20160182378 Basavaraja et al. Jun 2016 A1
20160188527 Cherian et al. Jun 2016 A1
20160234071 Nambiar et al. Aug 2016 A1
20160239399 Babu et al. Aug 2016 A1
20160253078 Ebtekar et al. Sep 2016 A1
20160254968 Ebtekar et al. Sep 2016 A1
20160261564 Foxhoven et al. Sep 2016 A1
20160277368 Narayanaswamy et al. Sep 2016 A1
20170005948 Melander et al. Jan 2017 A1
20170024260 Chandrasekaran et al. Jan 2017 A1
20170026294 Basavaraja et al. Jan 2017 A1
20170026470 Bhargava et al. Jan 2017 A1
20170041342 Efremov et al. Feb 2017 A1
20170054659 Ergin et al. Feb 2017 A1
20170097841 Chang et al. Apr 2017 A1
20170099188 Chang et al. Apr 2017 A1
20170104755 Arregoces et al. Apr 2017 A1
20170147297 Krishnamurthy et al. May 2017 A1
20170149878 Mutnuru May 2017 A1
20170163531 Kumar et al. Jun 2017 A1
20170171158 Hoy et al. Jun 2017 A1
20170264663 Bicket et al. Sep 2017 A1
20170339070 Chang et al. Nov 2017 A1
Foreign Referenced Citations (13)
Number Date Country
101719930 Jun 2010 CN
101394360 Jul 2011 CN
102164091 Aug 2011 CN
104320342 Jan 2015 CN
105740084 Jul 2016 CN
2228719 Sep 2010 EP
2439637 Apr 2012 EP
2645253 Nov 2014 EP
10-2015-0070676 May 2015 KR
M394537 Dec 2010 TW
WO 2009155574 Dec 2009 WO
WO 2010030915 Mar 2010 WO
WO 2013158707 Oct 2013 WO
Non-Patent Literature Citations (70)
Entry
Author Unknown, “Understanding Multiple Spanning Tree Protocol (802.1s),” Document ID: 24248, Cisco Systems, Inc., Apr. 17, 2007, 13 pages.
Author Unknown, “IEEE Standard for Information technology. Part 3: Media Access Control (MAC) Bridges,” ANSI/IEEE Std 802.1D, 1998 Edition, 373 pages.
Author Unknown, “IEEE Standard for Local and metropolitan area networks, Virtual Bridged Local Area Networks,” IEEE Std 802.10-2005, May 19, 2006, 303 pages.
Seaman, Mick, “Spanning Vines,” Rev. 0.1, Mar. 5, 2002, 6 pages.
Amedro, Brian, et al., “An Efficient Framework for Running Applications on Clusters, Grids and Cloud,” 2010, 17 pages.
Author Unknown, “5 Benefits of a Storage Gateway in the Cloud,” Blog, TwinStrata, Inc., Jul. 25, 2012, XP055141645, 4 pages, https://web.archive.org/web/20120725092619/http://blog.twinstrata.com/2012/07/10//5-benefits-of-a-storage-gateway-in-the-cloud.
Author Unknown, “Joint Cisco and VMWare Solution for Optimizing Virtual Desktop Delivery: Data Center 3.0: Solutions to Accelerate Data Center Virtualization,” Cisco Systems, Inc. and VMware, Inc., Sep. 2008, 10 pages.
Author Unknown, “A Look at DeltaCloud: The Multi-Cloud API,” Feb. 17, 2012, 4 pages.
Author Unknown, “About Deltacloud,” Apache Software Foundation, Aug. 18, 2013, 1 page.
Author Unknown, “Architecture for Managing Clouds, A White Paper from the Open Cloud Standards Incubator,” Version 1.0.0, Document No. DSP-IS0102, Jun. 18, 2010, 57 pages.
Author Unknown, “Cloud Infrastructure Management Interface—Common Information Model (CIMI-CIM),” Document No. DSP0264, Version 1.0.0, Dec. 14, 2012, 21 pages.
Author Unknown, “Cloud Infrastructure Management Interface (CIMI) Primer,” Document No. DSP2027, Version 1.0.1, Sep. 12, 2012, 30 pages.
Author Unknown, “cloudControl Documentation,” Aug. 25, 2013, 14 pages.
Author Unknown, “Interoperable Clouds, A White Paper from the Open Cloud Standards Incubator,” Version 1.0.0, Document No. DSP-IS0101, Nov. 11, 2009, 21 pages.
Author Unknown, “Microsoft Cloud Edge Gateway (MCE) Series Appliance,” Iron Networks, Inc., 2014, 4 pages.
Author Unknown, “Open Data Center Alliance Usage: Virtual Machine (VM) Interoperability in a Hybrid Cloud Environment Rev. 1.2,” Open Data Center Alliance, Inc., 2013, 18 pages.
Author Unknown, “Real-Time Performance Monitoring on Juniper Networks Devices, Tips and Tools for Assessing and Analyzing Network Efficiency,” Juniper Networks, Inc., May 2010, 35 pages.
Author Unknown, “Use Cases and Interactions for Managing Clouds, A White Paper from the Open Cloud Standards Incubator,” Version 1.0.0, Document No. DSP-ISO0103, Jun. 16, 2010, 75 pages.
Author Unknown, “Apache Ambari Meetup What's New,” Hortonworks Inc., Sep. 2013, 28 pages.
Author Unknown, “Introduction,” Apache Ambari project, Apache Software Foundation, 2014, 1 page.
Baker, F., “Requirements for IP Version 4 Routers,” Jun. 1995, 175 pages, Network Working Group, Cisco Systems.
Beyer, Steffen, “Module “Data::Locations?!”,” YAPC::Europe, London, UK,ICA, Sep. 22-24, 2000, XP002742700, 15 pages.
Blanchet, M., “A Flexible Method for Managing the Assignment of Bits of an IPv6 Address Block,” Apr. 2003, 8 pages, Network Working Group, Viagnie.
Borovick, Lucinda, et al., “Architecting the Network for the Cloud,” IDC White Paper, Jan. 2011, 8 pages.
Bosch, Greg, “Virtualization,” last modified Apr. 2012 by B. Davison, 33 pages.
Broadcasters Audience Research Board, “What's Next,” http://lwww.barb.co.uk/whats-next, accessed Jul. 22, 2015, 2 pages.
Cisco Systems, Inc. “Best Practices in Deploying Cisco Nexus 1000V Series Switches on Cisco UCS B and C Series Cisco UCS Manager Servers,” Cisco White Paper, Apr. 2011, 36 pages, http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-558242.pdf.
Cisco Systems, Inc., “Cisco Unified Network Services: Overcome Obstacles to Cloud-Ready Deployments,” Cisco White Paper, Jan. 2011, 6 pages.
Cisco Systems, Inc., “Cisco Intercloud Fabric: Hybrid Cloud with Choice, Consistency, Control and Compliance,” Dec. 10, 2014, 22 pages.
Cisco Technology, Inc., “Cisco Expands Videoscape TV Platform Into the Cloud,” Jan. 6, 2014, Las Vegas, Nevada, Press Release, 3 pages.
Citrix, “Citrix StoreFront 2.0” White Paper, Proof of Concept Implementation Guide, Citrix Systems, Inc., 2013, 48 pages.
Citrix, “CloudBridge for Microsoft Azure Deployment Guide,” 30 pages.
Citrix, “Deployment Practices and Guidelines for NetScaler 10.5 on Amazon Web Services,” White Paper, citrix.com, 2014, 14 pages.
CSS Corp, “Enterprise Cloud Gateway (ECG)—Policy driven framework environments,” original published on or about Feb. 11, 2012; 1 page; http://www.css-cloud.com/platform/enterprise-cloud-gateway.php.
Fang K., “LISP MAC-EID-TO-RLOC Mapping (LISP based L2VPN),” Network Working Group, Internet Draft, CISCO Systems, Jan. 2012, 12 pages.
Ford, Bryan, et al., Peer-to-Peer Communication Across Network Address Translators, In USENIX Annual Technical Conference, 2005, pp. 179-192.
Gedymin, Adam, “Cloud Computing with an emphasis on Google App Engine,” Sep. 2011, 146 pages.
Good, Nathan A., “Use Apache Deltacloud to administer multiple instances with a single API,” Dec. 17, 2012, 7 pages.
Herry, William, “Keep It Simple, Stupid: OpenStack nova-scheduler and its algorithm”, May 12, 2012, IBM, 12 pages.
Hewlett-Packard Company, “Virtual context management on network devices”, Research Disclosure, vol. 564, No. 60, Apr. 1, 2011, Mason Publications, Hampshire, GB, Apr. 1, 2011, 524.
Juniper Networks, Inc., “Recreating Real Application Traffic in Junosphere Lab,” Solution Brief, Dec. 2011, 3 pages.
Kenhui, “Musings on Cloud Computing and IT-as-a-Service: [Updated for Havana] Openstack Computer for VSphere Admins, Part 2: Nova-Scheduler and DRS”, Jun. 26, 2013, Cloud Architect Musings, 12 pages.
Kolyshkin, Kirill, “Virtualization in Linux,” Sep. 1, 2006, XP055141648, 5 pages, https://web.archive.ora/web/20070120205111/http://download.openvz.ora/doc/openvz-intro.pdf.
Kumar, S., et al., “Infrastructure Service Forwarding for NSH,”Service Function Chaining Internet Draft, draft-kumar-sfc-nsh-forwarding-00, Dec. 5, 2015, 10 pages.
Kunz, Thomas, et al., “OmniCloud—The Secure and Flexible Use of Cloud Storage Services,” 2014, 30 pages.
Lerach, S.R.O., “Golem,” http://www.lerach.cz/en/products/golem, accessed Jul. 22, 2015, 2 pages.
Linthicum, David, “VM Import could be a game changer for hybrid clouds”, InfoWorld, Dec. 23, 2010, 4 pages.
Logan, Marcus, “Hybrid Cloud Application Architecture for Elastic Java-Based Web Applications,” F5 Deployment Guide Version 1.1, 2016, 65 pages.
Lynch, Sean, “Monitoring cache with Claspin” Facebook Engineering, Sep. 19, 2012, 5 pages.
Meireles, Fernando Miguel Dias, “Integrated Management of Cloud Computing Resources,” 2013-2014, 286 pages.
Meraki, “meraki releases industry's first cloud-managed routers,” Jan. 13, 2011, 2 pages.
Mu, Shuai, et al., “uLibCloud: Providing High Available and Uniform Accessing to Multiple Cloud Storages,” 2012 IEEE, 8 pages.
Naik, Vijay K., et al., “Harmony: A Desktop Grid for Delivering Enterprise Computations,” Grid Computing, 2003, Fourth International Workshop on Proceedings, Nov 17, 2003, pp. 1-11.
Nair, Srijith K. et al., “Towards Secure Cloud Bursting, Brokerage and Aggregation,” 2012, 8 pages, www.flexiant.com.
Nielsen, “SimMetry Audience Measurement—Technology,” http://www.nielsen-admosphere.eu/products-and-services/simmetry-audience-measurement-technology/, accessed Jul. 22, 2015, 6 pages.
Nielsen, “Television,” http://www.nielsen.com/us/en/solutions/measurement/television.html, accessed Jul. 22, 2015, 4 pages.
Open Stack, “Filter Scheduler,” updated Dec. 17, 2017, 5 pages, accessed on Dec. 18, 2017, https://docs.openstack.org/nova/latest/user/filter-scheduler.html.
Quinn, P., et al., “Network Service Header,” Internet Engineering Task Force Draft, Jul. 3, 2014, 27 pages.
Quinn, P., et al., “Service Function Chaining (SFC) Architecture,” Network Working Group, Internet Draft, draft-quinn-sfc-arch-03.txt, Jan. 22, 2014, 21 pages.
Rabadan, J., et al., “Operational Aspects of Proxy-ARP/ND in EVPN Networks,” BESS Worksgroup Internet Draft, draft-snr-bess-evpn-proxy-arp-nd-02, Oct. 6, 2015, 22 pages.
Saidi, Ali, et al., “Performance Validation of Network-Intensive Workloads on a Full-System Simulator,” Interaction between Operating System and Computer Architecture Workshop, (IOSCA 2005), Austin, Texas, Oct. 2005, 10 pages.
Shunra, “Shunra for HP Software; Enabling Confidence in Application Performance Before Deployment,” 2010, 2 pages.
Son, Jungmin, “Automatic decision system for efficient resource selection and allocation in inter-clouds,” Jun. 2013, 35 pages.
Sun, Aobing, et al., “IaaS Public Cloud Computing Platform Scheduling Model and Optimization Analysis,” Int. J. Communications, Network and System Sciences, 2011, 4, 803-811, 9 pages.
Szymaniak, Michal, et al., “Latency-Driven Replica Placement”, vol. 47 No. 8, IPSJ Journal, Aug. 2006, 12 pages.
Toews, Everett, “Introduction to Apache jclouds,” Apr. 7, 2014, 23 pages.
Von Laszewski, Gregor, et al., “Design of a Dynamic Provisioning System for a Federated Cloud and Bare-metal Environment,” 2012, 8 pages.
Wikipedia, “Filter (software)”, Wikipedia, Feb. 8, 2014, 2 pages, https://en.wikipedia.org/w/index.php?title=Filter_%28software%29&oldid=594544359.
Wikipedia; “Pipeline (Unix)”, Wikipedia, May 4, 2014, 4 pages, https://en.wikipedia.org/w/index.php?title=Pipeline2/028Unix%29&oldid=606980114.
Ye, Xianglong, et al., “A Novel Blocks Placement Strategy for Hadoop,” 2012 IEEE/ACTS 11th International Conference on Computer and Information Science, 2012 IEEE, 5 pages.
Related Publications (1)
Number Date Country
20170331730 A1 Nov 2017 US
Continuations (1)
Number Date Country
Parent 13168571 Jun 2011 US
Child 15663503 US