Progressive MAC address learning

Abstract

One embodiment of the present invention provides a switch in a network of interconnected switches. The switch includes a storage device, a hardware management apparatus, and a layer-2 management apparatus. The storage device stores a forwarding table, which includes an entry comprising a MAC address and an egress port for the MAC address. The hardware management apparatus determines whether a destination MAC address of a frame is present in a hardware table in memory of the switch. The layer-2 management apparatus, in response to a determination that the destination MAC address is not present in the hardware table, looks up a first entry comprising the destination MAC address in the forwarding table, and creates a second entry comprising the destination MAC address in the hardware table based on the first entry.

Description

BACKGROUND

Field

This disclosure relates to communication networks. More specifically, this disclosure relates to a system and method for efficient media access control (MAC) address learning.

Related Art

The exponential growth of the Internet has made it a popular delivery medium for a variety of applications running on physical and virtual devices. Such applications have brought with them an increasing demand for bandwidth. As a result, equipment vendors race to build larger and faster switches with versatile capabilities, such as network virtualization and multi-tenancy, to accommodate diverse network demands efficiently. However, the size of a switch cannot grow infinitely. It is limited by physical space, power consumption, and design complexity, to name a few factors. Furthermore, switches with higher capability are usually more complex and expensive. More importantly, because an overly large and complex system often does not provide economy of scale, simply increasing the size and capability of a switch may prove economically unviable due to the increased per-port cost.

A flexible way to improve the scalability of a switch system is to build a fabric switch. A fabric switch is a collection of individual member switches. These member switches form a single, logical switch that can have an arbitrary number of ports and an arbitrary topology. As demands grow, customers can adopt a “pay as you grow” approach to scale up the capacity of the fabric switch.

Meanwhile, layer-2 and layer-3 (e.g., Ethernet and Internet Protocol (IP), respectively) switching technologies continue to evolve. IP facilitates routing and end-to-end data transfer in wide area networks (WANs) while providing safeguards for error-free communication. On the other hand, more routing-like functionalities are migrating into layer-2. Notably, the development of the Transparent Interconnection of Lots of Links (TRILL) protocol allows Ethernet switches to function more like routing devices. TRILL overcomes the inherent inefficiency of the conventional spanning tree protocol, which forces layer-2 switches to be coupled in a logical spanning-tree topology to avoid looping. TRILL allows routing bridges (RBridges) to be coupled in an arbitrary topology without the risk of looping by implementing routing functions in switches and including a hop count in the TRILL header.

As Internet traffic becomes more diverse, network virtualization is becoming progressively more important as a value proposition for network architects. In addition, the evolution of virtual computing has made multi-tenancy attractive and, consequently, placed additional requirements on the network. For example, a switch may couple a large number of virtual servers and learn the virtual MAC addresses of the virtual servers. It is often desirable that the network infrastructure can provide a scalable MAC address management for a large number of end devices.

While today's networks support many desirable features, some issues remain unsolved in efficiently facilitating scalable MAC address learning.

SUMMARY

One embodiment of the present invention provides a switch in a network of interconnected switches. The switch includes a storage device, a hardware management apparatus, and a layer-2 management apparatus. The storage device stores a forwarding table, which includes an entry comprising a media access control (MAC) address and an egress port for the MAC address. The hardware management apparatus determines whether a destination MAC address of a frame is present in a hardware table in memory of the switch. The layer-2 management apparatus, in response to a determination that the destination MAC address is not present in the hardware table, looks up a first entry comprising the destination MAC address in the forwarding table, and creates a second entry comprising the destination MAC address in the hardware table based on the first entry.

In a variation on this embodiment, the switch also includes a learning management apparatus, which learns a first MAC address from a local edge port. The layer-2 management apparatus then creates respective entries in the forwarding and hardware tables comprising the learned MAC address.

In a variation on this embodiment, the switch also includes a learning management apparatus, which extracts a remote MAC address learned at a port of a second switch from a payload of a notification message. The layer-2 management apparatus then creates an entry in the forwarding table comprising the remote MAC address.

In a variation on this embodiment, the layer-2 management apparatus, in response to a determination that the destination MAC address is present in the hardware table, identifies an egress port for the frame from an entry comprising the destination MAC address in the hardware table.

In a variation on this embodiment, if the frame has been received via a first port in a first chip of the switch, the layer-2 management apparatus is precluded from creating an entry comprising the destination MAC address in a hardware table of a second chip of the switch.

In a variation on this embodiment, the layer-2 management apparatus clears the second entry in response to identifying an aging out event for the destination MAC address. However, the first entry remains in the forwarding table.

In a further variation, the layer-2 management apparatus clears the first entry in response to identifying an aging out event of a remote switch in a notification message. The destination MAC address has been learned at a port in the remote switch.

In a variation on this embodiment, the switch and a remote switch participate in a virtual link aggregation group coupling an end device with the switch and the remote switch. The hardware table of the switch includes at least one different MAC address than a remote hardware table of the remote switch.

In a variation on this embodiment, the layer-2 management apparatus, in response to determining a failure to the lookup, floods the frame in the network.

In a variation on this embodiment, the network is a switch group operating as a single Ethernet switch. A respective switch of the network is associated with a group identifier identifying the switch group.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A illustrates an exemplary network with support for progressive media access control (MAC) address learning, in accordance with an embodiment of the present invention.

FIG. 1B illustrates exemplary forwarding and hardware tables for supporting progressive MAC address learning, in accordance with an embodiment of the present invention.

FIG. 1C illustrates exemplary progressive MAC address learning in switch hardware, in accordance with an embodiment of the present invention.

FIG. 2A illustrates an exemplary progressive MAC address learning process based on an Address Resolution Protocol (ARP) request, in accordance with an embodiment of the present invention.

FIG. 2B illustrates an exemplary progressive MAC address learning process based on an initial communication from an end device, in accordance with an embodiment of the present invention.

FIG. 3A presents a flowchart illustrating the process of a switch learning a progressive MAC address via a local edge port, in accordance with an embodiment of the present invention.

FIG. 3B presents a flowchart illustrating the process of a switch learning a progressive MAC address from a notification message, in accordance with an embodiment of the present invention.

FIG. 3C presents a flowchart illustrating the process of a switch timing out a learned MAC address, in accordance with an embodiment of the present invention.

FIG. 4 presents a flowchart illustrating the process of a switch forwarding a packet based on learned progressive MAC addresses, in accordance with an embodiment of the present invention.

FIG. 5A illustrates an exemplary virtual link aggregation group with support for progressive MAC address learning, in accordance with an embodiment of the present invention.

FIG. 5B illustrates an exemplary high-availability for a virtual link aggregation group with support for progressive MAC address learning, in accordance with an embodiment of the present invention.

FIG. 6 illustrates an exemplary switch with support for progressive MAC address learning, in accordance with an embodiment of the present invention.

In the figures, like reference numerals refer to the same figure elements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the claims.

Overview

In embodiments of the present invention, the problem of facilitating scalable MAC address learning in a switch is solved by incorporating only the MAC addresses currently in use in the switch in the hardware table of the switch. With existing technologies, an end device can operate as a host machine with a plurality of virtual machines, each of which may have multiple MAC addresses. As a result, a switch coupling a large number of such servers may need to learn a large number of MAC addresses. Furthermore, in a network, such as a fabric switch, where a learned MAC address is shared among the member switches of the network, the burden of a respective switch learning a respective MAC address can be significant. This can become a bottleneck for scaling up such a network.

To solve this problem, embodiments of the present invention allow a switch in a network to identify conversational MAC addresses (i.e., the MAC addresses in use in active communications). The switch can learn a MAC address via a local edge port (e.g., based on Ethernet MAC address learning) or from the payload of a notification message from a remote switch (e.g., in a fabric switch). The switch creates an entry comprising a respective learned MAC address and a corresponding egress port identifier in a forwarding table (e.g., a software table). If the switch identifies a remotely learned MAC address as a destination MAC address, the switch considers that MAC address as a conversational MAC address. The switch also considers a locally learned MAC address as a conversational MAC address since it has received at least one packet from that MAC address.

The switch then incorporates the entry comprising the conversational MAC address from the forwarding table into the local hardware table (e.g., an Exact Match (EXM) Table in the Content Addressable Memory (CAM)). As a result, the hardware table only includes the MAC address currently in use in an active communication at the switch. In this way, the switch uses its limited hardware table resources for the MAC addresses currently in use, thereby allowing the switch to support a large number of active communications and scale up a network.

In some embodiments, the network is a fabric switch. In a fabric switch, any number of switches coupled in an arbitrary topology may logically operate as a single switch. The fabric switch can be an Ethernet fabric switch or a virtual cluster switch (VCS), which can operate as a single Ethernet switch. Any member switch may join or leave the fabric switch in “plug-and-play” mode without any manual configuration. In some embodiments, a respective switch in the fabric switch is a Transparent Interconnection of Lots of Links (TRILL) routing bridge (RBridge). In some further embodiments, a respective switch in the fabric switch is an Internet Protocol (IP) routing-capable switch (e.g., an IP router).

It should be noted that a fabric switch is not the same as conventional switch stacking. In switch stacking, multiple switches are interconnected at a common location (often within the same rack), based on a particular topology, and manually configured in a particular way. These stacked switches typically share a common address, e.g., an IP address, so they can be addressed as a single switch externally. Furthermore, switch stacking requires a significant amount of manual configuration of the ports and inter-switch links. The need for manual configuration prohibits switch stacking from being a viable option in building a large-scale switching system. The topology restriction imposed by switch stacking also limits the number of switches that can be stacked. This is because it is very difficult, if not impossible, to design a stack topology that allows the overall switch bandwidth to scale adequately with the number of switch units.

In contrast, a fabric switch can include an arbitrary number of switches with individual addresses, can be based on an arbitrary topology, and does not require extensive manual configuration. The switches can reside in the same location, or be distributed over different locations. These features overcome the inherent limitations of switch stacking and make it possible to build a large “switch farm,” which can be treated as a single, logical switch. Due to the automatic configuration capabilities of the fabric switch, an individual physical switch can dynamically join or leave the fabric switch without disrupting services to the rest of the network.

Furthermore, the automatic and dynamic configurability of the fabric switch allows a network operator to build its switching system in a distributed and “pay-as-you-grow” fashion without sacrificing scalability. The fabric switch's ability to respond to changing network conditions makes it an ideal solution in a virtual computing environment, where network loads often change with time.

It should also be noted that a fabric switch is distinct from a VLAN. A fabric switch can accommodate a plurality of VLANs. A VLAN is typically identified by a VLAN tag. In contrast, the fabric switch is identified by a fabric identifier (e.g., a VCS identifier), which is assigned to the fabric switch. A respective member switch of the fabric switch is associated with the fabric identifier. Furthermore, when a member switch of a fabric switch learns a media access control (MAC) address of an end device (e.g., via layer-2 MAC address learning), the member switch generates a notification message, includes the learned MAC address in the payload of the notification message, and sends the notification message to all other member switches of the fabric switch. In this way, a learned MAC address is shared with a respective member switch of the fabric switch.

In this disclosure, the term “fabric switch” refers to a number of interconnected physical switches which form a single, scalable logical switch. These physical switches are referred to as member switches of the fabric switch. In a fabric switch, any number of switches can be connected in an arbitrary topology, and the entire group of switches functions together as one single, logical switch. This feature makes it possible to use many smaller, inexpensive switches to construct a large fabric switch, which can be viewed as a single logical switch externally. Although the present disclosure is presented using examples based on a fabric switch, embodiments of the present invention are not limited to a fabric switch. Embodiments of the present invention are relevant to any computing device that includes a plurality of devices operating as a single device.

Although the present disclosure is presented using examples based on an encapsulation protocol, embodiments of the present invention are not limited to networks defined using one particular encapsulation protocol associated with a particular Open System Interconnection Reference Model (OSI reference model) layer. For example, embodiments of the present invention can also be applied to a multi-protocol label switching (MPLS) network. In this disclosure, the term “encapsulation” is used in a generic sense, and can refer to encapsulation in any networking layer, sub-layer, or a combination of networking layers.

The term “end device” can refer to any device external to a network (e.g., which does not perform forwarding in that network). Examples of an end device include, but are not limited to, a physical or virtual machine, a conventional layer-2 switch, a layer-3 router, or any other type of network device. Additionally, an end device can be coupled to other switches or hosts further away from a layer-2 or layer-3 network. An end device can also be an aggregation point for a number of network devices to enter the network. An end device hosting one or more virtual machines can be referred to as a host machine. In this disclosure, the terms “end device” and “host machine” are used interchangeably.

The term “hypervisor” is used in a generic sense, and can refer to any virtual machine manager. Any software, firmware, or hardware that creates and runs virtual machines can be a “hypervisor.” The term “virtual machine” is also used in a generic sense and can refer to software implementation of a machine or device. Any virtual device which can execute a software program similar to a physical device can be a “virtual machine.” A host external device on which a hypervisor runs one or more virtual machines can be referred to as a “host machine.”

The term “VLAN” is used in a generic sense, and can refer to any virtualized network. Any virtualized network comprising a segment of physical networking devices, software network resources, and network functionality can be referred to as a “VLAN.” “VLAN” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. “VLAN” can be replaced by other terminologies referring to a virtualized network or network segment, such as “Virtual Private Network (VPN),” “Virtual Private LAN Service (VPLS),” or “Easy Virtual Network (EVN).”

The term “packet” refers to a group of bits that can be transported together across a network. “Packet” should not be interpreted as limiting embodiments of the present invention to layer-3 networks. “Packet” can be replaced by other terminologies referring to a group of bits, such as “frame,” “cell,” or “datagram.”

The term “switch” is used in a generic sense, and can refer to any standalone or fabric switch operating in any network layer. “Switch” can be a physical device or software running on a computing device. “Switch” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. Any device that can forward traffic to an external device or another switch can be referred to as a “switch.” Examples of a “switch” include, but are not limited to, a layer-2 switch, a layer-3 router, a TRILL RBridge, or a fabric switch comprising a plurality of similar or heterogeneous smaller physical switches.

The term “RBridge” refers to routing bridges, which are bridges implementing the TRILL protocol as described in Internet Engineering Task Force (IETF) Request for Comments (RFC) “Routing Bridges (RBridges): Base Protocol Specification,” available at http://tools.ietf.org/html/rfc6325, which is incorporated by reference herein. Embodiments of the present invention are not limited to application among RBridges. Other types of switches, routers, and forwarders can also be used.

The term “edge port” refers to a port on a network which exchanges data frames with a device outside of the network (i.e., an edge port is not used for exchanging data frames with another member switch of a network). The term “inter-switch port” refers to a port which sends/receives data frames among member switches of the network. The terms “interface” and “port” are used interchangeably.

The term “switch identifier” refers to a group of bits that can be used to identify a switch. Examples of a switch identifier include, but are not limited to, a media access control (MAC) address, an Internet Protocol (IP) address, and an RBridge identifier. Note that the TRILL standard uses “RBridge ID” (RBridge identifier) to denote a 48-bit intermediate-system-to-intermediate-system (IS-IS) System ID assigned to an RBridge, and “RBridge nickname” to denote a 16-bit value that serves as an abbreviation for the “RBridge ID.” In this disclosure, “switch identifier” is used as a generic term, is not limited to any bit format, and can refer to any format that can identify a switch. The term “RBridge identifier” is also used in a generic sense, is not limited to any bit format, and can refer to “RBridge ID,” “RBridge nickname,” or any other format that can identify an RBridge.

The term “tunnel” refers to a data communication where one or more networking protocols are encapsulated using another networking protocol. Although the present disclosure is presented using examples based on a layer-3 encapsulation of a layer-2 protocol, “tunnel” should not be interpreted as limiting embodiments of the present invention to layer-2 and layer-3 protocols. A “tunnel” can be established for and using any networking layer, sub-layer, or a combination of networking layers.

Network Architecture

FIG. 1A illustrates an exemplary network with support for progressive media access control (MAC) address learning, in accordance with an embodiment of the present invention. As illustrated in FIG. 1A, a network 100 includes member switches 101, 102, 103, 104, and 105. Network 100 can be a TRILL network and a respective member switch, such as switch 103, can be a TRILL RBridge. Network 100 can also be an IP network and a respective member switch, such as switch 103, can be an IP-capable switch, which calculates and maintains a local IP routing table (e.g., a routing information base or RIB), and is capable of forwarding packets based on its IP addresses. In some embodiments, network 100 is a fabric switch, and one or more switches in fabric switch 100 can be virtual switches (e.g., a software switch running on a computing device).

Switch 103 is coupled to end devices 111 and 112, switch 105 is coupled to end devices 113 and 114, and switch 102 is coupled to end devices 115 and 116. Member switches in network 100 use edge ports to communicate with end devices and inter-switch ports to communicate with other member switches. For example, switch 103 is coupled to end devices 111 and 112 via edge ports and to switches 101, 102, and 104 via inter-switch ports. End devices 111 and 112 can reside within the same local area network. As a result, a packet from end device 111 to end device 112 can be forwarded without switch 103 forwarding it.

With existing technologies, end devices 111-116 can operate as host machines hosting a plurality of virtual machines, each of which may have multiple MAC addresses. As a result, switches 102, 103, and 105 may need to learn a large number of MAC addresses. If network 100 is a fabric switch, where a learned MAC address is shared among the member switches, the burden of a respective switch in network 100 learning a respective MAC address can be significant. This can become a bottleneck for scaling up network 100.

To solve this problem, switches in network 100 can identify conversational MAC addresses. During operation, a switch in network 100, such as switch 103, learns end device 111's MAC address via a local edge port (e.g., based on Ethernet MAC address learning). Upon learning the MAC address, switch 103 creates an entry comprising the MAC address and a corresponding egress port identifier of the port coupling end device 111 in a forwarding table 181 (e.g., a software table, such as a forwarding database). In some embodiments, switch 103 creates a notification message, includes the learned MAC address in the notification message, and sends the notification message to other switches in network 100.

Upon receiving the notification message, a respective other switch stores the learned MAC address in the local forwarding table in association with switch 103's identifier. For example, switch 102 stores the learned MAC address and switch 103's identifier in forwarding table 185. In some embodiments, the entry includes a bitmap indicating the MAC address's association with switch 103. For example, network 100 includes five switches. Hence, a bitmap of five bits can indicate the association between a MAC address and a corresponding switch. For example, the entry in forwarding table 185 can include a bitmap “00100,” which indicates that the MAC address is associated with switch 103.

Similarly, switch 105 learns end device 113's MAC address via a local edge port and creates an entry comprising the MAC address and a corresponding egress port identifier in a forwarding table 183. In the same way, switch 102 learns end device 115's MAC address via a local edge port and creates an entry comprising the MAC address and a corresponding egress port identifier in a forwarding table 185. Switches 102 and 105 can also share the respective learned MAC addresses with other switches in network 100. As a result, switch 103 learns the MAC addresses of end devices 113 and 115, and stores them in forwarding table 181. A respective switch in network 100, either locally or remotely, can also learn the respective MAC addresses of end devices 112, 114, and 116.

However, a switch in network 100 only includes the locally learned MAC addresses in the hardware table (e.g., the EXM table in a CAM). For example, switch 103 only includes the MAC addresses of end devices 111 and 112 in local hardware table 182. Similarly, switch 105 includes the MAC addresses of end devices 113 and 114 in local hardware table 184, and switch 102 includes the MAC addresses of end devices 115 and 116 in local hardware table 186. A switch includes an entry comprising a remotely learned MAC address in a local hardware table if that MAC address is in use in an active communication (i.e., is a conversational MAC address).

During operation, end device 112 sends a packet 190 to end device 116. Switch 103 receives the packet and learns the source MAC address of packet 190 (i.e., end device 112's MAC address), as described above. Switch 103 determines that the destination MAC address of packet 190 is not present in hardware table 182. Switch 103 then checks whether the MAC address is present in forwarding table 181. Since switch 103 has learned end device 116's MAC address from a notification message from switch 102, forwarding table 181 includes an entry comprising end device 116's MAC address and the switch identifier of switch 102. Switch 102 then creates a corresponding entry in hardware table 182 with end device 116's MAC address and a corresponding egress port.

In some embodiments, the unavailability of end device 116's MAC address in hardware table 182 causes an exception at switch 103. This exception can be triggered at the edge ports of switch 103. In response to the exception, switch 103 consults forwarding table 181 to check whether forwarding table 181 includes end device 116's MAC address. Without the support for maintaining a conversational MAC address, switch 103 floods network 100 if a destination MAC address is not present in forwarding table 181. The support for maintaining the conversational MAC address can be indicated by a set value of a Learning in Progress (LiP) bit in switch 103.

Switch 103 thus determines the destination MAC address of packet 190 as a conversational MAC address and creates an entry in hardware table 182 based on the corresponding entry in forwarding table 181. In some embodiments, switch 103 also sets a flag of the entry in the forwarding table indicating that the MAC address is a conversational MAC address. Switch 103 can also add the entry in a download list (e.g., can add the index of the entry in the download list). The layer-2 management apparatus of switch 103 consults the download list and generates the corresponding entries of the download list in hardware table 182.

In this way, hardware table 182 only includes the MAC address currently in use in an active communication at switch 103. This allows switch 103 to use the limited resources of hardware table 182 for the conversational MAC addresses, thereby allowing switch 103 to support a large number of active communications. Upon crating the entry in hardware table 182, switch 103 encapsulates packet 190 in an encapsulation header 193 and generates encapsulated packet 192. Switch 103 includes switch 102's switch identifier as the egress switch identifier in encapsulation header 193 based on the entry in hardware table 182. Switch 103 then forwards encapsulated packet 192 based on the forwarding entry in hardware table 182.

Encapsulation header 193 can be a fabric encapsulation header (e.g., an encapsulation header used to forward the packet in a fabric switch) or a tunnel header (e.g., an encapsulation header used to forward the packet via a tunnel). Examples of a fabric encapsulation header include, but are not limited to, a TRILL header, an IP header, an Ethernet header, and a combination thereof. Examples of a tunnel include, but are not limited to, Virtual Extensible Local Area Network (VXLAN), Generic Routing Encapsulation (GRE), and its variations, such as Network Virtualization using GRE (NVGRE) and openvSwitch GRE. The VLAN identifier of a global VLAN can be included in the encapsulation header.

If switch 103 does not find an unknown destination MAC address of a packet in forwarding table 181, switch 103 creates an entry comprising the unknown destination MAC address in forwarding table 181 and indicates that the corresponding packet has been flooded in network 100. Switch 103 can also create an entry with the MAC address in hardware table 182, indicate flooding in the corresponding virtual local area network (VLAN), and clear the LiP bit. Indicating in the entry that the packet has been flooded ensures that there are not further exceptions for the same MAC address. When switch 103 receives the switch identifier associated with the MAC address, switch 103 updates the corresponding entries in tables 181 and 182.

In some embodiments, progressive MAC address learning supports protection against Denial of Service (DoS) attacks. Upon identifying the exception, switch 103 identifies the corresponding conversational MAC address from forwarding table 181 and includes it in hardware table 182. During a DoS attack, the same unknown destination MAC address continues to generate the exception until the unknown destination MAC address is included in hardware table 182. To prevent such an attack, progressive MAC address learning uses the LiP bit. Upon detecting the exception, the hardware driver of switch 103 (e.g., the application-specific integrated circuit (ASIC) driver) sets the LiP bit and provides the unknown destination MAC address from forwarding table 181 to the layer-2 management apparatus of switch 103. The apparatus clears the LiP bit while creating an entry comprising the MAC address in hardware table 182. When the MAC address is not present in forwarding table 181, the hardware driver can clear the LiP bit. In this way, the unknown destination MAC address of a DoS attack does not repeatedly occupy resources of switch 103.

In some embodiments, a respective member switch of network 100 (e.g., switch 103) runs a control plane with automatic configuration capabilities (e.g., based on IP or Fibre Channel (FC)) and forms a logical Ethernet switch based on the automatic configuration capabilities of the control plane. To an external end device, such as end device 112, network 100 can appear as a single Ethernet switch. Upon joining network 100 via the control plane, a respective member switch receives an automatically assigned identifier corresponding to the logical Ethernet switch. However, unlike an FC fabric, the data packets in network 100 can be encapsulated and forwarded based on another forwarding protocol. Examples of this forwarding protocol include, but are not limited to, Ethernet, TRILL, and IP. Furthermore, a respective member switch of network 100 can be associated with a group identifier, which identifies network 100 as a group of interconnected switches. If network 100 is a fabric switch, this group identifier can be a fabric identifier identifying the fabric switch.

In some embodiments, network 100 maintains a port profile for a respective virtual machine. A port profile represents Fibre Channel over Ethernet (FCoE) configuration, VLAN configuration, data center bridging (DCB) configuration, quality of service (QoS) configuration, and/or security configuration of one or more virtual machines. The MAC address of a virtual machine associates the corresponding port profile with the virtual machine. If a switch receives a packet with a source MAC address not associated with a port profile from a hypervisor or with a conflicting MAC address, the switch floods the packet in the network. Port profile management in a switch is specified in U.S. Patent Publication No. 2011/0299413, titled “Port profile management for virtual cluster switching,” the disclosure of which is incorporated herein in its entirety.

Suppose that end device 112 is in communication with end device 116 and end device 111 is in communication with end device 114. Hence, switch 103 considers the MAC addresses of end devices 114 and 116 as conversational MAC addresses. Similarly, switch 102 considers end device 112's MAC address and switch 105 considers end device 111's MAC address as conversational MAC addresses. FIG. 1B illustrates exemplary forwarding and hardware tables for supporting progressive MAC address learning, in accordance with an embodiment of the present invention. In this example, the MAC addresses of end devices 111-116 are MAC addresses 131-136, respectively.

Suppose that a respective switch of network 100 has learned, either locally or remotely, MAC addresses 131-136. Hence, forwarding tables 181, 183, and 185 of switches 103, 105, and 102, respectively, include forwarding entries comprising MAC addresses 131-136. On the other hand, hardware table 182 of switch 103 includes locally learned MAC addresses 131 and 132, and conversational MAC addresses 134 and 136 of end devices 114 and 116, respectively. Similarly, hardware table 184 of switch 105 includes locally learned MAC addresses 133 and 134, and conversational MAC address 131 of end device 111; and hardware table 186 of switch 102 includes locally learned MAC addresses 135 and 136, and conversational MAC address 132 of end device 112.

In some embodiments, progressive MAC address learning can be based on a respective hardware chip. FIG. 1C illustrates exemplary progressive MAC address learning in switch hardware, in accordance with an embodiment of the present invention. In conventional MAC address learning (i.e., source MAC address learning), if a MAC address is learned from a port of a particular hardware chip (e.g., an ASIC chip), the learned MAC address is replicated to all chips of the switch to facilitate switching decisions. Suppose that switch 103 includes chips 152, 154, and 156. An edge port on chip 152 couples end device 111, an edge port on chip 156 couples end device 112, and an inter-switch port on chip 154 couples switch 102.

In conventional MAC address learning, the MAC addresses of end devices 111 and 112 are shared among chips 152, 154, and 156. However, the number of MAC addresses learned at switch 103 can be high. To solve this problem, the hardware table in a chip, which is not in communication with a remote MAC address, does not include the remote MAC address. For example, since end device 111 is coupled to chip 152 and is in communication with end device 116, the hardware table of chip 152 includes MAC address 136. Similarly, since end device 112 is coupled to chip 156 and is in communication with end device 114, the hardware table of chip 156 includes MAC address 134.

However, the hardware table of chip 152 does not include MAC address 134, and the hardware table of chip 156 does not include MAC address 136. If an end device coupled with chip 152 starts communicating with end device 114, the layer-2 management apparatus of switch 103 creates an entry in the hardware table of chip 152 comprising MAC address 134 from forwarding table 181. This allows a switch to support progressive MAC address learning for a respective chip in the switch. In some embodiments, switch 103 adds the entry comprising MAC address 134 to the download list of chip 152.

Progressive MAC Address Learning

FIG. 2A illustrates an exemplary progressive MAC address learning process based on an ARP request, in accordance with an embodiment of the present invention. During operation, end device 112 sends an ARP request for the IP address of end device 116 (operation 202). Upon receiving the ARP request, switch 103 learns MAC address 132 of end device 112, and since MAC address 132 is a locally learned MAC address, creates corresponding entries in both forwarding table 181 and hardware table 182 (operation 204). Since the ARP request is a broadcast message, switch 103 forwards the ARP request to a respective other switch in network 100 (operation 206). In some embodiments, switch 103 can use a all-switch multicast tree in network 100 to distribute the ARP request.

Upon receiving the ARP request, switch 105 learns MAC address 132. Since MAC address 132 is learned via an inter-switch port, switch 105 creates a corresponding entry only in forwarding table 183 (operation 208). This entry can include an identifier of (or a bitmap associated with) switch 103. Switch 105 forwards the ARP request via a respective edge port (operation 210). As a result, end device 116, which is coupled to switch 105 via an edge port, receives the ARP request. Since the ARP request is for the IP address of end device 116, end device 116 creates an ARP reply comprising MAC address 136 of end device 116 (operation 212). End device 116 then sends the ARP reply to switch 105 (operation 214).

Switch 105 receives the ARP reply, learns MAC address 136, and since MAC address 136 is a locally learned MAC address, creates corresponding entries in both forwarding table 183 and hardware table 184 (operation 216). Since the ARP reply is a unicast message to end device 112, switch 105 forwards the ARP reply to switch 103 (operation 218). Switch 103 receives the ARP reply, learns MAC address 136, and since MAC address 136 is learned via an inter-switch port, creates a corresponding entry only in forwarding table 181 (operation 220). Switch 103 then forwards the ARP reply to end device 112 (operation 222).

Upon receiving the ARP reply with MAC address 136, end device 112 initiates its communication with end device 116. FIG. 2B illustrates an exemplary progressive MAC address learning process based on an initial communication from an end device, in accordance with an embodiment of the present invention. During operation, end device 112 sends its first packet for end device 116 to switch 103 (operation 252). Upon receiving the first packet, switch 103 determines that the destination MAC address of the first packet, which is MAC address 136, is not present in hardware table 182. Switch 103 then consults forwarding table 181, identifies destination MAC address 136, and creates a corresponding entry in hardware table 182 (operation 254).

Based on the entry in hardware table 182, switch 103 forwards the first packet to switch 105 (operation 256). Switch 105 forwards the first packet to end device 116 (operation 258). End device 116 then sends a response packet to switch 105 (operation 260). Upon receiving the response packet, switch 105 determines that the destination MAC address of the response packet, which is MAC address 132, is not present in hardware table 184. Switch 105 then consults forwarding table 183, identifies destination MAC address 132, and creates a corresponding entry in hardware table 184 (operation 262). Based on the entry in hardware table 184, switch 105 forwards the response packet to switch 103 (operation 264). Switch 105 forwards the response packet to end device 112 (operation 266).

Operations

FIG. 3A presents a flowchart illustrating the process of a switch learning a progressive MAC address via a local edge port, in accordance with an embodiment of the present invention. During operation, the switch receives a packet (e.g., an Ethernet frame comprising an IP packet) from an end device via a local edge port (operation 302). The switch learns the end device's MAC address (operation 304) and creates an entry in the forwarding and hardware tables comprising the learned MAC address and its associated information (operation 306). Associated information includes, but is not limited to, an egress port (e.g., from which the MAC address has been learned), a corresponding VLAN tag, and one or more flags (e.g., to indicate flooding, download to hardware table, etc).

In some embodiments, the switch shares a respective learned MAC address with other switches of the network (e.g., in a fabric switch). The switch then generates a notification message comprising the learned MAC address and its associated information (operation 308) and forwards the notification message to a respective other member switch of the network (operation 310). In some embodiments, the notification message is a control message, which carries the learned MAC address in its payload. This control message can be encapsulated in an encapsulation header (e.g., a fabric or tunnel encapsulation header). The switch can forward the notification message via an all-switch multicast tree in the network.

FIG. 3B presents a flowchart illustrating the process of a switch learning a progressive MAC address from a notification message, in accordance with an embodiment of the present invention. During operation, the switch receives a notification message comprising a MAC address learned at a remote switch (i.e., at a port in the remote switch) via a local inter-switch port (operation 352). If the notification message is encapsulated in an encapsulation header, the switch can decapsulate the encapsulation header. The switch extracts the MAC address and its associated information from the notification message (operation 354). The switch then creates an entry in the local forwarding table comprising the MAC address and its associated information extracted from the notification message (operation 356).

When a switch learns a MAC address based on progressive MAC address learning and has an entry comprising the learned MAC address in the hardware table, an aging out event (e.g., a timeout exception) can occur due to inactivity of that MAC address. For example, the MAC address may no longer be involved (e.g., as a source or destination address) in an active communication. A switch can maintain the aging out process at a switch-level or at a chip-level. When an entry comprising a conversational MAC address ages out in the hardware table, the switch removes the entry from the hardware table. The switch also unsets the flag indicating that the MAC address is a conversational MAC address in the corresponding entry in the forwarding table.

FIG. 3C presents a flowchart illustrating the process of a switch timing out a learned MAC address, in accordance with an embodiment of the present invention. During operation, the switch detects an aging event for a MAC address in the hardware table (operation 372). The switch then checks whether the MAC address is a locally learned MAC address (i.e., learned from a local edge port) (operation 374). If the MAC address is a locally learned MAC address, the switch removes the respective entries comprising the MAC address from the local forwarding and hardware tables (operation 376). The switch generates a notification message indicating the aging out for the MAC address (e.g., a control message for clearing a learned MAC address) (operation 378). The switch then forwards the notification message to a respective other member switch (operation 380).

On the other hand, if the MAC address is not a locally learned MAC address, the switch removes the entry comprising the MAC address from the local hardware table (operation 382). The switch receives a notification message indicating the aging out for the MAC address (e.g., a control message for clearing the MAC address from a remote switch) (operation 384). The switch then removes the entry comprising the MAC address from the local forwarding table (operation 386).

FIG. 4 presents a flowchart illustrating the process of a switch forwarding a packet based on learned progressive MAC addresses, in accordance with an embodiment of the present invention. During operation, the switch receives a data packet via a local edge port (operation 402) and identifies the destination MAC address of the packet (operation 404). The switch checks whether the local hardware table includes the identified MAC address (i.e., the identified MAC address has a corresponding entry in the hardware table) (operation 406). If the hardware table does not include the identified MAC address, the switch looks up (i.e., searches for) the identified MAC address in the forwarding table (operation 408) and checks whether the lookup has been successful (operation 410).

If the lookup has not been successful, switches in the network have not learned the MAC address yet. Hence, the switch creates an entry in the forwarding table comprising the identified MAC address and indicates flooding for the MAC address (operation 412). This prevents multiple exceptions for the same unknown MAC address. The switch then encapsulates the packet in an encapsulation header and floods the encapsulated packet in the network (operation 414). It should be noted that the switch also floods the received packet via its local edge ports. In some embodiments, to flood the packet in the network, the switch assigns a multi-destination identifier as the egress switch identifier of the encapsulation header. The switch then forwards the encapsulated packet based on the multi-destination identifier. For example, the switch may forward the packet via a multicast tree which includes a respective switch of the network.

If the lookup has been successful, the switch has found the MAC address in the forwarding table. The switch then obtains the entry comprising the identified MAC address from the forwarding table (operation 416) and creates a corresponding entry in the hardware table (operation 418), as described in conjunction with FIG. 1B. If the hardware table includes the identified MAC address (operation 406) or the switch has created a entry comprising the identified MAC address in the hardware table (operation 418), the switch identifies an egress switch identifier associated with the identified MAC address in the hardware table (operation 420).

The switch then encapsulates the packet in an encapsulation header and assigns the egress switch identifier in the encapsulation header (operation 422). The switch forwards the encapsulated packet via a local inter-switch port associated with the egress switch identifier (operation 424). It should be noted that if the destination MAC address is a local MAC address, the switch can switch the received packet to that destination MAC address without encapsulating the packet based on layer-2 frame switching.

Link Aggregation Group (LAG)

FIG. 5A illustrates an exemplary virtual link aggregation group with support for progressive MAC address learning, in accordance with an embodiment of the present invention. In this example, switches 103, 104, and 105 couple an end device 512 via a virtual link aggregation group (VLAG) 520. Switches 103, 104, and 105 are configured to operate in a special “trunked” mode for end device 512. End device 512 views switches 103, 104, and 105 as a common virtual switch, with a corresponding virtual address. End device 512 is considered to be logically coupled to the virtual switch via logical links.

The virtual switch is considered to be logically coupled to switches 103, 104, and 105, optionally with zero-cost links. While forwarding data frames from end device 512, switches 104 and 105 mark the data frames with the virtual switch's address as their source address. As a result, other switches in network 100 can learn that end device 512 is reachable via the virtual switch. Hence, multi-pathing can be achieved when other switches choose to send data frames to the virtual switch (which is marked as the egress switch in the frames) via switches 103, 104, and 105.

In the following description, switches which participate in virtual link aggregation group 520 are referred to as “partner switches.” Since the three partner switches function as a single logical switch, the MAC address reachability learned by a respective switch is shared with the other partner switch. For example, during normal operation, end device 512 may choose to send its outgoing data frames only via the link to switch 105. As a result, only switch 105 would learn end device 512's MAC address 532. This information is then shared by switch 105 with switches 103 and 104.

However, a hardware table in a switch (or in a chip) includes a conversational MAC address if that switch (or chip) has actively participated in a communication. During operation, end device 512 communicates with end device 116. End device 512 sends a packet 542 to end device 116. Packet 542 includes MAC address 532 as the source MAC address and MAC address 136 as the destination MAC address. Switch 105 receives packet 542 and identifies destination MAC address 136 as a conversational MAC address. Switch 105 then obtains MAC address 136 from the local forwarding table and creates an entry comprising MAC address 136 in the local hardware table. On the other hand, since switch 104 has not received a packet for this communication, switch 104's hardware table does not include MAC address 136.

Suppose that end device 512 is coupled with chips 152 and 156 of switch 103 and the links coupling these chips participate in virtual link aggregation group 520. These links form a link aggregation group (LAG) between end device 512 and switch 103. End device 512 sends a packet 544 to end device 116. Packet 544 includes MAC address 532 as the source MAC address and MAC address 136 as the destination MAC address. Chip 152 receives packet 544, identifies MAC address 532 as a conversational MAC address, and includes MAC address 136 in the hardware table of chip 152. On the other hand, since chip 156 has not received a packet for this communication, chip 156's hardware table does not include MAC address 136, as described in conjunction with FIG. 1C.

FIG. 5B illustrates an exemplary high-availability for a virtual link aggregation group with support for progressive MAC address learning, in accordance with an embodiment of the present invention. Suppose that a failure 560, which can be a link failure, a node failure, or both, makes switch 105 unreachable from end device 512. As a result, switch 105 becomes unavailable in virtual link aggregation group 520. End device 512 then stops forwarding traffic via the port(s) coupling switch 105 and diverts that traffic to other active ports in virtual link aggregation group 520. If the diverted traffic flows to a chip or switch in which MAC address 136 is already in the hardware table, communication between end devices 512 and 116 continues via that chip or switch.

On the other hand, the diverted traffic can flow to a new chip or switch in which MAC address 136 has not been in the hardware table. The chip or node then looks up in the local forwarding table the entry comprising MAC address 136 and creates a corresponding entry in the local hardware table. For example, end device 512 can send a packet 548 to switch 104. Packet 548 includes MAC address 532 as the source MAC address and MAC address 136 as the destination MAC address. Switch 104 receives packet 542 and identifies destination MAC address 136 as a conversational MAC address. Switch 104 then obtains the entry comprising MAC address 136 from the local forwarding table and creates a corresponding entry comprising MAC address 136 in the local hardware table.

Exemplary Switch

FIG. 6 illustrates an exemplary switch with support for progressive MAC address learning, in accordance with an embodiment of the present invention. In this example, a switch 600 includes a number of communication ports 602, a packet processor 610, a layer-2 management apparatus 630, a hardware management apparatus 632, and a storage device 650. In some embodiments, packet processor 610 adds an encapsulation header to a packet. In some embodiments, switch 600 includes a network management apparatus 620, which maintains a membership in a network of interconnected switches (e.g., in a fabric switch). A respective switch of the network is associated with a group identifier identifying the network.

In some embodiments, the network is a fabric switch. Switch 600 maintains a configuration database in storage device 650 that maintains the configuration state of a respective switch within the fabric switch. Network management apparatus 620 maintains the state of the fabric switch, which is used to join other switches. Under such a scenario, communication ports 602 can include inter-switch communication channels for communication within a fabric switch. This inter-switch communication channel can be implemented via a regular communication port and based on any open or proprietary format (e.g., a TRILL or IP protocol).

Storage device 650 stores a forwarding table, which includes an entry comprising a MAC address and an egress port for the MAC address. Hardware management apparatus 632 determines whether a destination MAC address of a frame is present in a hardware table in memory of switch 600. Layer-2 management apparatus 630, in response to a determination that the destination MAC address is not present in the hardware table, looks up a first entry comprising the destination MAC address in the forwarding table, and creates a second entry comprising the destination MAC address in the hardware table based on the first entry.

In some embodiments, switch 600 also includes a learning management apparatus 640, which learns a first MAC address from a local edge port. Layer-2 management apparatus 630 then creates respective entries in the forwarding and hardware tables comprising the learned MAC address. Learning management apparatus 640 can also extract a remote MAC address learned at a port of a second switch from a payload of a notification message. Layer-2 management apparatus 630 then creates an entry in the forwarding table comprising the remote MAC address.

Layer-2 management apparatus 630, in response to a determination that the destination MAC address is present in the hardware table, identifies an egress port for the frame from an entry comprising the destination MAC address in the hardware table. If the frame has been received via a first port (e.g., one of the communication ports 602) in a first chip of switch 600, layer-2 management apparatus 630 is precluded from creating an entry comprising the destination MAC address in a hardware table of a second chip of switch 600.

Layer-2 management apparatus 630 clears the second entry in response to identifying an aging out event for the destination MAC address. However, the first entry remains in the forwarding table. On the other hand, layer-2 management apparatus 630 clears the first entry in response to identifying an aging out event of a remote switch in a notification message. Layer-2 management apparatus 630, in response to determining a failure to the lookup, floods the frame in the network.

Note that the above-mentioned modules can be implemented in hardware as well as in software. In one embodiment, these modules can be embodied in computer-executable instructions stored in a memory which is coupled to one or more processors in switch 600. When executed, these instructions cause the processor(s) to perform the aforementioned functions.

In summary, embodiments of the present invention provide a switch and a method for scalable MAC address learning. In one embodiment, the switch includes a storage device, a hardware management apparatus, and a layer-2 management apparatus. The storage device stores a forwarding table, which includes an entry comprising a MAC address and an egress port for the MAC address. The hardware management apparatus determines whether a destination MAC address of a frame is present in a hardware table in memory of the switch. The layer-2 management apparatus, in response to a determination that the destination MAC address is not present in the hardware table, looks up a first entry comprising the destination MAC address in the forwarding table, and creates a second entry comprising the destination MAC address in the hardware table based on the first entry.

The methods and processes described herein can be embodied as code and/or data, which can be stored in a computer-readable non-transitory storage medium. When a computer system reads and executes the code and/or data stored on the computer-readable non-transitory storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the medium.

The methods and processes described herein can be executed by and/or included in hardware modules or apparatus. These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.

The foregoing descriptions of embodiments of the present invention have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit this disclosure. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. The scope of the present invention is defined by the appended claims.

Claims

1. A switch, comprising: a storage device configured to store a first forwarding data structure, wherein a respective entry in the first forwarding data structure includes a first media access control (MAC) address and an egress port for the first MAC address;hardware management circuitry configured to determine whether a destination MAC address of a data frame is present in a second forwarding data structure in a content-addressable memory of the switch; andlayer-2 management circuitry configured to: in response to determining that the destination MAC address is not present in the second forwarding data structure, identify a first entry comprising the destination MAC address in the first forwarding data structure; andcreate a second entry comprising the destination MAC address in the second forwarding data structure based on the first entry; anddetermine a first egress port for the data frame based on the second entry in the second forwarding data structure.

2. The switch of claim 1, further comprising learning management circuitry configured to learn a second MAC address from an edge port of the switch; and wherein the layer-2 management circuitry is further configured to create respective entries comprising the learned MAC address in the first forwarding data structure and the second forwarding data structure.

3. The switch of claim 1, further comprising learning management circuitry configured to extract a third MAC address learned at a port of a second switch from a payload of a notification message; and wherein the layer-2 management circuitry is further configured to create an entry in the first forwarding data structure comprising the third MAC address.

4. The switch of claim 1, wherein the data frame has been received via a first port in a first chip of the switch; and wherein the layer-2 management circuitry is precluded from creating an entry comprising the destination MAC address in a third forwarding data structure of a second chip of the switch.

5. The switch of claim 1, wherein the layer-2 management circuitry is further configured to clear the second entry in response to identifying an aging out event for the destination MAC address, wherein the first entry remains in the first forwarding data structure.

6. The switch of claim 5, wherein the layer-2 management circuitry is further configured to clear the first entry in response to identifying an aging out event of the destination MAC address in a notification message from a remote switch, wherein the destination MAC address has been learned at a port of the remote switch.

7. The switch of claim 1, wherein the switch and a remote switch participate in a virtual link aggregation group for an end device, wherein a plurality of links of the virtual link aggregation group operate as a single link; and wherein the second forwarding data structure of the switch includes at least one different MAC address than a remote forwarding data structure of the remote switch.

8. The switch of claim 1, wherein the layer-2 management circuitry is further configured to, in response to not identifying the destination MAC address in the first forwarding table, flood the data frame in a network of interconnected switches.

9. The switch of claim 8, wherein the network of interconnected switches is identified by a fabric identifier, and wherein a respective switch of the network of interconnected switches is associated with the fabric identifier.

10. A computer-executable method, comprising: storing a first forwarding data structure in a storage device of a switch, wherein a respective entry in the first forwarding data structure includes a media access control (MAC) address and an egress port for the first MAC address;determining whether a destination MAC address of a data frame is present in a second forwarding data structure in a content-addressable memory of the switch;in response to determining that the destination MAC address is not present in the second forwarding data structure, identifying a first entry comprising the destination MAC address in the first forwarding data structure;creating a second entry comprising the destination MAC address in the second forwarding data structure based on the first entry; anddetermining a first egress port for the data frame based on the second entry in the second forwarding data structure.

11. The method of claim 10, further comprising: learning a second MAC address from an edge port of the switch; andcreating respective entries comprising the learned MAC address in the first forwarding data structure and the second forwarding data structure.

12. The method of claim 10, further comprising: extracting a third MAC address learned at a port of a second switch from a payload of a notification message; andcreating an entry in the first forwarding data structure comprising the third MAC address.

13. The method of claim 10, wherein the data frame has been received via a first port in a first chip of the switch; and wherein the switch is precluded from creating an entry comprising the destination MAC address in a third forwarding data structure of a second chip of the switch.

14. The method of claim 10, further comprising clearing the second entry in response to identifying an aging out event for the destination MAC address, wherein the first entry remains in the first forwarding data structure.

15. The method of claim 14, further comprising clearing the first entry in response to identifying an aging out event of the destination MAC address in a notification message from a remote switch, wherein the destination MAC address has been learned at a port of the remote switch.

16. The method of claim 10, wherein the switch and a remote switch participate in a virtual link aggregation group for an end device, wherein a plurality of links of the virtual link aggregation group operate as a single link; and wherein the second forwarding data structure of the switch includes at least one different MAC address than a remote forwarding data structure of the remote switch.

17. The method of claim 10, further comprising, in response to not identifying the destination MAC address in the first forwarding table, flooding the data frame in a network of interconnected switches.

18. The method of claim 10, wherein the network of interconnected switches is identified by a fabric identifier, and wherein a respective switch of the network of interconnected switches is associated with the fabric identifier.

19. The method of claim 10, wherein the first entry further includes a bitmap, wherein a set bit in the bitmap indicates a switch from which the first MAC address has been learned.

20. A computing system, comprising: a processor; anda memory storing instructions that when executed by the processor cause the system to perform a method, the method comprising: storing a forwarding data structure in a storage device of a switch, wherein a respective entry in the forwarding data structure includes a media access control (MAC) address and an egress port for the MAC address;determining whether a destination MAC address of a data frame is present in a second forwarding data structure in a content-addressable memory of the switch;in response to determining that the destination MAC address is not present in the second forwarding data structure, identifying a first entry comprising the destination MAC address in the first forwarding data structure; andcreating a second entry comprising the destination MAC address in the second forwarding data structure based on the first entry; anddetermining a first egress port for the data frame based on the second entry in the second forwarding data structure.

21. The switch of claim 1, wherein the first entry further includes a bitmap, wherein a set bit in the bitmap indicates a switch from which the first MAC address has been learned.