Patent Application
20160197831
This application relates to enhancements for a SDN (Software Defined Network) system to enable hash-routing. More specifically, the application relates to Software Defined Networking (e.g., OpenFlow) and the deployment and usage of Distributed Hash Tables, for example, as part of Information Centric Networking (ICN).
Multiple ICN solutions have been designed, including DONA, CCN, PubSub, and Netlnf, which are reviewed in [1], as well as CONET [2].
The present application pertains to methods and apparatus for enhancing SDNs, such as OpenFlow networks, to facilitate the deployment and usage of Distributed Hash Tables (DHTs), for example as part of Information Centric Networking (ICN).
According to a first aspect, the present invention pertains to methods and apparatus for creating a Distributed Hash Table (DHT) among a plurality of DHT nodes in a Software Defined Networking (SDN) domain comprising a DHT Control Plane (DHTCP) module adapted to manage a plurality of DHT nodes to maintain a DHT, the DHTCP comprising a processor adapted to receive messages from DHT nodes indicating status of the DHT nodes, including joining and leaving the DHT, determine a range-based distribution among the DHT nodes of (key, value) pairs of a DHT as a function of the keys of the (key, value) pairs, and send configuration messages to the DHT nodes for configuring each DHT node to store at least one range of keys corresponding to ((key, value)) pairs of said DHT.
According to another aspect, the present application pertains to a method and apparatus for handling Distributed Hash Table (DHT) routing requests in a Software Defined Networking (SDN) domain comprising a plurality of SDN switches interconnecting a plurality of DHT nodes, each DHT node containing a portion of a DHT, the method and/or apparatus comprising receiving at a first one of the DHT nodes a Publish request corresponding to a content object, the Publish request including a (key, value) pair, where the key corresponds to a content ID and the value is the location where the content object is stored, if the key in the Publish request is in a portion of the DHT not contained at the first DHT node, the first DHT node forwarding the Publish request to a first one of the SDN switches, the first SDN switch including a forwarding table in which ranges of keys of (key, value) pairs are mapped to DHT nodes, and the first SDN switch matching the key from the Publish request with one of the key ranges in the forwarding table and forwarding the Publish request toward a second DHT node, the second DHT node being the DHT node to which the key of the ((key, value)) pair in the Publish request maps, and the second DHT node receiving the Publish request and storing the (key, value) pair found in the Publish request in its DHT portion.
According to another aspect, the present application pertains to a method and apparatus for processing Distributed Hash Table (DHT) routing requests in a Software Defined Networking (SDN) domain comprising a plurality of SDN switches interconnecting a plurality of DHT nodes, each DHT node containing a portion of a DHT, the method and/or apparatus comprising receiving at a first one of the DHT nodes a Subscribe request for a content object, the Subscribe request including a key, where the key corresponds to a content ID, if the key in the Subscribe request does not match a key in the portion of the DHT of the first DHT node, the first DHT node forwarding the Subscribe request to a first one of the SDN switches, and the first SDN switch including a forwarding table in which ranges of keys corresponding to (key, value) pairs are mapped to DHT nodes, the first SDN switch matching the key from the Subscribe request with one of the key ranges in the forwarding table, and forwarding the Subscribe request toward a second DHT node, the second DHT node being the DHT node to which the key of the ((key, value)) pair in the Subscribe request maps.
According to another aspect, the present application pertains to a method and apparatus implemented in a Software Defined Networking (SDN) switch/router comprising transmitting to an SDN controller a message including Hash Routing Control (HRC) capabilities of the switch/router, and receiving from the SDN controller a message including at least one hash routing descriptor indicating how the switch/router is to process routing requests.
According to another aspect, the present application pertains to a method and apparatus implemented in a Software Defined Networking (SDN) switch/router comprising maintaining a flow table for routing data packets in a SDN network, receiving from an SDN controller a flow table modification message defining a change to the flow table maintained at the switch/router, wherein the flow entry modification message further includes at least one of (1) an information element (IE) specifying a method for extracting hash function inputs from the data packets, (2) an IE specifying how a hash function is calculated from the hash function inputs, and (3) an IE specifying a range of hash function outputs to which the flow entry applies, and updating the flow table using the IEs in the flow entry modification message.
According to another aspect, the present application pertains to a method and apparatus implemented in a Software Defined Networking (SDN) controller for configuring an SDN network for Hash Routing Control (HRC) comprising transmitting a features request message to a SDN switch/router requesting information disclosing hash routing control (HRC) features of the switch/router, and receiving in response to the features request a features reply message, the features response message including an HRC information element (IE) disclosing the HRC capabilities of the switch/router.
According to another aspect, the present application pertains to a method and apparatus implemented in a Software Defined Networking (SDN) switch/router comprising maintaining a first flow table for routing data packets in a SDN network, maintaining a second flow table for routing the data packets according to a Hash Routing control (HRC), receiving from an SDN controller a flow table modification message defining a change to one of the first and second flow tables, the flow table modification message identifying a hash function and identifying a condition applicable to a hash value calculated using the hash function, and updating the first or second flow table according to the flow entry modification message.
According to another aspect, the present application pertains to a method and apparatus method implemented in a Software Defined Networking (SDN) controller comprising transmitting a flow table modification message to a SDN switch/router, wherein the flow table modification message defines a change to a flow table, identifies a hash function, and identifies a condition applicable to a hash value calculated using the hash function.
A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings wherein:
The ICN design in [3] is based on a Scalable Multi-level Virtual Distributed Hash Table (SMVDHT). SMVDHT enables name resolution and routing. Every routing area, domain, and Autonomous System (AS) deploys a DHT formed with content routers. These DHTs are connected in a hierarchical fashion to form the SMVDHT. SMVDHT is used to store a storage location associated with a given content ID. This information is advertised to upper layer DHTs in an aggregated form. In content requests and responses, an Object ID (OID) is present as a shim layer between the IP layer and transport headers. When a content request reaches a content router, the content router queries the SMVDHT for the location of the object (or the next hop toward this location). Each DHT may individually be inside a single domain (which may or may not be an SDN-controlled domain).
NetInf[4] supports a range of name-based and name-resolution-based object retrieval methods. Name resolution systems that are considered in the Netlnf architecture include Multilevel DHT (MDHT) [5], which is composed of several entangled DHTs. A given DHT node may belong to one or several DHTs which are handling different scopes, ranging from local area networks to a global internetwork scope. Some of these DHTs typically are inside a single domain, while some other of these DHTs typically are more global and span several domains.
2.1 General Overview
Structured overlay networks place assumptions on how nodes are organized in a distributed environment and emphasize decentralization, scalability, and fault tolerance. They are based on the notion of a semantic-free index identifying nodes (peers) as well as data objects (e.g., the index for data objects may be obtained from hashing that object). Keys and nodes share the same identifier space, which makes it possible for the distributed system to map a key to a particular node where the (key, value) pair (i.e., the previously mentioned key and its associated data object) is to be stored. Each peer maintains a routing table (e.g., node IDs and IP addresses) of a set of neighbors, which enables the routing of insertion, lookup, and removal messages to their final destination in one or more hops.
Structured overlay networks may be deployed inside a single administrative domain or in a wide area network. Cluster-type deployments can focus on limiting the number of hops to reach the destination node (in one extreme case there may be 1-hop DHTs, for example), while systems designed for wide area network deployment need to balance hop count with other characteristics like the size of the routing table, network diameter (i.e., the “longest shortest-path” between two nodes in the network), and ability to cope with change (e.g., churn). As presented in [13] scalable distributed data structures enabling structure overlay networking in single domain environments include not only DHTs but also other structures, such as Linear Hashing based structures, and tree based structures. Nevertheless, their application prevalent in cluster type applications while DHTs have a wider range of applications. As a result, DHTs were included in the design of several ICN systems. While the present specification focuses on DHTs, other hash-based distributed systems also can benefit from the innovations disclosed herein.
DHTs are decentralized distributed systems providing a lookup service for (key, value) pairs. Every node participating in the DHT can retrieve a value associated with a key (key, value) storage. Mapping is distributed among the participating nodes. DHTs typically are resistant, i.e. react well, to churn (nodes leaving/joining the system). As noted in [10], a DHT can be classified as a structured peer-to-peer system (i.e., it enables efficient discovery of data by organizing peers in relation to the key space), while other peer-to-peer systems, including such systems as the Bittorrent file sharing service, typically are unstructured (i.e., there is no a priori relation between data and peers storing this data). Since DHTs rely on associating data with storage peers, one fundamental building block of DHTs is consistent hashing [12], which is a technique that enables the addition or removal of an element without significantly changing the mapping of keys to elements. The technique requires only K/n keys to be remapped on average in response to a node joining/leaving the system, where K is the number of keys and n is the number of nodes. Therefore, churn does not lead to massive transfer of data between peers.
Design of Distributed Hash Tables is an active field of research and development. One survey [10] describes six major DHT designs (CAN, Chord, Tapestry, Pastry, Kademlia, and Viceroy). The Wikipedia DHT page [11] provides a high level description of DHTs in general and lists applications employing DHTs. We note that DHTs have reached a level of maturity such that they can be used in live, large scale products (e.g., Amazon's Dynamo [6], Bittorrent's Search Functionality [9]). As an example of implementation including advanced features, OpenDHT [6] was a free, open DHT service linked to a research project. It is discontinued, but its code base [8] (built using concept from the Pastry DHT) is still available. Notably, this implementation makes it possible to add several values associated with the same key (then a request for the key will return all values). As an example of an advanced feature, a secret (e.g., a hash value of a secret pass phrase) is provided along with a key/value pair in a PUT operation; later a REMOVE operation specifying key, value (or hash of value) and secret will remove the (key, value) pair.
A one-hop DHT was developed as part of the SEATTLE system [14] to enable a scalable Ethernet architecture, more efficient than Ethernet bridging. SEATTLE can make use of a multi-level one-hop DHT, where a new object can be inserted in a regional and/or backbone level DHT. As mentioned earlier, some proposed ICN designs [3] [4] use such a multi-level (and possibly one-hop) DHT.
2.2 Model of DHT Routing
Single-hop DHTs rely on the fact that the whole system routing table is known by all participant nodes. On the other hand, in a multiple hop overlay network, a node routing table holds many of that node's close neighbors (close in the sense that their node IDs are close within the given network geometry), and relatively fewer of that node's distant neighbors. One typical example of geometry used in DHTs is a ring, i.e., node IDs and keys are represented on a one dimensional cyclic space (e.g., used in Chord DHT); other examples of geometries are d-dimensional tori (e.g., used in CAN) and XOR (e.g., used in Kademlia).
Some key elements of DHT routing are:
Today, the software that controls forwarding rules of switches and routers is typically implemented directly in the device by the vendor. This makes it difficult to innovate in this domain because operators cannot quickly deploy new applications and services, researchers cannot experiment with new protocols in real world network settings, and it can be difficult for vendors to adapt their product fast enough to meet evolving market requirement.
SDN is a new approach to networking [15]. In SDN, control and data planes are separated, and communication between them is enabled through a SDN control protocol. The most widespread implementation of this protocol is OpenFlow [22] [23]. The primary use case for SDN today is within data centers. Nevertheless, the range of application for SDN is much wider. With SDN, operators and Researchers can manage network resources through SDN programs running on control plane entities. Today, multiple vendors implement OpenFlow devices (e.g., Cisco, HP, Juniper) which are deployed in data centers [24].
With reference to
The OpenFlow specifications do not mention that matching rules can target ranges of values (as illustrated in the table below from the OpenFlow 1.1 specifications).
3.1 Detailed SDN Summary
SDN is a key enabler of true network virtualization. SDN makes it possible for each individual hardware switch to be part of multiple Layer 2 and Layer 3 virtual networks and have each virtual network managed by an independent network controller.
Realization of the SDN paradigm requires several components to be in place:
If one were to draw an analogy to computer programming, the configuration protocol is assembly language, the “programming language/environment” is Java/Visual Basic/etc., and the higher-level tools are Web-based development environments that allow quick and easy development of web services.
The remainder of this section discusses the state of the art in developing these components.
3.1.1 OpenFlow, a SDN Enabler
OpenFlow is the protocol that provides access to computer hardware and a standardized way of configuring such hardware. Originally developed at Stanford University [22], it has become the de-facto single standard in the SDN community. This makes OpenFlow immensely important, e.g., it is as if in the early days of microprocessors (late 1970's /early 1980's) the whole world adopted a single Assembly Language and all the processors had to be programmable in that single Assembly Language.
OpenFlow is a communications protocol that gives access to the data plane of a network switch or router over the network [17]. It allows the path of network packets through the network of switches to be determined by software running on multiple routers. This separation of the control from the forwarding allows for more sophisticated traffic management than is feasible using access control lists (ACLs) and routing protocols.
OpenFlow enables networks to evolve and enables flow segregation for routers, switches, virtual switches, and access point. It provides an open protocol to program the flow-tables stored in most switches and routers. It is based on a standardized interface for adding and removing flow entries and is considered a technology-enabling network virtualization.
Switching (forwarding by destination MAC) and routing (i.e., forwarding by destination IP) are usually performed through FIB (Forwarding Information Base, also known as Forwarding Table) which are optimized for destination addresses and RIB (Routing Information Base, also known as Routing Table). RIB/FIB are populated and updated by hop-to-hop and fully distributed routing protocol (e.g., OSPF) or Border Gateway Protocol (BGP).
The idea behind OpenFlow is to introduce an external entity (the controller) to update RIB and FIB on routers and switches, allowing the definition of a fine-grained Flow Forwarding function (more than just forwarding packets based on MAC or IP addresses) [18]
3.1.2 SDN Northbound Interfaces
The term SDN Northbound interface relates to the Application Programming Interface (API) used by applications to interact with the SDN controllers. As presented in [15], one can analogize the SDN stack to an Operating System stack. In the text below, we review the various types of ways end users or developers can interact with the SDN controllers and analogize the SDN stack to an Operating System via
Northbound API is different from, and enabled by, the SDN programmable interface. Typically, the programmable interface is designed to enable writing a range of applications, while Northbound APIs tend to be offered on top of one of those more specialized applications. At this time, Northbound APIs are defined on a per-SDN project basis. These are early attempts, there is no coordinated effort to define this type of interface at this time, and capabilities vary greatly between implementations.
One primary interface offered by OpenFlow controllers in general (at least Open Source ones) is a programmatic interface, that enables developers to write applications. Some applications are typically developed along with the controller, either as an example or as full blown projects. For example, Floodlight (open source controller initiated) has a firewall application and a port down reconciliation application, Trema has a sliceable switch and a flow manager application. There is a fine line between these applications and other internal software components, e.g., Floodlight has a “learning switch” component, but this is an application in Maestro.
Those components or applications may be designed to offer a northbound interface (e.g., using REST). Floodlight offers a Northbound REST API that can be used for operations such as getting statistics, listing switches, getting the shortest path between two switches, and proactively inserting flows (reactive insertion is not supported). NOX/PDX (open source controller initiated by Nicira) provides a similar web service interface. Trema's sliceable switch application provides a REST interface.
One primary usage of the northbound interface is the integration with a cloud stack, namely OpenStack in the case of Floodlight and Ryu (another open source Controller initiated by NTT). This makes it possible for a cloud provider to interconnect safely the instances of a single tenant together.
Another usage of the northbound interface is to provide a user interface (e.g., browser based for example for Floodlight and Nox/Pox). More generally, a northbound interface can enable the development of over-the-top applications making use of OpenFlow—beyond the already given examples of a cloud stack or a user interface.
Reference [16] presents an attempt to implement content request routing using SDN.
In [25], the authors implemented the core network function of PubSub (PURSUIT's ICN architecture) using SDN (OpenFlow), which included mapping the concept of flows into ICN. In that design built upon LIPSIN, forwarding identifiers (FIs) are independent from Content ID in order to enable reusing the same FI for several deliveries between the same hosts. The SDN controller sets forwarding actions in switches, in entries keyed with the FI.
In [26], SDN (OpenFlow) is used to implement ICN following the CONET design. In that case, the key used for forwarding entries in OF switches is a tag with a one-to-one mapping to content ID. The mapping is performed by a Name Resolution System (NRS) when an interest packet (i.e., a content request) enters the CONET network through a border router. This mapping expires after use. The controller hosts the NRS and sets forwarding rules in OF switches using the tag. From the experimentation standpoint, CONET CSS is deployed under OFELIA (OpenFlow 1.0 based network) by mapping the content name into Transmission Control Protocol (TCP) source and destination ports. Flow Tables are modified in a reactive mode to get general processing rules and in proactive mode in the event of new content cached.
In papers from WinLab/Huawei [29] [30], SDN is used to enable content centric caching for HTTP traffic in an SDN domain. Enhancements proposed to OpenFlow include (a) switch capabilities announcement, (b) a new control message asking a switch to parse HTTP metadata, (c) a new message from the switch to the controller providing HTTP metadata such as content length, and (d) new behavior in the controller to process such an incoming message from the switch. Flows themselves are unmodified OF IP flows.
Work performed in Seoul National University [33] let the SDN controller perform the mapping between a content name and a private IP address where the content is stored.
In [31] a SDN controller estimates content popularity and controls forwarding rules to optimize an ICN network built using CCNx. This system uses OpenFlow to control IP flows in OF Switches, and CCNx is run as an overlay over IP. Hashes of content name are used as keys to OF flows.
4.1 Issues in Combining ICN and SDN
The ICN design presented in aforementioned reference [3], which is used herein as an exemplary base system, relies on an underlying transport protocol (e.g., such as provided by today's TCP/IP stack) to enable the DHT-based content location resolution and later to transport content requests and responses. This design requires support from the internal routing protocol (such as OSPF) to help form the DHT. Moreover, it also requires all content routers to be part of a DHT network, and therefore to implement the full DHT stack. The development of SDN (including the OpenFlow protocol) enables centralization of traffic control inside a domain.
The designs described in the present document leverage SDN/OpenFlow and enhance it as needed to support more centralized control of DHT lookup and content transport inside a domain. This work has a direct application in certain ICN designs, but can also be useful more generally for any DHT-based systems. Prior work combining ICN and SDN consisted in having the controller set up flows using a key associated with a particular content request, or possibly several related content requests (e.g., in [25] the flow can be associated with several content requests/responses between two nodes). The key itself can be a forwarding ID calculated by a topology layer in [25] a tag derived from the content ID in [26], a hash of content name in [31], or an existing key used today for IP flows. In none of the reviewed cases is SDN involved in facilitating DHT based lookup. Moreover, in many cases the SDN controller is involved on a per-content request basis, which leads to a potential bottleneck, while the present work aims to limit SDN control traffic to control plane operations only (i.e., individual content requests or publications should not require involvement of the SDN Controller).
Therefore, this specification describes how SDN can be used to facilitate the operation of DHT-based ICN designs (and more generally DHT-based systems), especially in term of scalability and network efficiency.
It also generalizes these concepts by describing in more detail a SDN mechanism that can be used to control hash routing, which can then be used by upper layer SDN applications such as DHT or ICN protocol stacks.
This specification makes the usual distinction between control plane and data plane. Control plane functions include, for example, system configuration, management and control. These are typically performed infrequently. On the other hand, data plane functions include, for example, the forwarding of data, and are therefore typically invoked frequently, e.g. on each arriving packet, or on each arriving data object. Control plane may sometimes be referred to as the “slow path”. The data plane may sometimes be referred to as “fast path” or “forwarding plane” in the literature.
One aspect of the present invention is moving the routing aspect of DHTs down to the network fabric, and more generally to splitting the DHT stack functionalities into control plane and data plane. Control plane features include joining, leaving, and maintaining DHT node status in the DHT network, which ultimately result in building and maintaining the routing table. Control plan features also include migrating (key, value) pairs from one node to another in order to support join/leave operations. Data plane features include setting a new (key, value) pair and associating a value with a particular key. Control plane functions can be efficiently implemented in a centralized entity, for example by an enhanced SDN controller. The DHT routing table, which is currently built and maintained by every DHT node, can be effectively built by the enhanced SDN controller and programmed into the network (e.g., the network switches) making use of a new key-range forwarding SDN capability. This effectively programs the DHT routing table inside the network, instead of inside every DHT node.
The first described embodiment of this design is discussed in detail below in section 6.1 and illustrated in
At first glance, it may seem counter-productive to centralize the control plane of a DHT because: (1) this can introduce a single point of failure, which DHTs are designed to avoid and (2) entirely centralizing the DHT into a single database is more intuitive. However, as to the first point, this invention is primarily aimed at SDN-controlled networks, and SDN domains already separate network control from data plane, and already have to deal with failure of the controller, for example, using redundancy schemes. Therefore, this design builds upon this infrastructure and benefits from the failsafe measures that are inherent to deploy SDN. Moreover, this design does not involve the SDN controller for every content request or DHT operation, but only for significant changes in the DHT (e.g., when new nodes join or leave the DHT), which reduces the impact of a temporary failure of the SDN controller, especially in cases where the DHT is made of stable nodes, and where multiple hash functions can be used concurrently to implement redundancy within the DHT. As to the second point (the alternative of replacing the DHT with a fully centralized solution), using a DHT for data plane avoids the cost of deploying and maintaining a single database, and dimensioning/designing the network for centralized access to that database. Finally, another question is what is the gain offered by using SDN to control a DHT, as opposed to single hop DHT, in which every node includes the whole DHT routing table. The advantages of the SDN-based solution: (1) a centralized control can be easier to manage, upgrade, and debug, (2) it can ensure that the state of all DHT nodes is consistent at all time and can react quickly to a node failure, (3) it simplifies the implementation in the DHT node, (4) it enables nodes which are not part of the DHT to interact with the DHT without passing through a DHT node as a gateway (therefore, no need to configure the gateway or to discover it).
Section 7 below describes the enhanced SDN system in more detail. In particular, it discusses the low-level key-range routing control SDN discussed in section 6 more generally as a more flexible Hash-Routing control (HRC) SDN feature. In particular, the exemplary OpenFlow signaling is discussed more generally, i.e., not only matching a specific field and use range routing on this field, but also calculating a hash-value on a specific field, and applying range-based routing on this value, which effectively is a general form of hash-routing. SDN control of Hash-Routing includes a northbound interface, SDN control logic, a Southbound (e.g., OpenFlow) interface update, and logic within the SDN router/switch forwarding engine.
Section 7 further describes SDN applications making use of this hash-routing control feature, especially describing their own northbound API, and how they translate northbound API operations into hash-routing south-bound API calls.
Described herein are designs to implement an ICN Name Resolution System (NRS), or other structured peer-to-peer network, inside a domain. General aspects of this design include:
6.1 Enabling DHT-Based ICN Systems using SDN
A first embodiment pertains to ICN system such as [3] or [4], relying on one or more (possibly hierarchical) single-domain DHTs to map a content ID to a locator. Support for multiple domain DHT-based resolution is not considered in this section in order to focus on the core mechanisms, but is discussed in the third embodiment in section 6.3. This first embodiment can therefore be extended to an ICN system making use of such multi-domain DHT.
In this embodiment, as illustrated in
This method can be used to implement an NRS-based ICN system (such as the one described in [3] efficiently in a SDN domain (i.e., content routers also are DHT nodes and form the NRS). The control interfaces between the controller and the overlay nodes may be indirect, e.g., may pass through an overlay control function which uses a northbound interface of the SDN controller. Also, the requester nodes do not need to know the DHT routing table, and, thus, even nodes which are not part of the DHT can request a (key, value) pair from this DHT (e.g., in [4], a completely different set of nodes may be Content Routers and NRS DHT nodes). As a related advantage, the Name Resolution System can be implemented independently from the rest of the ICN system, even using another technology than DHT, effectively hiding the storage behind a DHT ‘get/set’ network API, e.g., for small domains, all name-location (key, value) pairs may be stored in a single node.
6.1.1 Single-Domain DHT-Based NRS
In a single-hop DHT, every node needs to maintain the whole routing table, i.e., at least one entry per node present in the DHT network. In a multi-hop DHT, nodes maintain a subset of this routing table. An algorithm specific to the DHT implementation is run by a sender node wishing to send a message related to a given key (e.g., a get or a publish message). This algorithm correlates the key with a particular node which is responsible for storing the related (key, value) pair. As mentioned above, this algorithm depends on the particular DHT design. In this invention, the whole DHT routing table as well as the key-node association algorithm is implemented as a distinct network function, herein termed “DHT control plane function” (DHTCP). DHTCP can be implemented as a separate entity (single node or distributed function) or as a part of the SDN Controller stack.
DHTCP offers an interface to DHT nodes to join or leave the DHT network and also for maintenance messages (e.g., keep-alive messages; e.g., update message to enable transfer of (key, value) pairs between nodes after a node joins or leaves). Therefore, individual DHT nodes do not need to implement this part of the DHT stack, which is effectively transferred to the DHTCP function. The DHT nodes still need to implement data plane functions (e.g., to send and receive get and set messages), but only need to communicate with the DHTCP API 407 for control plane functions (join/leave/keep alive/etc.). As a result, DHTCP 407 is aware of the full DHT routing table (i.e., node locators, and node IDs in DHT ID space), as well as the key-node association algorithm. DHTCP is therefore able to partition the DHT identifier space into Nr ranges, with Nr≧Nn, where Nn is the number of nodes in the DHT. For example, in the simplest case, Nr=Nn. In other cases, such as the SEATTLE one-hop DHT [14], there may be several virtual nodes per physical node, resulting in Nr>Nn, but Nr should stay bounded, typically in a linear fashion with Nn. In this particular example, we would have Nr≦Nv*Nn, where Nv is the maximum number of virtual nodes per physical node.
Finally, DHTCP can build the key-range-based routing table for this DHT.
Messages exchanged between DHTCP 407 and DHT nodes 405 typically should make it possible for the DHTCP to inform each DHT node of the key range it is responsible for (e.g., in JOIN response, in keep-alive response, and/or using indication messages from DHTCP to the DHT nodes). This information is necessary for the data plane DHT function on each node to properly process incoming get/set queries, and to enable migration of (key, value) pairs to a new node, for example. In an example of migration strategy, upon receiving a JOIN request, the DHTCP computes the new ID ranges associated with all DHT nodes, and requests a migration from certain DHT nodes (typically a small number of nodes, thanks to the consistent hashing technique) toward the given new node ID. Then it updates the network with updated ranges and deletes the migrated data from the old DHT nodes.
In addition to migration, data replication also is important in DHTs in order to enable recovery from individual DHT node failures. This feature typically can be considered as a control plane function, where the DHTCP computes the replication map (i.e., which key range should be replicated in which DHT node) and provides it to DHT nodes, which execute the replication using destination-based forwarding (i.e., messages are forwarded by routers/switches based on the destination node ID, not the content-ID/key). Other replication strategies exist, such as using different hashing functions to compute the key. That is, two (or more) different hashing functions can be provided, each generating different value for a given key, each value identifying a different location of the data. Such an embodiment can be implemented using this invention without any further change in the network side since then only the end points are involved. Publishers may calculate Nh hashes and send Nh SET/PUBLISH requests, and subscribers may select one of the hash functions and send one related GET/SUBSCRIBE request.
Referring to
For sake of illustration, this transformation can be performed using the following exemplary “range-to-strict matching translation” algorithm. Initially, all values in the range are listed in binary form, and then formed into groups with the longest series of common bits. To illustrate, consider the range 20-30 (in base 10) as shown below (including the values adjacent to that range). This range starts with 10100 (base 2) and ends with 11110 (base 2).
The range 20-30 can be expressed by a criterion. In this case, the criterion is the union of the following four bitmasks (where x is a wildcard bit matching any of 0 or 1): 101xx base 2, 110xx base 2, 1110x base 2 and 11110 base 2. The algorithm can for example be expressed as:
Assuming that an existing OpenFlow field is used to store the hash value in the packet, and that this field supports bitmask (e.g., Ethernet field or IP address field), then the existing OpenFlow protocol can be used without modification.
This operation typically increases the number of rules in a manner highly dependent on the range. This translation is an exemplary way to enable this design without modifying the OpenFlow protocol. However, in some cases, it may be more efficient to modify the SDN/OF protocol to implement ranges, and enhance the SDN device to understand and implement this range matching. If ranges are enabled in an enhanced version of the SDN/OF protocol in accordance with the present invention, then, in one embodiment, the OF switch/router may internally implement the aforementioned range-matching by decomposing range matching into a plurality of bit-field strict matchings (rather than performing the algorithm in the SDN controller). Beyond this simple implementation, high speed range matching algorithms have been studied already, e.g., in [34]. Note that key-range matches typically will be combined with strict matches on other fields. Typically, ‘GET’ (or ‘SUBSCRIBE’) and ‘SET’ (or ‘PUBLISH’) request methods of the proper DHT data plane protocol should be identified by OF switches/routers using strict match of certain message fields. For those messages only, the key information element should be evaluated for inclusion in a key-range, and, if there is a match, the appropriate next-hop is determined from the forwarding entry.
The above mentioned techniques, including the technique of translating range-based rules into strict match rules has broader applications than the above-described embodiment. For instance, it may be used to support any of the ICN and hash routing techniques described hereinbelow.
Referring again to
Table 620 in
One example of implementation for messages 615 (step 3) in
unint64_t oid_range_value_min[4]; /* First value of Object ID within
the range (256 bits value) */
unint64_t oid_range_value_max[4]; /* Last value of Object ID within
the range (256 bits value) */
In the example where the DHT protocol is above UDP or TCP, the data structure above would set the UDP or TCP port to the required value. The IP address could be set, for example, to a specific anycast address, which all DHT message would need to use. The interest of such an anycast IP address is that any non-SDN router can be configured to route such packets toward the part of the network where the SDN-enabled router or switches will be able to forward the request toward their destination.
The entries “application protocol/index/method” and “oid range min/max” typically should refer to an additional description of the application protocol. This additional description can for example use regular expressions to describe how to match packets and extract the relevant piece of information from them (note that regular expressions are used today to enable packet matching in deep packet inspection systems). In another example, the application protocol can be binary and it can be sufficient to specify offsets for matching and capture of information from the packet. In any case, this additional application protocol description may be part of the configuration of the switch/router, either through pre-configuration or through configuration by the SDN controller.
The field “application_specific_index” can enable matching a DHT network ID, which is used to discriminate between several DHT networks.
One assumption here is that the network elements controlled through SDN are sufficient to enable proper range-based forwarding toward a destination. For example, all switches/routers of the domain may be SDN-controlled. Alternately, only core switches/routers of the domain may be SDN controlled, and the other routers/switches may be configured in such a way that DHT GET/SET messages will be properly forwarded toward these core router/switches. For example, DHT GET/SET messages that are meant to be forwarded based on key range could systematically use an anycast destination IP address. Non-SDN controlled switches/routers can forward these messages toward an OF-controlled router/switch, which will properly forward the message as described in this work.
In a more recent version of the OpenFlow specification (v1.4.0, https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.4.0.pdf), the format of match fields was modified and is now realized using a compact type-length-value (TLV) encoding, the OpenFlow Extensible Match (OXM). OXM TLV's type is composed of an “OXM class” and a “match field” (as part of the type), which states on which packet field this TLV applies; an example of class is the “OpenFlow Basic” class, and example of values within this class include UDP Source Address, Multi-Protocol Label Switching (MPLS) Label, etc. The header additionally includes a bit indicating if a bitmask is present and the length of the field. The value of the TLV can be the desired value of the field, plus if appropriate a bit mask which indicates which bits of the value should be matched.
One way to implement ranges in this scheme is to add a new flag in the header indicating that the value encoded is a range, i.e., a pair of value_min and value_max. This new flag may require extending the length of the header, e.g., by one byte. Alternatively, since the basic length of the value is known from the type, simply setting the existing mask flag to 0 while keeping the length doubled from the normal value can implicitly mean that the value of the TLV is a range. In all cases, the encoding of the range in the TLV can simply be the concatenation of 2 values (min and max).
6.1.2 ICN Publish and Subscribe Operations
In the context of an ICN design using a multi-level hierarchical DHT as part of content routing [3], every single DHT level is implemented inside a single domain. All or some of these domains may implement the SDN-enabled DHT routing described above. Let's consider that the system is in steady state, i.e., each DHT network is configured and maintained by its DHTCP function, and each DHT node holds the (key, value) pairs it is responsible for.
The steps involved in an exemplary content publication are shown in
The content router 703 sends a PUBLISH request 712 over one or more interfaces facing the inside of its domain. There is no need for a destination information element, but the message includes information elements holding (a) a key (content ID or data derived from a content ID), and (b) a DHT data plane protocol header (e.g., IP/UDP/HTTP POST/XML).
Note that, in
When reaching an intermediate node (OF switch or router) 705, the intermediate node matches the content of the message header with its forwarding table. Based on strict match of some information elements such as protocol and method, the intermediate node 705 proceeds with matching the key with key-ranges associated with different forwarding entries, and, if the key belongs to one of these ranges, it selects the related forwarding entry and message 713 is forwarded (possibly through one or more switches/routers, e.g., switch 707) to the relevant content router that is responsible for that key, e.g., router 709. The DHT node 709 stores the (key, value) pair and replies to the source node with message 715.
The reply message 715 is forwarded using destination-based forwarding rules. For example, the locator (e.g., IP address) of the content subscriber 701 could be obtained from the original request and used as the destination of the response. Other schemes are available, such as source routing such as using the locator of content router 703 which will relay the response to the content subscriber 701.
In the diagram, the return path is different from the request path (passing through SDN switch 711 to content router 703 to content publish 701, rather than through SDN switch 707 to SDN switch 705 to content router 701), but if a source routing scheme is used (i.e., forwarding path is recorded in the original request message) the request and response path would be identical.
The role of the reply message (5) typically is to indicate the success or failure of the publication operation.
The steps involved in content requests are shown in
Content router 803 sends a NAME RESOLUTION (a DHT GET message) request message 818 over one or more interface facing nodes (e.g., SDN switch 805) from its own domain. Again, there is no need for a destination information element, but the message 818 includes information elements holding (a) a key (content ID or derived from content ID), and (b) a DHT data plane protocol header (e.g., IP/UDP/HTTP GET/XML).
The DHT GET message 818 is forwarded as shown by message 819 to the content router 807 containing the corresponding (key, value) pair.
The DHT node 807 responsible for this key reads the (key, value) pair from its local storage, as represented by 820, and includes it in its reply message 821. The value is a locator where the content with the given content ID/key can be retrieved.
The message 821 is forwarded through the network fabric from SDN switch 805 to content router 803 via message(s) 822 until it reaches the content router 803. Note that the response messages 821 and 822, do not necessarily use range based routing. For example, the responding DHT node may simply send the response back to the requesting content router using regular IP routing (e.g., using the IP address of the requesting content router as destination IP address of the response).
Using the locator from the response 821, 822 as the destination, the content router 803 forwards a SUBSCRIBE request 823 toward the content source 809 through the network fabric (in this case SDN switch 804 and content router 805). Note that the locator may point to the content source, a cache, or a border router that will forward the request further outside of the domain.
The Content Source 809 sends back the content object in message 824, to the subscriber 805 through the network fabric, e.g., through content router 805, SDN switch 804, content router 803, and, finally, to the requesting content subscriber 801. For example, this can be done using regular IP routing, where the content source 809 sends the response using the IP address of the subscriber 801 as destination IP address.
6.1.3 Separation between Content Routers and NRS
The method described above applies in the context of an ICN design using a DHT-based Name Resolution System distinct from the content routers infrastructure and that can be queried by any node, such as in Netlnf [4]. However, additionally any node may send a GET/SET message to the DHT-based NRS, not only the nodes that are members of the DHT. Therefore, the initial content requester or publisher may send a message toward the NRS, without the need to indicate a NRS entry point as a destination.
Also note that, in the Netlnf system [4], there may be multiple entangled DHTs, some of which may not be inside the same SDN domain. For those higher level DHTs (i.e., with a more global scope), an over-the-top DHT not making use of the present invention may be used; or alternatively, such multi-domain DHTs may be implemented in accordance with the embodiment described below in section 6.3.
6.2 Enabling SDN-Controlled Overlay DHTs
A second embodiment is suitable outside the scope of an ICN system and could apply to any DHT that is within a SDN domain. In particular, the system description of section 6.1.1 stands on its own and can be applied for any overlay DHT that is entirely contained within one SDN domain (and may be expanded to several domains as described in section 6.3). The present section adds further details to section 6.1.1. These overlay DHTs can be deployed over today's TCP/IP stack or can use another transport mechanism. Their implementation can be derived from existing DHT implementations such as Chord, Pastry, etc. As described in section 6.1.1, the adaptation of an existing DHT implementation comprises implementing the DHT Control part (including handling of node keep alive/join/leave and associated data migration) into a centralized entity (e.g., part of the SDN controller or a third party entity), and having DHT nodes communicate with this centralized entity using a simple API (e.g., including JOIN, LEAVE, DATA_TRANSFER and KEEP ALIVE messages). In particular, several independent overlay DHT networks can be supported in a same SDN domain by combining key-range matching with other information elements, such as an overlay DHT network ID. For example, the OpenFlow interface may be enhanced to add new fields “uint32_t_dht_id; /* DHT identifier. */” and “uint16_t dht_protocol; /* DHT protocol −0=Chord, 1=Pastry, etc. */” in the data structure described in 6.1.1. These information elements can be used by OF switches and routers to match packets depending on the particular DHT overlay network (and protocol) they belong to. Particular DHT protocols may need to be enhanced to support a network ID field to facilitate efficient identification by OF switches/routers.
Examples of DHT Control interface include the following messages:
This can, for example, enable efficient implementation of multiple parallel naming services. Hence, rather than having one centralized naming resolution service such as today's DNS, there may be multiple naming services used in a competing or complementary fashion (e.g., one for each Information or Service Centric Network scheme supported in a network; e.g., one for each P2P overlay network).
When an overlay DHT node, e.g., DHT node 1005b, joins overlay DHT #2, the first step is for it to send a JOIN DHT message to the DHTCP 1007b for overlay DHT #2, and receive its local key range from the DHTCP, collectively represented at 1011. Then, DHTCP 1007b updates the key ranges for the DHT nodes under its control (e.g., DHT nodes 1005b and 1005c) and sends messages (not shown) to the DHT nodes under its control (e.g., 1005b, 1005c) setting new range assignments. DHTCP 1007b also sends that same information to the SDN Controller 1003, as shown by message 1013. SDN Controller 1003 then determines updated key ranges for all of the DHT nodes in its domain, including DHT nodes 1005a-1005c for DHTs #1 and #2 and sends messages (1015) to the SDN switches 1002a, 1002b, 1002c setting new forwarding tables and rules for the overlay networks based on the new key range assignments.
When a client node, e.g., client node 1019b, issues a SET /GET request (message 1017), it is routed through the network to the appropriate DHT nodes 1005a-1005c and beyond as dictated by the new forwarding tables and rules.
If, as depicted, DHTCP 1007b is an external entity, the SDN northbound interface can include the message SET_KEY_RANGE 1013 as defined below. SET_KEY_RANGES
Note that, in this exemplary API, a DHTCP would send a single such message describing all key ranges toward a DHT node, which would overwrite any previous message related to this particular DHT node within this particular DHT network. Variants of this API may enable a more fine-grained control (e.g., add/update/remove messages).
In the opposite case, i.e., if DHTCP is implemented inside the SDN controller, then the “DHT Control API” described above will become the northbound interface (e.g., the DHT node can directly use the SDN northbound API in this case). Moreover, the DHTCP/SDN API described above (with SET_KEY_RANGE) becomes an internal interface in this case. Hence, one can summarize the SDN Controller role as:
6.3 Enabling SDN-Controlled DHT across Multiple Domains
Both in the context of DHT for ICNs (first embodiment) or in the more general case (second embodiment), one interesting use case is a DHT spanning several domains. Some or all individual domains may implement a SDN-controlled DHT as described herein.
6.3.1 Cooperative Multi-Domain SDN-Enabled DHT Case
In a first “cooperative multi-domain SDN-enabled DHT” case, illustrated in
As in the single-domain case, DHTCP has the opportunity to optimize node ID allocation to maximize the opportunities for merging forwarding flow entries. One simple strategy is to partition the DHT ID space between domains, which makes it possible to forward DHT traffic toward all DHT nodes of a given domain typically with a single flow entry.
6.3.2 Hybrid Case
In a second, “hybrid” case, an SDN domain may internally control some of the DHT nodes while other DHTs nodes are outside of the domain. New border functions are introduced inside SDN domains, which are acting as proxies between the SDN-controlled DHT nodes and the outside DHT nodes. These border elements can forward DHT messages toward internal and external DHT IDs, as configured by the DHTCP. To enable this, the DHTCP can participate in the outside DHT on behalf of the inside nodes. DHTCP deals with control plane functions (including maintaining DHT routing tables) and inside DHT nodes deal with data plane functions. The border element forwards inward messages toward the DHTCP (control plane message) or DHT nodes (data plane messages). The border element performs some Network Address Translation (NAT)-like function or proxy function to outward messages in such a way that DHTCP messages related to one DHT node appear to be sent by this DHT node. From the outside DHT node's standpoint, inside DHT nodes implement the full DHT stack (i.e., border elements transparently divert some of the traffic to/from DHTCP).
In one optimized embodiment, the node ID allocation scheme may be designed to allocate a single partition of the ID space to certain domains (e.g., one per domain). This makes it possible to merge flow entries in DHT routing tables both inside SDN-controlled networks and in DHT nodes outside such networks. For example, all DHT nodes within a domain can be configured to use a given prefix in their node ID, where this prefix is permanently allocated to this domain.
7.1 Discussion on Hash Routing Control
Today, one way to implement hash-routing is to have a flow entry that redirects all packets of a certain type to the controller (e.g., all TCP SYN). The controller then calculates the hash value using certain field(s) from the packet, and checks an internal hash-routing table to determine how to process the packet (e.g., forward or drop). The controller can then configure a new flow for this particular TCP session.
This solution can become inefficient in cases where hash-routing is to be applied to a large number of packets. For example, in the case of ICN, every content request and/or response may be subject to hash-routing. In the particular case of CCN, there is a ratio of one request to one data packet, with typical data packet sizes of a few kilobytes, which leads to a very large number of content requests, and since there is no end-to-end transport session, it could become necessary to follow the simple method described above, i.e., to involve the controller for every single packet.
Today, there is no generic method to enable SDN-controlled hash-routing where the hash value calculation and forwarding decision is located inside the router/switch, without involving the controller.
7.2 Hash Routing Control (HRC) in SDN
Hash-routing comprises making a forwarding (or routing) decision based on a hash value. This hash value can be present in a field of an incoming packet or it can be calculated from information element(s) present in the incoming packet. Section 6 describes how a SDN controller can define a flow entry using a range of values on a particular field in the message. This can be generalized by enabling the SDN controller to configure the SDN router/switch to calculate a hash value from an incoming packet and then use this value to match a particular flow entry.
7.2.1 Enhancement of SDN Controller
The SDN Controller 3 first should discover the SDN-enabled network elements and interconnect with them for control. Once that phase is complete, the SDN Controller existing components, such as the IP routing SDN application, can configure the network as usual. Then, the HRC SDN Controller 3 obtains and possibly updates the HRC configuration of the routers/switches using the interface described in section 7.2.2.2. At this point, the SDN Controller 3 is aware of the HRC capabilities (i.e., support HRC and supported hash functions and hash input methods) available in each SDN device 1207. The SDN controller 1203 is now ready to accept North-Bound API commands from applications.
Some SDN router/switches 1207 may not be HRC aware. When adding a new HRC rule, the SDN Controller 3 should configure these non-HRC devices as needed to enable messages to be forwarded between HRC-enabled routers/switches. Therefore, typically, it is not necessary for the whole network to be HRC-aware to enable proper hash-routing control.
7.2.2 Enhancement of SDN South-Bound Interface
The HRC South-Bound Interface 1204 enhancement may be divided into two general aspects. First, the core of the interface enhancement comprises additional information elements describing the HRC flow entries for use in messages adding or otherwise controlling flow entries. Second, additional interface enhancements comprise new messages that enable control of the parameters of the core enhancement, including the hash function ID and hash input method ID.
7.2.2.1 Core HRC Enhancement Description
HRC is hereinbelow illustrated in a typical implementation of SDN, namely, OpenFlow, described in the OpenFlow specifications Error! Reference source not found. However, it should be understood that this is merely an exemplary embodiment and that this work may be applied to other Software Defined Networking approaches (such as FORCES for example).
While the primary HRC enhancements will be the HRC matching enhancements described later in this section, HRC is a capability that can be made known by the SDN device to the SDN controller during the existing OFPT_FEATURES_REQUEST/OFPT_FEATURES_REPLY message exchange, where the SDN controller requests information from the SDN device, and the device replies with a data structure ofp_switch_features, including its capabilities in a “capabilities” bitmap. Shown below in Table 3 is the OpenFlow ofp_capabilities bitmap showing an enhancement in the form of the last entry (underlined), i.e., the OFPC_HASH_ROUTING_CONTROL element that discloses the HRC capabilities, if any, of the device.
OFPC_HASH_ROUTING_CONTROL = 1 << 8 /* HRC Support. */
In OpenFlow, the SDN controller can update the flow table in the SDN switch/router 7 using a “Modify Flow Entry” Message. This message includes the ofp_match data structure, which describes how a packet matches this particular flow entry.
An enhanced embodiment of the ofp_match data structure in OpenFlow is shown below in Table 4. Particularly, the ofp_match data structure (part of an “Enhanced Modify Flow Entry” message) may be enhanced to include hash routing specific information elements. This enhancement may include adding the number of hash routing descriptors that are also present in the message, seen below as the new unint8_t num_hash_routing_descriptors entry (underlined). In the general case, when there is no hash routing specified in a command, there is no additional data. If there is (typically one) hash routing component in the flow description, the field is set to 1 and the additional hash routing information is carried in a second data structure in the same message.
unint8_t num_hash_routin_descriptors; /* Number of hash routing
description entries included in structure */
An OpenFlow message carrying ofp_match with a non-zero n=num_hash_routing_descriptors will also carry n new data structure ofp_hash_routing_descriptors, as shown below in Table 5:
struct ofp_hash_routing_descriptors {
struct ofp_hash_function_specification hash_function_specification;
struct ofp_hash_input_specification hash_input_specification;
struct ofp_range_specification range_specification;
In the structures above, hash_input_specification designates which method is used by the SDN device to extract the hash function input from an incoming message, and hash_function_specification designates how the hash function is calculated based on this input to generate a hash value. Range_specification completes the set of parameters that is needed for the SDN device and specifies the range of hash values to take a hash-routing match decision. Exemplary embodiments of the data structures of these three descriptors are shown below in Table 6.
struct ofp_hash_function_specification {
uint_32t hash_function_id;
uint32t size;
/* other hash function parameters may be added here */
};
struct ofp_hash_input_specification {
uint_32t hash_input_id;
/* certain hash input methods may require parameters, which
can be added here */
};
struct ofp_range_specification {
/* this example uses 32bit values - if the hash value space is larger,
it may still use the same range specification and consider
that the min and max values specified are the most significant
bits of the actual min and max hash values, setting the least
significant bits to 0 for the min value and 1 for the max value. */
uint32t min_value_included;
uint32t max_value_included;
};
The data structures above carry certain identifiers that need to be pre-agreed upon between the SDN controller and the SDN device:
As described above, the controller 1203 and the device 1207 should agree on certain IDs used to specify an HRC flow entry. Typically, the device has a pre-configured list of supported hash function methods and input methods, which are fixed for a given version of the software. While it is possible to extend OpenFlow to enable transmitting this information, it may be more appropriate to use a network configuration protocol for this type of operation, e.g., network configuration protocols such as NetConf, SNMP, or RESTConf. The following REST-based API model may be implemented using RESTConf, for example:
In a more elaborate setting, the SDN device could offer a finer grained configuration of possible input methods. For example, again using NetConf/SNMP/RESTConf protocol, the SDN Controller could set a desired input method by specifying a combination of protocols (among a list of possible protocols recognized by the device) and field identifiers (among a list of possible protocol header fields recognized by the device). The SDN device would then store the desired input method along with a unique ID, and return this ID to the controller.
As discussed above in section 7.1, cases like Hash-Routing on HTTP URI over TCP can be implemented without HRC using the controller. Nevertheless, HRC still may be used in this case. Upon match, the SDN device could, in this case, insert a new flow entry for the TCP flow over which the match was detected.
7.2.3 Enhancement of Control Function in a SDN-Controlled Switch/Router
The control function of the switch/router 1207 implements the following logic: (1) receiving SDN South-Bound Interface messages from the SDN controller, enhanced with hash-routing information elements and (2) updating the flow table based on this input.
The control function 1205 is therefore updated to process HRC South-Bound commands, such as the Enhanced Modify Flow Entry message. This processing extracts the fields from the control message and populates the flow table structure. The format of a flow entry may be enhanced as shown below in Table 9 (in which new fields are indicated in underlining and existing information elements taken from the OpenFlow specifications Error! Reference source not found.are indicated without underlining).
HRC-Enhancements:
A number N of “Hash Routing
Match Descriptor” (0 for non-
hash-routing flows, 1 or more
for hash-routing flows).
N “Hash Routing Match
Descriptor” data structure (see
Tables 5 and 6 in section
7.2.2.1 above), which includes
a description of the hash
function, input method,
and range.
Additionally, the Control Function 1206 of the SDN Switch/Router 1207 should enable the other south-bound API functions, such as the advertisement of the supported hash functions/input types/range specifications. For hash functions and input types, the forwarding engine implementation (which may not be extensible because of efficiency reason, e.g., performed in hardware) may limit the actual hash functions and input types that are available. Typically, the forwarding engine may implement a single range specification method and the switch control function can perform the conversion between multiple range specification methods made available over the South-Bound interface and the single range specification implemented in the engine.
7.2.4 Enhancement of Forwarding Function in SDN-Controlled Switch/Router
Currently, the forwarding function of a SDN Switch/Router comprises selecting flow entries based on matching rules (exact value match). HRC-support should enhance the forwarding function to support packet matching based on the new HRC-enhanced flow entry described in Table 9 above:
OpenFlow already supports defining multiple flow tables and flow entry points to other flow tables. Therefore, a possible way to implement hash-routing is to use a first non-hash-routing flow entry in a first table as an entry point to a second flow table containing several hash-routing entries. For example, the first flow entry can match all packets with a destination IP address set to a certain anycast address. The second flow table can then contain multiple similar flow entries that differ primarily by their range and forwarding action, but which typically share the same hash function and input method. Furthermore, one important design consideration for the forwarding engine is that it should enable line-speed operation. For example, hash function calculation and range matching may be implemented entirely or partially in hardware. Another speed optimization feature may be to cache recently calculated hash values in such a way that packets belonging to the same flow will not in general result in the same hash value being calculated multiple times.
In accordance with one embodiment as shown in
Otherwise, flow proceeds from decision step 1326 to step 1334 to select the next flow entry in the table. Then it determines if that flow entry is an HRC-enabled flow entry (step 1336). If not, flow proceeds largely as described above in connection with
On the other hand, if it is an HRC-enabled flow entry, flow instead proceeds from decision step 1336 to step 1340, where it is determined if the packet match parameters match those of the flow entry. If not, flow proceeds back to step 1326 to start the process again for the next flow entry in the table. If so, flow instead proceeds to step 1342 where the switch determines the hash function and extracts the hash input fields using the algorithms dictated in the flow entry. Next, in step 1344, it checks to see if it already has a cached copy of the hash value for the matching and hash function and inputs. If so, it retrieves the hash value (step 1348). If not, it calculates the hash value using the identified hash function (step 1346). In either event, next, in step 1350, it checks if the hash value is within the range of (key; value) pairs stored at this switch. If so, it proceeds through steps 1352, 1354, and 1356, which are substantially the same as steps 1303, 1307, and 1309 in
7.2.5 Exemplary Embodiment
For this exemplary embodiment, the topology shown in
In this example network, UE 1411 is running a networking application that makes use of a HRC-aware SDN Application implemented in the SDN controller 1401. To illustrate the most general case, further assume that only a subset of the SDN devices is HRC capable (here, only router 1403 and border router 1409 are HRC capable). No assumptions are made on the nature of the HRC-aware SDN application. In this example, the SDN application on UE 1411 decides to split CCN traffic into two flows, one through router 1405 and one through router 1407.
Hence, as shown by signal flows 1505, 1507, 1509, and 1511, the SDN Controller 1401 requests (and receives) the Hash functions and Hash function inputs from each of the HRC capable routers 1403 and 1409. (Note that the double arrowed lines with one full arrow at one end and one line arrow at the opposite end represent a request (the full arrow) and the response thereto (the line arrow).
Next at 1513, the networking application on UE 1411 makes use of a HRC-aware SDN Application, A, implemented in the SDN controller 1401. The nature of application A is not relevant. For example, it may be a DHT Control Plane application running on the SDN Controller. It also may be any other SDN application that makes use of hash routing, such as a load balancing SDN application. In that case, the Networking App on UE 1411 may, for example, configure the SDN load balancing application to divide the load equally between two caches. At 1515, application A in SDN Controller 1401 decides to set up hash routing to split the content between UE 1413 and 1411 in both directions between routers 1405 and 1407 for load balancing purposes. So, at 1517, SDN Controller 1401 sets flows on SDN routers 1405 and 1407 to forward CCN packets between border router 1409 and router 1403. Next, as shown at 1519, the controller 1401 first configures the flow table in HRC capable router 1403 with a flow entry F1 that will forward name resolution packets in a first range to router 1405 for all traffic that is headed outside of the network. Similarly, as shown at 1211, the Controller 1401 also configures router 1405 with a flow entry F2 that will forward name resolution packets in a second range to router 1407. Alternately, both flow entries could be sent in a single message.
Next, as shown at 1523 and 1525, Controller 1401 similarly configures the flow table of HRC capable router 1409 with flow entries F3 and F4 for forwarding name resolution requests in the first range to router 1405 and name resolution requests in the second range to router 1407.
Another network 1500 issues a CCN request 1526 to the border router to which it has access, router 1409, for content that is located at UE 1413 (although the request does not identify the content by its location). In step 1527, router 1409 processes the key in the request through the first table and finds a match to a flow in the table. That flow typically matches incoming content requests for content available locally, and indicates that the request should be processed through a different table, table B. Then, router 1409 looks in flow table B and finds a match with entry F4 for the range within which the CCN request falls (step 1529). Thus, it forwards the CCN request to router 1407 as shown at 1531. Router 1407 determines that the requested content object is not found in its cache (step 1533) and forwards the request towards router 1403 (step 1535). Router 1403 determines that the requested content is in UE 1411 (step 1537) and forwards the request to UE 1411 (step/message 1539). UE 1411 returns the request content object to router 1403 (step/message 1541).
Next, at step 1543, router 1403 matches the CCN content response to a flow in its flow table. This flow typically matches outgoing responses to requests for content available locally, and indicates that the response should be processed through table A. Then, in step 1545, router 1403 processes the key through table A and finds a match for the CCN response in flow F2 of flow table A and, therefore, forwards the request as dictated by flow entry F2, ie., to router 1407 (step/message 1547).
Router 1407, which is not an HRC capable router, determines that it should forward the response to router 1409 (it also may cache the content for future use), as shown at step/message 1549. Next, HRC capable router 1409 determines that it should forward the content response to the other network 1500 (step 1553) and performs a conventional CCN forwarding of the content to the other network 1500 (step 1555).
In the above-described embodiment, the SDN device needs to calculate the hash function as part of every flow matching (as long as this flow includes a hash routing matching descriptor). As an optimization, the result of such a calculation may be cached. Therefore, even if the flow table has multiple flows using the same hash function and hash function input field, but with a different range, then only one hash value will need to be calculated for the first flow. Then, subsequent flows matching operations will reuse this value. Consequently, this embodiment should not require unnecessary processing.
7.3 Hash Routing Control (HRC) in SDN—Alternate Embodiment
The embodiment described in the previous section requires a relatively complex structure (hash routing match descriptor) to be included in the definition of a flow. Thus, while that embodiment is a practical one, there are other embodiments that require less intrusive changes to the OpenFlow protocol.
OpenFlow protocol supports a so-called “Apply-Action” instruction. When a packet matches a flow with an “Apply-Action” instruction configured, the switch will immediately execute the action specified by the instruction. The actions are those defined in the OpenFlow protocol, e.g., Push a new VLAN tag.
An alternative embodiment of HRC in SDN utilizes and enhances such “Apply-Action” instruction to simplify the definition of HRC-related flows. The key ideas of this alternative solution include:
In accordance with the embodiment, flow entries are created to send hashed packets to their routing destinations using the output of the HRC-FUNCTION action. The “HRC-FUNCTION” action may have several parameters, such as: a function ID, a Table ID, and, possibly, a metadata mask. This action triggers the calculation of a function from the input packet and sets the result of the function as metadata (possibly using the metadata mask to delimit which part of the metadata will be updated with the function output). Note that a storage location other than the existing metadata field could be used, e.g., a dedicated new metadata information element. Nevertheless, this embodiment still is likely less disruptive to the existing OpenFlow specifications than the previous embodiment.
Presently, the metadata field length is 64-bits. If the function output is larger than 64 bits (or larger than the specified mask), then, typically, only the most significant bits of the function output will be copied over. Then the packet is processed through the table specified in the Table ID part of the action. Note that such Table ID is also optional to the “HRC-FUNCTION” action because a “Write-Action: Table ID” instruction would have the same effect of redirecting the hashed packets to the flow table with Table ID. With that being said, the Table ID parameter does keep the first flow table simple and “HRC-FUNCTION” action independent. The function ID identifies the actual function to be used. This function can be pre-defined by the SDN device and/or configured by the SDN controller using a configuration API. Such configuration includes the input field(s), e.g., the destination IP address, a content name in a CCN request. The SDN device and OpenFlow protocol are therefore enhanced to support an Enhanced Modify Flow Entry message. The format of a flow entry (both in the OpenFlow protocol and in the internal flow representation in the SDN device) is enhanced as shown below in Table 10 (new fields are indicated with underlining and existing information elements are taken from the OpenFlow specifications Error! Reference source not found.).
HRC-Enhancements:
Additional fields in ofp_match:
Unint64_t metadata_min; /* minimum
value of metadata (included) */
Uint64_t metadata_max; /* maximum
value of metadata (included) */
HRC-Enhancements:
New action “ HRC-FUNCTION”
supported by “Apply-Action”
instruction.
FUNCTION action parameters:
7.3.1 Example of Application of the Alternate Embodiment of HRC in SDN
Prior to configuring the flows, the SDN Controller uses NetConf to configure function ID #0 to use the MurmurHash3 hash function, and to use the CCNx content name field as input. The previous section already discussed how the SDN controller can obtain support hash functions and input methods (see ListSupportedHashFunctions( )and ListSupportedHashFunctionlnputs( )above). An additional NetConf command is proposed to enable the SDN controller to configure the hash function in the SDN device. Table 11 below shows an exemplary ConfigureHashFunction( )configuration API function.
Table 12 below shows two flow tables of an exemplary embodiment in accordance with this alternative. Note that the two flow tables, Table 0 and Table 23, can be configured as two parts of a single table in a database.
In steps/messages 1623 and 1625, the SDN Controller 1401 performs similar operations for router 1409, as shown.
In this embodiment, after router 1409 receives the CCN content request from the other network 1500 (message 1526 in
7.4 Hash Routing Control (HRC) in SDN—Other Alternate Embodiments
7.4.1 Application to other SDN Frameworks
While the two embodiments discussed above apply to OpenFlow, it is understood that similar solutions can be devised by enhancing other SDN southbound protocols (e.g., ForCES). The general idea is that the SDN controller specifies SDN flows (or rules) associated with a hash function, a descriptor to extract the input of the hash function, and a range of acceptable values for the output of the hash function.
7.5 Exemplary SDN Applications That May Use SDN Hash Routing Control
7.5.1 ICN Name Resolution System
Some ICN systems can be designed to use a hash-routing based name resolution system. Such systems can benefit from being implemented using SDN as a building block, as described in section 6. Section 7.2 detailed an enhancement to SDN enabling these ICN systems.
A simple north-bound API of such an ICN Name Resolution System (NRS) was described in section 6. Every ICN router can use it to join or leave the ICN NRS. The ICN NRS Application (within the SDN Controller) can then divide the hash value space between all ICN routers, and then configure the HRC-aware SDN devices to forward content requests and publish messages toward the appropriate ICN router using hash routing. Non HRC-aware SDN switch/routers can, for example, be configured to forward traffic to a given anycast IP address toward the closest HRC-aware SDN switch/router.
7.5.2 Overlay Distributed Hash Table
A second class of SDN application making use of the HRC feature described herein is a DHT application.
The north-bound API of such a DHT SDN application was described in section 6. It includes Join( ) Leave( ) TransferData( )and KeepAliveQ. These API actions can, for example, be implemented using a REST design using HTTP and JSON encoding. Any IP host can use this API to Join( ) a DHT network. The DHT SDN application can divide the hash value space between all DHT nodes, using a consistent hashing technique to minimize the fraction of the hash value space that gets remapped from one DHT node to another when a new node joins or leaves the network. The mapping between hash value space and DHT nodes can be entirely determined by the DHT SDN application once it knows which nodes belong to the DHT (and possibly taking into account the CPU/storage capacities of the nodes, which can be given in the Join( )message). Then, the DHT SDN application can use the method described in section 7.2 to configure the hash-routing flow entries in the SDN devices of the network.
7.5.3 Hash-Routing Based Peering
U.S. Provisional Patent Application No. 61/934,540, which is incorporated herein fully by reference, describes Hash-Routing based Peering (HRP), i.e., a method enabling networks to cooperate to divide a hash value space between them, which then enable the use of hash routing to cooperatively increase network efficiency. This cooperation could be enabled through a fully distributed mechanism or through a centralized, SDN-based mechanism using some of the concepts and embodiments disclosed in the present specification.
U.S. Provisional Patent Application No. 61/934,540 describes two north-bound APIs of such HRP SDN application, namely, an exterior routing API and an interior routing API. The exterior routing API includes:
The interior routing API includes:
7.5.4 Load Balancing in IP Network
As a fourth class of SDN application that may make use of the HRC feature in the present invention, consider an exemplary load balancing SDN application, (hereinafter LBS). LBS is deployed in a SDN domain that has some HRC-aware SDN devices. A client networking application uses LBS's API to set pools of caches, set traffic filters to match traffic that will be handled by the caches, and update load status for each cache (note that, alternatively, the caches may also update this information themselves, using the API). The northbound API shown in Table 13 below is an example of such an API and how this API can make use of the underlying HRC features described in this specification.
A call to ActivatePool could result in LBS configuring the non-HRC-aware part of the network to properly forward traffic toward one of the HRC-aware SDN devices. LBS divides the hash value space between all nodes configured in the pool. Then, HRC-aware SDN devices are configured with HRC-aware flow entries, directing matching packets toward the proper cache.
Subsequent calls to AddNodeQ,DeleteNode( )and UpdateNodeStatus( )will result in LBS recalculating the appropriate re-partition of the hash value space between caches, and reconfiguring HRC-aware SDN devices with the new or updated flow entries.
As shown in
The communications systems 100 may also include a base station 114a and a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the core network 106, the b,Internet 110, and/or the networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.
The base station 114a may be part of the RAN 104, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The cell may further be divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In another embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.
The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).
More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (HSUPA).
In another embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A).
In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA20001x, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.
The base station 114b in
The RAN 104 may be in communication with the core network 106, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. For example, the core network 106 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown in
The core network 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and/or other networks 112. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite. The networks 112 may include wired or wireless communications networks owned and/or operated by other service providers. For example, the networks 112 may include another core network connected to one or more RANs, which may employ the same RAT as the RAN 104 or a different RAT.
Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities, i.e., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links. For example, the WTRU 102c shown in
The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While
The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 116. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.
In addition, although the transmit/receive element 122 is depicted in
The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, for example.
The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 106 and/or the removable memory 132. The non-removable memory 106 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).
The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.
The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.
The processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality, and/or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.
As shown in
The core network 106 shown in
The RNC 142a in the RAN 104 may be connected to the MSC 146 in the core network 106 via an IuCS interface. The MSC 146 may be connected to the MGW 144. The MSC 146 and the MGW 144 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices.
The RNC 142a in the RAN 104 may also be connected to the SGSN 148 in the core network 106 via an IuPS interface. The SGSN 148 may be connected to the GGSN 150. The SGSN 148 and the GGSN 150 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between and the WTRUs 102a, 102b, 102c and IP-enabled devices.
As noted above, the core network 106 may also be connected to the networks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers.
The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.
Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in
The core network 106 shown in
The MME 162 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.
The serving gateway 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface. The serving gateway 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c. The serving gateway 164 may also perform other functions, such as anchoring data planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like.
The serving gateway 164 may also be connected to the PDN gateway 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.
The core network 106 may facilitate communications with other networks. For example, the core network 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the core network 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 106 and the PSTN 108. In addition, the core network 106 may provide the WTRUs 102a, 102b, 102c with access to the networks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers.
As shown in
The air interface 116 between the WTRUs 102a, 102b, 102c and the RAN 104 may be defined as an R1 reference point that implements the IEEE 802.16 specification. In addition, each of the WTRUs 102a, 102b, 102c may establish a logical interface (not shown) with the core network 106. The logical interface between the WTRUs 102a, 102b, 102c and the core network 106 may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management.
The communication link between each of the base stations 170a, 170b, 170c may be defined as an R8 reference point that includes protocols for facilitating WTRU handovers and the transfer of data between base stations. The communication link between the base stations 170a, 170b, 170c and the ASN gateway 172 may be defined as an R6 reference point. The R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102a, 102b, 100c.
As shown in
The MIP-HA 174 may be responsible for IP address management, and may enable the WTRUs 102a, 102b, 102c to roam between different ASNs and/or different core networks. The MIP-HA 174 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The AAA server 176 may be responsible for user authentication and for supporting user services. The gateway 178 may facilitate interworking with other networks. For example, the gateway 178 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. In addition, the gateway 178 may provide the WTRUs 102a, 102b, 102c with access to the networks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers.
Although not shown in
In one embodiment, an apparatus is implemented for creating a Distributed Hash Table (DHT) among a plurality of DHT nodes in a Software Defined Networking (SDN) domain comprising: a DHT Control Plane (DHTCP) module adapted to manage a plurality of DHT nodes to maintain a DHT, the DHTCP comprising a processor adapted to: receive messages from DHT nodes indicating status of the DHT nodes, including joining and leaving the DHT; determine a range-based distribution among the DHT nodes of (key, value) pairs of a DHT as a function of the keys of the (key, value) pairs; and send configuration messages to the DHT nodes for configuring each DHT node to store at least one range of keys corresponding to ((key, value)) pairs of said DHT.
The preceding embodiment may further comprise wherein the processor is further adapted to: determine, based on the determined distribution, a DHT routing table for SDN switches for handling data plane DHT data requests in the domain; and send the DHT routing table to each DHT node.
One or more of the preceding embodiments may further comprise wherein the data plane data requests comprises at least one of DHT GET and DHT SET requests.
One or more of the preceding embodiments may further comprise wherein the SDN Controller further includes a processor adapted to: determine forwarding tables for said SDN switches in the domain based on said routing table received from said DHTCP; and send messages to said SDN switches in said domain for configuring said SDN switches with said forwarding tables.
One or more of the preceding embodiments may further comprise wherein the forwarding tables include rules for forwarding DHT GET/SET messages as a function of at least one field in the DHT GET/SET messages having a predetermined relationship to at least one DHT keys.
One or more of the preceding embodiments may further comprise wherein the forwarding tables include rules for forwarding DHT GET/SET messages based on at least one field in the DHT GET/SET messages being within a range corresponding to a set of DHT keys.
One or more of the preceding embodiments may further comprise wherein the forwarding tables include rules for forwarding DHT GET/SET messages based on at least one field in the DHT GET/SET messages strictly matching a value corresponding to a DHT key.
One or more of the preceding embodiments may further comprise wherein the forwarding tables further include rules for forwarding DHT GET/SET messages based on a DHT network ID.
One or more of the preceding embodiments may further comprise wherein the DHTCP is disposed within the SND controller.
One or more of the preceding embodiments may further comprise wherein the DHTCP is configured as a northbound Application Program Interface (API) client of the SDN controller.
One or more of the preceding embodiments may further comprise wherein the DHTCP includes an Application Program Interface (API) for communications between the DHTCP and the DHT nodes.
One or more of the preceding embodiments may further comprise wherein the communications between the SDN Controller and the SDN switches use OpenFlow communication protocol.
One or more of the preceding embodiments may further comprise wherein the SDN domain comprises an Information Centric Network (ICN).
One or more of the preceding embodiments may further comprise wherein the messages between the DHTCP and the DHT nodes are DHT control plane messages.
One or more of the preceding embodiments may further comprise wherein the messages between the DHTCP and the SDN Controller are DHT control plane messages.
One or more of the preceding embodiments may further comprise wherein the DHTCP comprises a network node distinct from the SDN Controller.
One or more of the preceding embodiments may further comprise wherein the DHTCP comprises part of the SDN Controller.
One or more of the preceding embodiments may further comprise wherein the DHTCP comprises a plurality of DHTCPs, each DHTCP adapted to manage a different plurality of DHT nodes to maintain a different DHT, and wherein the SDN Controller determines forwarding tables for said SDN switches in the domain based on a plurality of routing tables received from the plurality of DHTCPs.
In another embodiment, a method of processing Distributed Hash Table (DHT) routing requests in a Software Defined Networking (SDN) domain comprising a plurality of SDN switches interconnecting a plurality of DHT nodes, each DHT node containing a portion of a DHT comprises: receiving at a first one of the DHT nodes a Publish request corresponding to a content object, the Publish request including a (key, value) pair, where the key corresponds to a content ID and the value is the location where the content object is stored; if the key in the Publish request is in a portion of the DHT not contained at the first DHT node, the first DHT node forwarding the Publish request to a first one of the SDN switches; the first SDN switch including a forwarding table in which ranges of keys of (key, value) pairs are mapped to DHT nodes, and the first SDN switch matching the key from the Publish request with one of the key ranges in the forwarding table and forwarding the Publish request toward a second DHT node, the second DHT node being the DHT node to which the key of the ((key, value)) pair in the Publish request maps; and the second DHT node receiving the Publish request and storing the (key, value) pair found in the Publish request in its DHT portion.
The preceding embodiment may further comprise wherein the Publish request further includes a DHT data plane protocol header.
One or more of the preceding embodiments may further comprise the second DHT node sending a reply to the first DHT node.
One or more of the preceding embodiments may further comprise: a subscriber node sending the Publish request to the first DHT node; and the first DHT node forwarding the reply to the subscriber node.
In another embodiment, a method of processing Distributed Hash Table (DHT) routing requests in a Software Defined Networking (SDN) domain comprising a plurality of SDN switches interconnecting a plurality of DHT nodes, each DHT node containing a portion of a DHT comprises: receiving at a first one of the DHT nodes a Subscribe request for a content object, the Subscribe request including a key, where the key corresponds to a content ID; if the key in the Subscribe request does not match a key in the portion of the DHT of the first DHT node, the first DHT node forwarding the Subscribe request to a first one of the SDN switches; and the first SDN switch including a forwarding table in which ranges of keys corresponding to (key, value) pairs are mapped to DHT nodes, the first SDN switch matching the key from the Subscribe request with one of the key ranges in the forwarding table, and forwarding the Subscribe request toward a second DHT node, the second DHT node being the DHT node to which the key of the ((key, value)) pair in the Subscribe request maps.
The preceding embodiment may further comprise: the second DHT node selecting the (key, value) pair corresponding to the key in the Subscribe request and sending the value in the selected (key, value) pair to the first content router; responsive to receiving the value from the second DHT node, the first DHT node forwarding the Subscribe request toward a network node identified by the value.
One or more of the preceding embodiments may further comprise wherein the Subscribe request further includes a DHT data plane protocol header and the key is in the header.
In another embodiment, a method implemented in a Software Defined Networking (SDN) switch/router comprises: transmitting to an SDN controller a message including Hash Routing Control (HRC) capabilities of the switch/router; and receiving from the SDN controller a message including at least one hash routing descriptor indicating how the switch/router is to process routing requests.
The preceding embodiment may further comprise: wherein the at least one hash routing descriptor comprises at least one of (1) a first descriptor designating a method to be used by the switch /router to extract a hash function input from an incoming message, (2) a second descriptor designating how the hash function is to be calculated based on the hash function input to generate a hash value, and (3) a third descriptor indicating a range of the hash value within which the switch/router is to perform a specified action.
In another embodiment, a method implemented in a Software Defined Networking (SDN) switch/router comprises: maintaining a flow table for routing data packets in a SDN network; receiving from an SDN controller a flow table modification message defining a change to the flow table maintained at the switch/router, wherein the flow entry modification message further includes at least one of (1) an information element (IE) specifying a method for extracting hash function inputs from the data packets, (2) an IE specifying how a hash function is calculated from the hash function inputs, and (3) an IE specifying a range of hash function outputs to which the flow entry applies; and updating the flow table using the IEs in the flow entry modification message.
The preceding embodiment may further comprise: wherein the change defined in the flow table modification message comprises one of an additional flow entry in the flow table, a modification to a flow entry in the flow table, and a deletion of a flow entry in the flow table.
One or more of the preceding embodiments may further comprise wherein the flow table modification message further comprises an IE disclosing a number of hash routing descriptors contained in the flow table modification message.
One or more of the preceding embodiments may further comprise wherein the flow table includes flow entries including a plurality of match descriptors for matching fields in the data packets, said match descriptors including at least a hash routing match descriptor disclosing (1) a number of hash routing flows that the switch/router is capable of processing and (2) a hash routing match descriptor data structure.
One or more of the preceding embodiments may further comprise wherein the hash routing match descriptor data structure includes (1) a description of a hash function, (2) a description of an input method for the hash function, and (3) a range value disclosing a range of entries in a Distributed Hash Table (DHT) to which the corresponding flow entry applies.
One or more of the preceding embodiments may further comprise: receiving a data packet; comparing at least one matching field in the data packet with at least one of the match descriptors in a flow entry in the DHT to determine if the at least one matching field in the data packet matches at least one matching descriptor in the flow entry; and, if the at least one matching field of the data packet matches the at least one field of the flow entry, using the description of the input method of the flow entry to extract a set of hash input fields from the data message and select a particular hash function.
One or more of the preceding embodiments may further comprise: calculating a hash value using the particular hash function and set of hash input values; determining if the hash value is within the range value of the flow entry; and, if the hash value is within the range value of the flow entry, executing the instruction associated with the flow entry.
One or more of the preceding embodiments may further comprise: determining if a hash value for the particular hash function and set of hash input fields is stored in a cache; and, if a hash value for the particular hash function and set of hash input fields is stored in a cache; retrieving the cached hash value; determining if the cached hash value is within the range value of the flow entry; and, if the hash value is within the range value of the flow entry, executing instructions associated with the flow entry.
One or more of the preceding embodiments may further comprise: executing an action set associated with the instructions.
One or more of the preceding embodiments may further comprise: determining if the action set includes an entry point into another flow table; and, if the action set includes an entry point into another flow table, processing the data packet through the another flow table.
In another embodiment, a method implemented in a Software Defined Networking (SDN) controller for configuring an SDN network for Hash Routing Control (HRC) comprises: transmitting a features request message to a SDN switch/router requesting information disclosing hash routing control (HRC) features of the switch/router; and receiving in response to the features request a features reply message, the features response message including an HRC information element (IE) disclosing the HRC capabilities of the switch/router.
The preceding embodiments may further comprise wherein the features reply message discloses that the switch/router is HRC capable.
One or more of the preceding embodiments may further comprise: transmitting a flow table modification message to the switch/router disclosing a flow entry in a hash table maintained at the switch/router.
One or more of the preceding embodiments may further comprise wherein the flow entry modification message includes at least one of (1) an IE disclosing the number of hash routing descriptions entries included in the flow entry modification message, (2) an IE specifying a method for extracting hash function inputs from a data packet, (3) an IE specifying how a hash function is calculated from the hash function inputs, and (4) an IE specifying a range of hash function outputs to which the flow entry applies.
In another embodiment, a method implemented in a Software Defined Networking (SDN) switch/router comprises: maintaining a first flow table for routing data packets in a SDN network; maintaining a second flow table for routing the data packets according to a Hash Routing control (HRC); receiving from an SDN controller a flow table modification message defining a change to one of the first and second flow tables, the flow table modification message identifying a hash function and identifying a condition applicable to a hash value calculated using the hash function; and updating the first or second flow table according to the flow entry modification message.
The preceding embodiments may further comprise wherein the condition applicable to the hash value comprises a range of hash values.
One or more of the preceding embodiments may further comprise wherein the first flow table includes flow entries including a plurality of match descriptors for matching fields in the data packets and wherein the flow table modification message is for the first flow table and includes a first apply action instruction to execute a processing of the data packets received at the switch/router through the hash function using certain information elements from the data packets as inputs and forwarding to the second flow table on a condition that certain of the match descriptors match certain matching fields in the data packets.
One or more of the preceding embodiments may further comprise wherein the second table includes flow entries, each including a range match descriptor and a second apply action, the range match descriptor comprising a range of values, and the second apply action comprising an action to be performed on a condition that the hash value calculated using the hash function is within the range specified by the range match descriptor.
One or more of the preceding embodiments may further comprise wherein the first apply action instructions include: a function identifier identifying the hash function; and a table identifier identifying the second table.
One or more of the preceding embodiments may further comprise wherein the first apply action instruction further sets the result of the hash function as metadata.
One or more of the preceding embodiments may further comprise wherein the first apply action instruction further includes: a metadata mask identifying a part of a metadata field into which the result of the function is placed.
In another embodiment, a method implemented in a Software Defined Networking (SDN) controller, the method comprises: transmitting a flow table modification message to a SDN switch/router, wherein the flow table modification message defines a change to a flow table, identifies a hash function, and identifies a condition applicable to a hash value calculated using the hash function.
The preceding embodiment may further comprise wherein the flow table modification message includes an apply action instruction to execute a forwarding of a data packet received at the switch/router to a second flow table on a condition that certain of the match descriptors match certain matching fields in the data packet.
Throughout the disclosure, one of skill understands that certain representative embodiments may be used in the alternative or in combination with other representative embodiments.
Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer readable medium for execution by a computer or processor. Examples of non-transitory computer-readable storage media include, but are not limited to, a read only memory (ROM), random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WRTU, UE, terminal, base station, RNC, or any host computer.
Moreover, in the embodiments described above, processing platforms, computing systems, controllers, and other devices containing processors are noted. These devices may contain at least one Central Processing Unit (“CPU”) and memory. In accordance with the practices of persons skilled in the art of computer programming, reference to acts and symbolic representations of operations or instructions may be performed by the various CPUs and memories. Such acts and operations or instructions may be referred to as being “executed”, “computer executed” or “CPU executed”.
One of ordinary skill in the art will appreciate that the acts and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU. An electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to or representative of the data bits.
The data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile (“e.g., Read-Only Memory (”ROM″)) mass storage system readable by the CPU. The computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system. It is understood that the representative embodiments are not limited to the above-mentioned memories and that other platforms and memories may support the described methods.
No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. In addition, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the terms “any of” followed by a listing of a plurality of items and/or a plurality of categories of items, as used herein, are intended to include “any of,” “any combination of,” “any multiple of,” and/or “any combination of multiples of” the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items. Further, as used herein, the term “set” is intended to include any number of items, including zero. Further, as used herein, the term “number” is intended to include any number, including zero.
Moreover, the claims should not be read as limited to the described order or elements unless stated to that effect. In addition, use of the term “means” in any claim is intended to invoke 35 U.S.C. §112, ¶116, and any claim without the word “means” is not so intended.
Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs); Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine.
A processor in association with software may be used to implement a radio frequency transceiver for use in a wireless transmit receive unit (WRTU), user equipment (UE), terminal, base station, Mobility Management Entity (MME) or Evolved Packet Core (EPC), or any host computer. The WRTU may be used m conjunction with modules, implemented in hardware and/or software including a Software Defined Radio (SDR), and other components such as a camera, a video camera module, a videophone, a speakerphone, a vibration device, a speaker, a microphone, a television transceiver, a hands free headset, a keyboard, a Bluetooth® module, a frequency modulated (FM) radio unit, a Near Field Communication (NFC) Module, a liquid crystal display (LCD) display unit, an organic light-emitting diode (OLED) display unit, a digital music player, a media player, a video game player module, an Internet browser, and/or any Wireless Local Area Network (WLAN) or Ultra Wide Band (UWB) module.
Although the invention has been described in terms of communication systems, it is contemplated that the systems may be implemented in software on microprocessors/general purpose computers (not shown). In certain embodiments, one or more of the functions of the various components may be implemented in software that controls a general-purpose computer.
In addition, although the invention is illustrated and described herein with reference to specific embodiments, the invention is not intended to be limited to the details shown. Rather, various modifications may be made in the details within the scope and range of equivalents of the claims and without departing from the invention.
This application takes priority of U.S. Provisional Patent Application No. 61/969,566 and U.S. Provisional Patent Application No. 61/866,579, the contents of which are incorporated as if fully set forth herein.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US14/50363 | 8/8/2014 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 61866579 | Aug 2013 | US | |
| 61969566 | Mar 2014 | US |
