This invention relates to the field of implicit or content routing in digital communications networks, and in particular to quality of service for content routing.
Content-based networks are described in A. Carzaniga, M. J. Rutherford, A. L. Wolf, A routing scheme for content-based networking, Department of Computer Science, University of Colorado, June 2003.
The field of “Implicit Routing” (or “content routing”) is an emerging networking technology. Implicit Routing is the act of forwarding customer data based on the content, rather than a networking header specifying an explicitly addressed destination.
A content router is a digital communications networking device which forwards content based on inspection of the contents of a message or document, rather than on an explicit destination address in the networking header of a packet or frame. An example of such a device is the 3200 multiservice message router from Solace Systems, Inc. Content routers must have connections between themselves so that they can communicate with each other and exchange both information needed to control the network, as well as to carry the content received from publishers from one content router to the next, in order to deliver it to the subscribers in the network that are interested in the content. In
A publisher is a computer, user or device that can insert content into the network. Another name commonly used in the literature is an event source or a producer. A publisher connects to a content router over a link, using a variety of techniques as explained above, and then the publisher can inject content into network 1. For example, link 41 connects publisher 11 to content router 2.
A subscriber is a computer, user or device that has expressed interest in some specific content. Another name commonly used in the literature is event displayers or consumers. A subscriber connects to a content router over a link, using a variety of techniques as explained above, and then the subscriber can receive content from the network 1. For example, link 42 connects subscriber 22 to content router 2.
The manner in which a content router learns of subscriptions from other routers in the network, and routes an incoming document to the correct set of egress links, is outside the scope of the present invention. One such scheme is described in our co-pending application Ser. No. 11/012,113 entitled “Implicit Routing in Content Based Networks”, as well as to “A. Carzaniga, M. J. Rutherford, A. L. Wolf, A routing scheme for content-based networking, Department of Computer Science, University of Colorado, June 2003”, the contents of both which are herein incorporated by reference.
In the prior art, content routers typically utilize a single link between a pair of routers, where link is taken to be a logical construct which can equate to a physical link, a virtual circuit, a TCP connection, etc. All traffic to be forwarded between the two routers utilizes the same link.
In
While the prior art addresses the publish/subscribe service, allowing publishers and subscribers to remain decoupled from one another, it does not address the issue of quality of service. Again considering the example network of
The invention provides a method of content-routing or implicit routing across a plurality of content routers that provides differentiated quality of service for various content being routed.
According to the present invention there is provided a method of handling documents in a content-based network including a plurality of content-based routers wherein one or more of said content-based routers serve as ingress routers, said method comprising storing a set of rules defining the assignment of priority to documents in accordance with attributes of said documents; assigning a priority to an incoming document from a publisher at a said ingress router by comparing an attribute of an incoming document against said stored set of rules; and routing said incoming document in accordance with its assigned priority to provide quality-of-service control in the content-based network.
The invention allows this quality of service differentiation to occur based on any of the content of a given document to be routed, and can interoperate with a variety of networking technologies that is used to interconnect the content routers.
In another aspect the invention the invention provides a content-based network comprising a plurality of content-based routers interconnected by links, wherein one or more of said content-based routers constitute ingress routers, wherein said ingress routers are configured to store a set of rules defining the assignment of priority to documents in accordance with their attributes; wherein said ingress routers are configured to assign a priority to incoming documents by comparing an attribute of an incoming document against said stored set of rules; and wherein said content-based routers are configured to route said incoming document in accordance with its assigned priority to provide quality-of-service control in the content-based network.
In yet another aspect the invention provides a content-based router for use as an ingress router in a content routed network, comprising a memory for storing a set of rules defining the assignment of priority to documents in accordance with their attributes; and a processor programmed to compare an attribute of an incoming document against said stored set of rules and assign a priority to said incoming document based on said comparison, whereby said document can be routed through the network in accordance with its assigned priority to provide quality-of-service control in the network.
The invention will now be described in more detail, by way of example only, with reference to the accompanying drawings, in which:—
In an exemplary embodiment, a content router routes documents formatted as Extensible Markup Language (XML) (refer to Extensible Markup Language (XML) 1.0 (Third Edition)”, W3C Recommendation 4 Feb. 2004, W3C (World Wide Web Consortium)) and utilizes subscriptions based on XML Path Language (XPath) (refer to reference “XML Path Language (XPath) Version 1.0”, W3C Recommendation 16 Nov. 1999, W3C (Word Wide Web Consortium)). Publishers connect to the content router via HTTP over TCP, although other connection methods are possible such as SMTP, FTP, TCP, etc.
In
Similarly, content router 2 sends document 61B to subscriber 22 over link 42 using HTTP, such as by using an HTTP POST message. The subscriber 22 processes the message and then sends an HTTP response back to the content router 2 indicating whether the POST operation was successful or not. The HTTP request (carrying the document) from the content router to the subscriber, and the HTTP response from the subscriber back to the content router, are carried over the same TCP link 42.
A link between a pair of content routers could actually be realized by a pair of TCP connections, as is shown in
An alternative approach is to use a single TCP connection between content routers, and to multiplex HTTP request messages and HTTP response messages in a given direction on the link.
In all subsequent figures, a single link is shown between content routers to carry documents in both directions. However, such a link can be actually implemented using a pair of links to separate HTTP requests in each direction as described above.
Assignment of Priority to a Received Document
The first step in providing quality of service is to determine the required quality of service of a document received from a publisher. This determination indicates the quality of service to be given to the document as it is routed from the publisher towards all content routers that service interested subscribers and then finally to deliver the document to the subscriber(s). This is done by applying a quality of service filter to each incoming document, expressed using XPATH expressions. In the description, the term “document” refers to a piece of content that is being content-routed by the content-routing network. It can be any type of content that is produced by a content producer and delivered to subscribers.
The quality of service of a document is expressed as a priority value, with four possible values 0 through 3. The definition of these values is shown in Table 1. Four priority levels are provided to balance the desire for more levels of differentiation with the desire to simplify the overall network configuration and control. A different number of priority values could be utilized without changing the present invention.
Note that in addition to matching the content of the document against a set of priority rules, a content router must also match the document against a set of subscriptions in order to route the document towards locally attached subscribers, or towards other content routers who in turn will route the document towards the interested end subscribers. This matching could be done as a separate step, or could be done in parallel with the priority assignment of block 78. In the preferred embodiment it is done as part of block 78, i.e. this block both assigns priorities to documents and determines where the document must be routed based on subscriptions. Block 79 then determines what egress links the document must be sent on given this information. Note that the general method of content routing is outside the scope of this invention.
An example of the data held in the Document Matching Priority Rules 80 is shown in Table 2. Each rule consists of three elements: a publisher ID 90, indicating which publisher this rule applies to, where a special value of * in the figure indicates a wildcard matching any publisher; an XPath Expression 91, indicating the specific XPath expression that is to be matched against the content of the received document; and an assigned priority 92, indicating the priority to be assigned to the document if the rule matches. In the example rule configuration of
An example of the information held in the Default Priority Rules 81 is shown in table 3. Each entry consists of two elements: a publisher ID 100, indicating which publisher this rules applies to, where a special value of * in the figure indicates a wildcard matching any publisher; and an assigned priority 101 indicating the priority to be assigned to the document if the rule matches. The assigned priority 101 takes on one of the values as defined in table 1. Note that in the default priority rules, a given publisher ID may only appear zero or one times. In addition, there is always exactly one entry for the wildcard (*) publisher ID.
The algorithm used by the Priority Assignment Block 78 is shown in
In block 78, a series of XPath expressions must be matched against the content of the document to determine whether the rule matches the received document. The method of applying an XPath expression against an XML document is well known in the art, and is widely available in both commercial and open-source software packages. In the simplest case, each XPath expression could be applied to the document one-by-one to see which match. However, in the presence of a large number of rules, the matching should be done in parallel for maximum performance. Two example algorithms known in the art for the application of a large number of XPath expressions against XML documents are as follows:
This algorithm of
An exemplary XML document is described below. If the document was to be received from publisher 71, the document matching priority rules of Table 2 would be consulted. Rules 93, 94, and 95 would be consulted since they apply to publisher 71. For the example document, rules 93 and 94 would match the content of the document. Then, rule 93 would be selected since it has the highest priority of matching rules. Thus, the document would be assigned a priority of 2.
If the same document was to be received from publisher 72, rules 96 and 97 would be considered, since they apply to publisher 72. Neither matches the document, so then rules 98 and 99 would be considered, since they are for the wildcard (*) publisher ID. Rules 98 and 99 would be found to match, and so the document would be assigned a priority of 1 (the highest priority of all matching rules).
Exemplary XML Document
Tagging the Document with the Assigned Document Priority
The document priority of the document that has been assigned by the ingress content router is sent, along with the document, to other content routers in the network. This is done so that:
The manner in which the priority of a document is signalled from one content router to another is described in our copending patent application Ser. No. 11/012,168. Note that a downstream content router, according to a local policy, could choose to over-ride the priority already assigned to the document by an upstream content router, and re-compute a new priority for the document according to local policies. This can be useful if a document is handed off from administrative domain to another, or if the downstream content router has policy information that is not available to the upstream content router that affects assignment of document priority.
Routing Based on Document Priority
Instead of utilizing a single link between content routers as per the prior art of
Each link is provisioned to carry a range of document priorities. A given priority can only be assigned to a single link between a given pair of routers. A link can be provisioned to carry one or more priorities. An exemplary assignment of priorities to the links of
As part of the content routing functionality, a source-based spanning tree is constructed with each content router at the root. The method for doing this is described in the patent application “Implicit Routing in Content Based Networks”. The addition of priority-based quality of service support, and the use of multiple links between routers as per
Given the links of Table 4, the per-priority spanning trees with content router 130 as the source router (tree root) is shown in
Armed with this information, the document routing block 79 of
When a link fails, the routing protocol distributes this notification as per the above-mentioned patent application Ser. No. 11/012,113. This allows the spanning trees for each source and each priority to be re-computed to reflect the new network topology. For example, if link 137A was to fail, the spanning tree for source content router 130, priority 0 would be recomputed to be that shown in
One or more link failures could result in a destination router being unreachable from the point of view of a given source router for a given document priority. To avoid this occurrence, the spanning tree for each priority is constructed using the algorithm shown in
The algorithm of
The algorithm of
As an example, in Table 4, consider the failure of links 134D and 135C. The spanning tree for source content router 130, priority 3 would become that shown in
It should also be noted that a more direct link for only a subset of the priorities could be added between a given pair of content routers. For example, in
Other policies are also possible other than the preferred embodiment described above. For example, the routing algorithm may wish to take into account other metrics or policies in addition to the document priority. For example, there could be a policy to limit the number of hops a document will need to take to reach a given destination content router, even if a lower or higher priority link must be used as a result.
Alternative Routing Methodology
An alternative embodiment to routing based on the document priority is as follows. Multiple links between content routers is still utilized as per the examples in
Another hybrid approach is to have the routing protocol only report one link to represent the link set between a pair of content routers, but to include the attribute that describes the set of priorities supported by the aggregate link. This has the advantage of still only requiring the reporting of a single link between content routers. If one of the constituent links changes states (e.g. fails), the aggregate link is re-distributed by the routing protocol with the set of supported priorities updated. In this manner, each router knows what priorities are currently supported over each link, and a per-source, per-priority graph can be constructed as described above, but the graphs will reference the aggregate links.
Queuing Based on Document Priority
Internal to the content router, once a document priority has been assigned to a document, it can be used to internally prioritize the treatment of the document within the content router. For example, when documents are queued to egress the router, per-priority queues can be utilized for each egress link. This allows queuing policies to control the order in which documents are emitted by the content router. For example, high priority documents should be sent preferentially over low-priority documents. Various queue strategies can be employed, such as weighted fair queuing. Egress bandwidth for each document priority can also be measured and controlled, ensuring that each priority gets a controlled percentage of the available bandwidth, for example. Egress priority queuing can be applied when determining the “next” link to service among all links with documents ready to transmit; and when queuing documents for transmission into a link when this link handles multiple priorities.
In addition, if the router is running low on resources, higher priority documents can be given precedence. For example, if the router is running out of memory 191 resources to buffer documents waiting for transmission, lower priority documents can be moved out of memory 191 onto persistent storage 193, to free up memory 191 resources for higher priority documents. Later, when memory 191 resources become available, the lower priority documents can be handled.
Priority servicing can also be employed by each router when receiving documents from other routers and deciding the order of processing and for transmitting to other routers or subscribers. For example, a receiving router can preferentially service high priority links from other router over lower priority links. In
Furthermore, egress priority queuing can be applied to documents within a connection when delivering documents over a given connection. For example, in reference to
Referring to
Link 228 only carries priority 3 documents, with associated queue 227. Link 230 only carries priority 2 documents with associated queue 229. Link 232 only carries priority 1 document, with associated queue 231, and link 234 only carries priority 0 documents, with associated queue 233. In this case the queue servicing block 237 each has one queue to service.
Scheduling block 235 then schedules traffic from each link as they are multiplexed onto port 236. Again, different scheduling techniques can be used, such as weighted round robin, strict priority, or various calendaring mechanisms as known in the art. Note that port 236 could be further broken down into logical ports, e.g. through the use of virtual LANs in the case of an Ethernet port, or channels in a channelized TDM or SONET interface, etc. In this case, another layer of queue servicing and traffic shaping can be introduced, to shape traffic into virtual LANs or channels, and then schedule traffic from each virtual LAN or channel onto the physical port.
Mapping Document Priorities to Diffserv Code Points
Differentiated Services (Diffserv) is defined in RFC 2474, “Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers”, December 1998, The Internet Society (IETF).
If the content-routed network is over-laid on to an IP network that supports Diffserv, then document priorities can be mapped to DiffServ code points to affect an end-to-end quality of service across the content-routed network, taking advantage of the Differentiated Services offered by the underlying IP network. This is illustrated in
The content routers are provisioned to have a configurable Differentiated Services Code Point (DSCP), in the range of 0 to 63 as per RFC 2474, per logical link, such as 150 and 151. When the logical link 150 and 151 are established (e.g. via TCP in the preferred embodiment), the provisioned Diffserv code point is set in each IP packet for the flow. This allows the underlying IP network (routers 142, 143 and 144) to treat each TCP flow 150 and 151 with the quality of service appropriate for the signalled Diffserv code point. This can include queuing disciplines, route used, etc. as is known in the art. For example, in
As was explained above in reference to
In
Table 5 gives an example configuration for content router 140 of the connections that it is to have to neighbour content router 141. In Table 5, entry 152 indicates that a TCP connection is to be made to far-end IP address 10.10.10.2, far-end TCP port number 80. This connection will be used to send documents to content router 141 that have a priority of 2 or 3. The DSCP value to be used for this TCP connection is 18; that is, content router 140 will mark each IP packet that is sent as part of this TCP flow with a DSCP value of 18. Entry 153 indicates that another TCP connection is also to be made to far-end IP address 10.10.10.2, far-end TCP port number 80. This connection will be used to send documents to content router 141 that have a priority of 0 or 1. The DSCP value to be used for this TCP connection is 10.
Table 6 gives an example configuration for content router 141 of the connections that it is to have to neighbour content router 140. In Table 6, entry 154 indicates that a TCP connection is to be made to far-end IP address 10.10.10.1, far-end TCP port number 80. This connection will be used to send documents to content router 140 that have a priority of 2 or 3. The DSCP value to be used for this TCP connection is 18; that is, content router 141 will mark each IP packet that is sent as part of this TCP flow with a DSCP value of 18. Entry 155 indicates that another TCP connection is also to be made to far-end IP address 10.10.10.1, far-end TCP port number 80. This connection will be used to send documents to content router 140 that have a priority of 0 or 1. The DSCP value to be used for this TCP connection is 10.
With the example configuration of Table 5 and Table 6, content router 140 will establish two TCP connections to content router 141. Since both connections go to the same destination port number on content router 141, it does not know, just by accepting the connection, what document priorities is to be carried by which connection. The DSCP code point is associated with the entire TCP flow, and so has to be set for each TCP flow. On the content router establishing the TCP connection, it knows the DSCP value to use from the configuration data it used to create the connection (i.e. Table 5 and Table 6). However, the content router accepting the TCP connection does not know which DSCP value to use. This information is sent from the content router establish the TCP connection to the content router accepting the TCP connection as follows. After the connection is established, the content router that established connection sends an initial HTTP message with information indicating the priority or priorities supported by the link. For example, such a message can be
which is sent initially on a link which supports priorities 2 and 3. The message is an HTTP POST message which carries an XML document which describes the configuration of the link. In the XML document, the <XmlLinkProtocol> tag indicates that the message describes the configuration of the link, and value or values inside the <Priority> tag describe the priorities supported by the link. If more than one priority is supported, a comma-separated list of values is provided. This message is sent over a link that supports priorities 2 and 3, such as the link described by the configuration 152 of Table 5 and 154 of Table 6.
The following message can be sent over a link that supports priorities 0 and 1, such as the link described by the configuration 152 of Table 5 and 155 Table 6:
Upon receiving such a message, the receiving content router uses the priority value or values, and consults its link configuration data (such as in Table 6 for content router 141) to determine the DSCP value it is to use for transmissions it makes on the TCP connection. It then sets the DSCP for the TCP link. Note that normally the DSCP value will be the same in each direction of transmission on a given TCP session, but this is not required, and this method allows each content router to use its own configuration to set the DSCP value.
Note that when using HTTP to transfer documents between content routers, the HTTP request, which carries the HTTP document, is responded to with an HTTP response, as is known in the art. For example, in
The act of mapping an XML document to a document priority based on matching rules on the content, as opposed to just using header information such as source and destination addresses as in non-content networks, performing priority queuing on ingress and egress of the router, and then routing the document in the content-routed network based on this document priority, and furthermore mapping document priority to a DSCP for treatment by the underlying IP network, allows quality of service to be delivered throughout the network, even though the underlying IP network does not understand how to process the contents of XML documents.
Diffserv code points can also be used in conjunction with an MPLS network. This is illustrated in
Note that in
It should also be noted that multiple flows, each with a different DSCP, can be mapped into the same LSP, using an E-LSP as per RFC 3270, “MPLS Support of Differentiated Services”, May 2002, The Internet Society (IETF). With an E-LSP, different qualities of service can still be offered within a single LSP, although resiliency (i.e. whether the LSP is re-routed or not) is common.
Mapping Document Priorities Directly to MPLS
Another method to provide QoS is to have the content routers be part of the MPLS network and directly signal LSPs using a label distribution protocol, as is known in the art. This is illustrated in
Other Network Technologies
It will be appreciated by one skilled in the art that there exists a plurality of other networking technologies for which content prioritization of documents can be mapped onto to achieve end-to-end quality of service. For example,
Another example is a Frame Relay network, where the logical links between content routers is done via a Frame Relay circuit, which again can be done via a signalled connection or a provisioned connection. Each Frame Relay circuit has an associate traffic descriptor to provide quality of service.
For Ethernet interfaces, Ethernet priority can be used tag each link between content routers with an appropriate Ethernet priority, which allows Ethernet switching equipment to give higher precedence to higher priority flows.
It will be appreciated by persons skilled in the art that many variants of the invention are possible.
All references mentioned above are herein incorporated by reference.
This application claims the benefit under 35 USC 119(e) of prior U.S. application No. 60/588,797, filed Jul. 19, 2004, the contents of which are herein incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
5430720 | Larsson et al. | Jul 1995 | A |
5974417 | Bracho et al. | Oct 1999 | A |
6104713 | Nagami et al. | Aug 2000 | A |
6298061 | Chin et al. | Oct 2001 | B1 |
6400954 | Khan et al. | Jun 2002 | B1 |
6434155 | Jones et al. | Aug 2002 | B1 |
6600724 | Cheng | Jul 2003 | B1 |
6798746 | Kloth et al. | Sep 2004 | B1 |
6816488 | Merchant et al. | Nov 2004 | B1 |
6901079 | Phadnis et al. | May 2005 | B1 |
6961342 | Uzun et al. | Nov 2005 | B1 |
7042842 | Paul et al. | May 2006 | B2 |
7068645 | Phadnis et al. | Jun 2006 | B1 |
7177295 | Sholander et al. | Feb 2007 | B1 |
20010056500 | Farber et al. | Dec 2001 | A1 |
20020016825 | Uchida et al. | Feb 2002 | A1 |
20020018447 | Yamada et al. | Feb 2002 | A1 |
20020039352 | El-Fekih et al. | Apr 2002 | A1 |
20020097747 | Kirkby et al. | Jul 2002 | A1 |
20020129158 | Zhang et al. | Sep 2002 | A1 |
20020141393 | Eriksson et al. | Oct 2002 | A1 |
20030007455 | Kohzuki et al. | Jan 2003 | A1 |
20030048778 | Davison | Mar 2003 | A1 |
20030053464 | Chen et al. | Mar 2003 | A1 |
20030084021 | Dweck et al. | May 2003 | A1 |
20030099237 | Mitra et al. | May 2003 | A1 |
20030118026 | Kuhl et al. | Jun 2003 | A1 |
20030128708 | Inoue et al. | Jul 2003 | A1 |
20030133417 | Badt, Jr. | Jul 2003 | A1 |
20030135556 | Holdsworth | Jul 2003 | A1 |
20030142680 | Oguchi | Jul 2003 | A1 |
20030182234 | Degroot | Sep 2003 | A1 |
20030204449 | Kotas et al. | Oct 2003 | A1 |
20040010752 | Chan et al. | Jan 2004 | A1 |
20040044780 | Eastham | Mar 2004 | A1 |
20040057437 | Daniel et al. | Mar 2004 | A1 |
20040190526 | Kumar et al. | Sep 2004 | A1 |
20040210663 | Phillips et al. | Oct 2004 | A1 |
20040226002 | Larcheveque et al. | Nov 2004 | A1 |
20040240658 | Delaney et al. | Dec 2004 | A1 |
20040250059 | Ramelson et al. | Dec 2004 | A1 |
20040260683 | Chan et al. | Dec 2004 | A1 |
20050005021 | Grant et al. | Jan 2005 | A1 |
20050021548 | Bohannon et al. | Jan 2005 | A1 |
20050105560 | Mann et al. | May 2005 | A1 |
20050160171 | Rabie et al. | Jul 2005 | A1 |
20050243732 | Bitar et al. | Nov 2005 | A1 |
Number | Date | Country | |
---|---|---|---|
20060013230 A1 | Jan 2006 | US |
Number | Date | Country | |
---|---|---|---|
60588797 | Jul 2004 | US |