The present invention pertains to the field of networks. More particularly, this invention relates to designing interconnect fabrics of networks.
A wide variety of networks may be viewed as a set of source nodes that communicate with a set of terminal nodes via an interconnect fabric. For example, a storage area network may be arranged as a set of computers as source nodes which are connected to a set of storage devices as terminal nodes via an interconnect fabric that includes communication links and devices such as hubs, routers, switches, etc. Devices such as hubs, routers, switches, etc., are hereinafter referred to as interconnect devices.
The communication requirements of an interconnect fabric may be characterized in terms of a set of flow requirements. A typical set of flow requirements specify the required communication bandwidth from each source node to each terminal node. The design of an interconnect fabric usually involves selecting the appropriate arrangement of physical communication links and interconnect devices and related components that will meet the flow requirements.
Prior methods for designing interconnect fabrics are usually based on manual design techniques. Such manual techniques may be based on heuristics and/or prepackaged structures. Unfortunately, such techniques are usually error prone and may result in an over-provisioned interconnect fabric that is more expensive than needed to meet the flow requirements. Moreover, such techniques are usually time-consuming and may require expert intervention, thereby increasing design costs.
A technique is disclosed for designing an interconnect fabric for communication between a set of source nodes and a set of terminal nodes. The technique may be used to efficiently and programmatically produce cost-effective interconnect fabrics over a range of design problems.
In one aspect, a method is provided for designing an interconnect fabric for communication between a set of source nodes and a set of terminal nodes. A design for an interconnect fabric is obtained that has an interconnect device layer. The layer is adjacent to either the set of source nodes or the set of terminal nodes. Flow sets that traverse the layer of interconnect devices are identified. A pair of the flow sets are merged thereby alleviating at least one port violation. The design may then be implemented.
The merging may add a second interconnect device layer to the design. The method may be repeatedly performed, adding additional interconnect device layers, until port violations are no longer present in the design
In another aspect, a system for designing an interconnect fabric for communication between a set of source nodes and a set of terminal nodes is provided. A design for an interconnect fabric has at least one interconnect device layer. A fabric design tool modifies the design for the interconnect fabric by identifying flow sets that traverse the layer of interconnect devices and merging a pair of the flow sets, thereby alleviating at least one port violation.
The fabric design tool may add a second interconnect device layer to the design by merging a pair of the flow sets. The fabric design tool may repeatedly add additional interconnect device layers to the design until port violations are no longer present in the design.
In a further aspect, a method of designing an interconnect fabric for communication between a set of source nodes and a set of terminal nodes is provided. A design for an interconnect fabric is obtained having a interconnect device layer adjacent to either the set of source nodes or the set of terminal nodes. Interconnect device layers are formed adjacent to either the set of source nodes or terminal nodes, and are, thus, added to the design, until the design satisfies a set of flow requirements between the source nodes and terminal nodes without port violations. To form the interconnect device layers, flow sets that traverse the interconnect device layer may be identified and pairs of the flow sets merged.
Other features and advantages of the present invention will be apparent from the detailed description that follows.
The present invention is described with respect to particular exemplary embodiments thereof and reference is accordingly made to the drawings in which:
At step 20, an arrangement of flow sets in the interconnect fabric is determined in response to a set of flow requirements for the source and terminal nodes. In one embodiment, step 20 is performed by generating a flow set for each flow specified in the flow requirements for the interconnect fabric.
Table 1 shows an example set of flow requirements for an interconnect fabric under design.
The flow requirements in this example specify three source nodes (source nodes 40–44 in the figures below) and three terminal nodes (terminal nodes 50–54 in the figures below). If an interconnect fabric is to meet the flow requirements, it must contain communication paths between all pairs of the source and terminal nodes 40–44 and 50–54 having positive flow requirements and must have sufficient bandwidth to support all of the flow requirements simultaneously.
In one embodiment, the source nodes 40–44 are host computers and terminal nodes 50–52 are storage devices and the bandwidth values a–h are numbers expressed in units of megabits per second. Thus, the interconnect fabric under design may be storage area network.
In other embodiments, there may be multiple flow requirements between a given source and terminal node pair. In such embodiments, the cells of Table 1 would contain a list of two or more entries.
At step 22, port violations which are associated with the arrangement of flow sets in the interconnect fabric are determined. Port violations are determined for each source node 40–42 and each terminal node 50–52. In general, the number of port violations for a node is equal to the sum, over all flow sets, of the number of required physical communication links to the node from that flow set, minus the number of available ports in the node because each flow set may require one or more physical communication links to a given source or terminal node in the network. In this example, the number of port violations for a node is equal to the number of flow sets connected to the node minus the number of available ports in the node because each flow set is carried by one physical communication link in the interconnect fabric.
In this example, each source node 40–42 and each terminal node 50–52 has two available ports for connections to the interconnect fabric. Therefore, the source node 40 has a port violation of one since each of its three flow sets requires one physical communication link to the source node 40 and the source node 40 has only two available ports. Similarly, the source nodes 42–44 have port violations of one and zero, respectively, and the terminal nodes 50–54 have port violations of zero, one, and one, respectively, in the interconnect fabric. In other examples, the number of available ports on the source nodes 40–42 and the terminal nodes 50–52 may differ and the number of physical communication links required by a flow set on a given source or terminal node it connects to may exceed one.
At step 24, at least one of the port violations is alleviated by merging a pair of the flow sets. Step 24 initially involves selecting the pair of flow sets in the current interconnect fabric that are to be merged. Initially, a candidate pair of flow sets is chosen that would alleviate the port violation on a node with the greatest port violation if merged. If there is more than one such candidate pair then one of the candidate pairs that alleviates a port violation on a node having the next greatest port violation is chosen from among them. If there is more than one such candidate pair then a pair of them that would be least costly to merge is chosen. The cost of merging two candidate pairs may be determined by choosing the least expensive interconnect device that is feasible for the merged flow set.
In the current state of the example interconnect fabric shown in
The candidate pairs of flow sets considered at step 24 must be feasible to merge. An example of a pair of flow sets that is not feasible to merge is a pair for which an interconnect device of sufficient bandwidth is not available. For example, a flow set having 60 units of bandwidth cannot be merged with a flow set having 50 units of bandwidth if the highest bandwidth interconnect device available is 100 units. Another example of a pair of flow sets that is not feasible to merge is a pair that would exceed the available ports on every available interconnect device of the resulting flow set. Candidate pairs that are not feasible to merge are bypassed at step 24 in favor of other candidate pairs.
If port violations still exist in the interconnect fabric after step 24, then another candidate pair of flow sets is selected and merged in a repeat of step 24. The method 200 loops through steps 22–24 in an iterative fashion until all port violations are eliminated or until no further merges are feasible.
In this embodiment, each flow set has at most one interconnect device associated with it and all of the flows for the flow set are routed through that interconnect device or a single communication link if possible. When a pair of flow sets are merged, their existing interconnect devices, if any, are replaced by a single interconnect device which may be of a different type than the replaced interconnect devices. Usually, the interconnect device that results in the least expensive routing of the flows in the flow set is chosen from among those interconnect devices that are feasible for the flow set.
The list of hosts and devices 110 may specify the hosts and devices which are to be interconnected by the interconnect fabric design 124.
The list of fabric node types 112 may specify available interconnect devices, such as hubs, routers, switches, etc.
The link type data 114 may specify a list of available communication links that may be employed in the interconnect fabric design 124 and any relevant constraints. There are numerous examples of available communication links including fiber optic links, fibre channel links, wire-based links, and links such as SCSI as well as wireless links.
The flow requirements data 116 may specify the desired flow requirements for the interconnect fabric design 124. The desired flow requirements may include bandwidth requirements for each pairing of the source and terminal nodes.
The port availability data 118 may specify the number of communication ports available on each source node and each terminal node and each available interconnect device.
The bandwidth data 120 may specify the bandwidth of each host and device port and each type of fabric node and link.
The cost data 122 may specify costs associated with the available communication links and interconnect devices that may be employed in the interconnect fabric design 124. The cost data 122 may also specify the costs of ports for source and terminal nodes and interconnect devices. Other relevant costs may also be indicated.
The interconnect fabric design 124 generated by the fabric design tool 100 includes a list of the physical communication links and interconnect devices and ports, etc. and may include cost data.
In some cases, an existing design for an interconnect fabric may not be entirely satisfactory. For example, the method 200 generally results in a fabric design of a single layer in which there are no links between device nodes. Under certain circumstances, a single-layer fabric may not eliminate all of the port violations. In which case, the method 200, by itself, may not result in a fabric design in which there are no port violations. This can occur due to various factors, such as flow requirements and port availability limits at the source, terminal and/or device nodes. Other design techniques, such as manual or other methods, may also result in a design for an interconnect fabric which is not entirely satisfactory.
As an example of design in which there exist port violations, assume that the set of design information 130 includes the flow requirements as given in Table 1 and also includes additional constraints on port availability. The additional port availability constraints, for this example, may be that instead of two ports, the source node 40 and the terminal node 52 each have only one port and that available interconnect devices cannot exceed four ports. With these constraints, the configuration of
Returning to
Thus, in one embodiment, the present invention addresses remaining port violations in an obtained interconnect fabric by recursively generating additional layers of interconnect fabric nodes. For port violations at source nodes, the problem (i.e. the current fabric configuration and the applicable design information) may be recast such that the device nodes are treated as the terminal nodes. Then, one or more additional layers of device nodes may be inserted between the source nodes and the device nodes to relieve the port violations at source nodes. This results in links between device nodes and, thus, increases the number of layers in the interconnect fabric. Similarly, for terminal port violations, the problem may be recast such that the device nodes are treated as the source nodes. Then, one or more additional layers of device nodes may be inserted in between the device nodes and the terminal nodes to relieve the terminal node port violations. This also results in links between the device nodes and, thus, increases the number of layers in the interconnect fabric.
In the step 302, a determination may be made as to whether there exist any port violations at source nodes. Assuming there is one or more such port violations, step 304 may be performed. As mentioned, for the example of
The dummy nodes may be thought of as being logically equivalent to a link or to a repeater node which has two ports. The dummy nodes need not actually be inserted into the fabric as physical devices, but are useful constructs to act as link terminators in the layer 400. It will be apparent, however, that the present invention may be implemented without the use of such dummy nodes. In which case, flow sets which traverse the layer 400 are merged for alleviating port violations without regard as to whether the corresponding links actually terminate in the layer 400. More particularly, flow sets between the layer 400 and source nodes may be merged for alleviating port violations at the source nodes and flow sets between the layer 400 and terminal nodes may be merged for alleviating port violations at terminal nodes.
Also in the step 304, the dummy nodes and device nodes in the layer 400 may be recast as terminal nodes. Thus, all of the flows which pass through the layer 400 may be treated as simply terminating in the layer 400. In the example of
Then, in a step 304, an additional layer of device nodes may be inserted between the source nodes and the layer currently closest to the source nodes. In the example of
Assuming the method 200 of
In the example of
The step 306 may terminate similarly to the method 200 of
Next, the determination step 302 may be repeated in which a determination is made as to whether there exist any port violations at the source nodes. If violations remain, then the steps 304 and 306 may be repeated to add another layer between the prior layer and the source nodes. This process may be performed recursively, each time adding a layer of devices to the interconnect fabric and reducing the number of port violations at the source nodes. Each new layer is added between the prior layer and the source nodes. Thus, in the example, if another layer were required, the layer of devices that includes the device 170 would next be recast as the terminal nodes.
In the example of
Once all of the port violations at the source nodes have been eliminated, one or more layers of device nodes may be added adjacent to the terminal nodes in order to alleviate any port violations at the terminal nodes. Thus, after a negative determination in the step 302, a step 308 may be performed.
In the step 308, a determination may be made as to whether there are any port violations at the terminal nodes. Assuming there is one or more such port violations, step 310 may be performed. As mentioned, for the example of
Thus, in the step 310, a layer of existing interconnect fabric devices that is adjacent to the terminal nodes that have port violations may be identified. In the example of
Thus, all of the flows which pass through the layer 400 may be treated as originating in the layer 400. In the example, the flow a may be treated as originating at dummy node 162, the flow d may be treated as originating at dummy node 164, the merged flow set having flows b and e and the merged flow set having flows c and f may be treated as originating at the device node 160, the flow g may be treated as originating in the dummy node 166 and the flow h may be treated as originating in the dummy node 168. The source nodes 40, 42 and 44 may be ignored in the step 310.
Then, in a step 312, an additional layer of device nodes may be inserted between the prior layer and the terminal nodes. The example of
Assuming the method 200 of
In the example of
The step 312 may terminate similarly to the method 200 of
Next, the determination step 308 may be repeated to determine whether there exist any additional port violations at the terminal nodes. If violations remain, then the steps 310 and 312 may be repeated to add another layer between the prior layer and the terminal nodes. This process may be performed recursively, each time adding a layer of devices to the interconnect fabric and reducing the number of port violations at the terminal nodes. Each new layer may be added between the prior layer and the terminal nodes. Thus, in the example, if another layer were required, the layer of devices that includes the device 172 would be recast as the source nodes.
In the example of
Once all of the port violations at the terminal nodes have been eliminated, the interconnect fabric design may be output from the method 300 in a step 316. The design may then be used to implement the interconnect fabric.
Assuming that dummy nodes were employed while forming the additional fabric layers, they may be removed in the step 316.
While the embodiment of
The foregoing detailed description of the present invention is provided for the purposes of illustration and is not intended to be exhaustive or to limit the invention to the precise embodiment disclosed. Accordingly, the scope of the present invention is defined by the appended claims.
This is a continuation-in-part of U.S. application Ser. No. 09/707,227, filed Nov. 6, 2000, the contents of which are hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
4920487 | Baffes | Apr 1990 | A |
5107489 | Brown et al. | Apr 1992 | A |
5113496 | McCalley et al. | May 1992 | A |
5138657 | Colton et al. | Aug 1992 | A |
5245609 | Ofek et al. | Sep 1993 | A |
5307449 | Kelley et al. | Apr 1994 | A |
5329619 | Pagé et al. | Jul 1994 | A |
5524212 | Somani et al. | Jun 1996 | A |
5581689 | Slominski et al. | Dec 1996 | A |
5634004 | Gopinath et al. | May 1997 | A |
5634011 | Auerbach et al. | May 1997 | A |
5651005 | Kwok et al. | Jul 1997 | A |
5802286 | Dere et al. | Sep 1998 | A |
5805578 | Stirpe et al. | Sep 1998 | A |
5815402 | Taylor et al. | Sep 1998 | A |
5831610 | Tonelli et al. | Nov 1998 | A |
5831996 | Abramovici et al. | Nov 1998 | A |
5835498 | Kim et al. | Nov 1998 | A |
5838919 | Schwaller et al. | Nov 1998 | A |
5857180 | Hallmark et al. | Jan 1999 | A |
5878232 | Marimuthu | Mar 1999 | A |
5970232 | Passint et al. | Oct 1999 | A |
6003037 | Kassabgi et al. | Dec 1999 | A |
6031984 | Walser | Feb 2000 | A |
6038219 | Mawhinney et al. | Mar 2000 | A |
6047199 | DeMarco | Apr 2000 | A |
6108782 | Fletcher et al. | Aug 2000 | A |
6141355 | Palmer et al. | Oct 2000 | A |
6148000 | Feldman et al. | Nov 2000 | A |
6157645 | Shobatake | Dec 2000 | A |
6195355 | Demizu | Feb 2001 | B1 |
6212568 | Miller et al. | Apr 2001 | B1 |
6247077 | Muller et al. | Jun 2001 | B1 |
6253339 | Tse et al. | Jun 2001 | B1 |
6331905 | Ellinas et al. | Dec 2001 | B1 |
6363334 | Andrews et al. | Mar 2002 | B1 |
6526420 | Borowsky et al. | Feb 2003 | B1 |
6539027 | Cambron | Mar 2003 | B1 |
6539531 | Miller et al. | Mar 2003 | B1 |
6557169 | Erpeldinger | Apr 2003 | B1 |
6570850 | Gutierrez et al. | May 2003 | B1 |
6594701 | Forin | Jul 2003 | B1 |
6598080 | Nagami et al. | Jul 2003 | B1 |
6603769 | Thubert et al. | Aug 2003 | B1 |
6611872 | McCanne | Aug 2003 | B1 |
6614796 | Black et al. | Sep 2003 | B1 |
6625777 | Levin et al. | Sep 2003 | B1 |
6628649 | Raj et al. | Sep 2003 | B1 |
6631128 | Lemieux | Oct 2003 | B1 |
6633909 | Barrett et al. | Oct 2003 | B1 |
6650639 | Doherty et al. | Nov 2003 | B1 |
6668308 | Barroso et al. | Dec 2003 | B1 |
6675328 | Krishnamachari et al. | Jan 2004 | B1 |
6687222 | Albert et al. | Feb 2004 | B1 |
6694361 | Shah et al. | Feb 2004 | B1 |
6697334 | Klincewicz et al. | Feb 2004 | B1 |
6697369 | Dziong et al. | Feb 2004 | B1 |
6697854 | Glassen et al. | Feb 2004 | B1 |
6701327 | Jones et al. | Mar 2004 | B1 |
6724757 | Zadikian et al. | Apr 2004 | B1 |
6744767 | Chiu et al. | Jun 2004 | B1 |
6757731 | Barnes et al. | Jun 2004 | B1 |
6766381 | Barker et al. | Jul 2004 | B1 |
6775230 | Watanabe et al. | Aug 2004 | B1 |
6778496 | Meempat et al. | Aug 2004 | B1 |
6804245 | Mitchem et al. | Oct 2004 | B1 |
6853970 | Gupta et al. | Feb 2005 | B1 |
6857027 | Lindeborg et al. | Feb 2005 | B1 |
20020083159 | Ward et al. | Jun 2002 | A1 |
20030065758 | O'Sullivan et al. | Apr 2003 | A1 |
20030145294 | Ward et al. | Jul 2003 | A1 |
20050021583 | Andrzejak et al. | Jan 2005 | A1 |
Number | Date | Country |
---|---|---|
WO-9617458 | Jun 1996 | WO |
Number | Date | Country | |
---|---|---|---|
20020083159 A1 | Jun 2002 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 09707227 | Nov 2000 | US |
Child | 10027564 | US |