Hierarchical resource management with maximum allowable allocation boundaries

Information

  • Patent Grant
  • 5889956
  • Patent Number
    5,889,956
  • Date Filed
    Thursday, July 18, 1996
    28 years ago
  • Date Issued
    Tuesday, March 30, 1999
    25 years ago
Abstract
A system for managing resources such as buffers and bandwidth which are allocated to competing entities through two or more levels in a telecommunications network is disclosed. The system provides a tool to allocate resources for use by individual entities. Each entity may be assigned a Minimum.sub.-- Guaranteed variable and a Maximum.sub.-- Allowed variable. When an entity requests resources the system determines if the entity is using its respective minimum guaranteed resource allocation which is specified by the Minimum.sub.-- Guaranteed variable. If the entity is not using its respective minimum guaranteed resource allocation, the system allocates a resource unit to the requesting entity. The system also allows a requesting entity to use additional resource units above the resource allocation specified by the Minimum.sub.-- Guaranteed variable, provided such resource units are available. If the entity has reached its respective minimum guaranteed resource allocation, but has not reached the respective maximum allowed resource allocation specified by the Maximum.sub.-- Allowed variable and no intervening level is using its respective maximum allowed resource allocation, then a resource unit is allocated to the requesting entity.
Description

FIELD OF THE INVENTION
The present invention is generally related to resource management techniques and more particularly to resource management techniques which dynamically adjust to conditions at multiple levels to provide efficient use of resources in a telecommunications network.
RELATED APPLICATION
A claim of priority is made to provisional application Ser. No. 60/001,498 entitled COMMUNICATION METHOD AND APPARATUS, filed Jul. 19, 1995.
BACKGROUND OF THE INVENTION
Basic techniques for management of finite resources such as bandwidth and buffers in multiple level hierarchical systems such as asynchronous transfer mode ("ATM") networks are generally known. The techniques include provision of sufficient resources to satisfy theoretical maximum demands and allocation of resources based on statistically calculated demands.
Trying to provide sufficient resources for theoretical maximum demands is costly and inefficient. The technique is costly because the hardware which comprises the resources is costly. The technique is inefficient because actual use is typically considerably less than the theoretical maximum, and a portion of the provided resources are thus under-utilized.
On the other hand, allocation of resources at one level of a hierarchical system of resources based upon an expectation of how much resource may be required under some given set of conditions may improve efficiency and reduce the cost of resources. The current state of the art for hierarchical systems must attempt this on the basis of requests specifying either a maximum or a nominal resource requirement. Resources are granted based upon some statistical basis. However, this technique lacks the control and flexibility required to manage the resources in an effective manner at the various levels of the hierarchy. Several unsatisfactory modes of operation are experienced. The resources granted may not be sufficient to satisfy the requirements under all conditions and as a result, the required resources may not be available when needed and data is lost. The granting of excess resources, while aiding in the prevention of data loss, nullifies the cost savings that can be realized. Such systems do not provide a means to manage the hierarchical resources in a dynamic manner to accommodate changing conditions. Furthermore, such systems do not allow for the maintenance of a pool of resources that may be shared in a controlled manner by all of the requesters at the immediately lower level of the hierarchical resources.
The provisioning of resources for hierarchical services under the current state of the art is accomplished by a request from the lower level to the next higher level for a specific amount of resource. The basis of granting this request may be either in terms of a maximum requirement or as a statistical expectation of the resource that will be required. Resources so granted may prove inefficient or may provide unsatisfactory performance. Once granted, however, changes in this assignment of resources may be quite difficult to effect since they will typically involve negotiations and changes to the resources provided by the higher level to the other peer lower level users of the resources.
SUMMARY OF THE INVENTION
Allocation of resources is governed by resource usage levels associated with resource using entities: a maximum allowed usage level and a minimum guaranteed usage level. A method of hierarchical resource management for allocation of resource units to a level 0 entity from a level N entity and intervening levels, where N is greater than or equal to two, includes: assigning a first variable to one or more entities specifying a maximum amount of resource to be allocated to the respective entity; and processing a request from the level 0 entity for a resource unit by: determining whether actual use by the level 0 entity is less than the allocation specified by the assigned first variable, provided a first variable is assigned to the level 0 entity, and denying the request if actual use is not less than the allocation specified by the assigned first variable, otherwise; determining whether actual use at each intervening level is less than the allocation specified by the assigned first variable at such level, provided the first variable is so assigned, and denying the request if actual use is not less than the allocation specified by the assigned first variable at any intervening level, otherwise; providing a resource unit to the level 0 entity. A second variable specifying the minimum guaranteed resource allocation may also be assigned to each respective entity. Requests for resources by an entity are granted if the requested utilization level is less than the minimum guaranteed resource allocation.
The first and second variables provide a tool for maximizing overall resource utilization by controlling resource utilization by individual entities in a manner which was not previously possible. Efficient, flexible, controlled use of resources is provided by ensuring that each entity has at least its respective minimum guaranteed resource allocation and by allowing entities to use available resources in excess of their respective minimum guaranteed allocations, up to the amount specified by the first variable. When a level 0 entity requests a resource unit, that resource unit is provided if actual use by the level 0 entity is less than the assigned minimum guaranteed amount specified by the second variable. A policy can then be implemented to assure availability of the minimum guaranteed resource allocation by ensuring that the minimum guaranteed amount available at one level is greater than or equal to the sum of all minimum provided amounts at the level below. Once the entity has been allocated its respective minimum guaranteed amount of resource, additional available resource may be utilized up to the amount specified by the respective first variable. Hence, overall resource utilization, and thus efficiency, are maximized by controlling resource utilization by individual entities.
With respect to the provisioning of resources in hierarchical structures, this invention provides a superior service to that of the previous art. Requests for resources are now made in terms of two variables: one specifying the maximum resource that this requester could use (the maximum allowed), the other specifying the minimum resource that is required by this requester. The basis of granting a request is the availability of the minimum resource. If additional resources are available, then the requester will be allowed to use up to the maximum allowed. The higher level must be able to provide resources equal to the sum of the minimum resource requirements for the requests that have been granted. The service provided to the lower level users by the higher level may be improved by having resources available up to the sum of the maximum allowed for all the lower level requests that have been granted. Furthermore, it is also made clear that there is no additional benefit to be gained by supplying resources above the sum of the maximum allowed of all the granted requests. A similar method of requesting and granting of resources would be employed at each level of a hierarchical system of resources. Thus this invention allows resources to be assigned and used more effectively and efficiently at each level of a hierarchical system than would otherwise be possible.
In an alternative embodiment, the minimum guaranteed resources are statistically shared among resources using entities at higher levels of the resource hierarchy. When the minimum guaranteed portion is statistically shared, the method is modified to implement statistical sharing policies. More particularly, if the level 0 entity is operating below the associated minimum guaranteed resource allocation, then the entity can use resources above the respective minimum guaranteed resource allocations at higher levels. This policy allows entities to continue using resources even if the pool of minimum guaranteed resources has been consumed. Additionally, if the level 0 entity is operating above the associated minimum guaranteed resource allocation, the resources at higher levels are granted only if there are minimum guaranteed resources at all higher levels. This policy leaves resources available for other entities that have not consumed their associated minimum guaranteed resources. Such statistical sharing is advantageous when, for example, there are numerous resource users which will not all request resources at the same time.





BRIEF DESCRIPTION OF THE DRAWING
Other features and advantages will become apparent in light of the following detailed description of the drawing in which:
FIG. 1 illustrates bandwidth allocation in a telecommunications network;
FIG. 2 illustrates multiple level resource management;
FIG. 3 is a flow chart of a method for consuming a resource;
FIG. 4 is a flow chart of a method for freeing a resource; and
FIG. 5 is a block diagram which illustrates hierarchical memory management.





DETAILED DESCRIPTION OF THE DRAWING
FIG. 1 illustrates bandwidth allocation in a telecommunications network. The network portion 10 illustrates an entity requiring network bandwidth, such as a company. The network portion 10 may share bandwidth with other network portions 12 on a fiber-optic or other link 14. The network portion 10 by way of example may represent the network portion within a company and in the present example includes a number of other entities requiring access to network bandwidth such as an engineering group 16, a sales and marketing group 18, and administration group 20, a programming department 22, a hardware department 24, a marketing department 26, a sales department 28, an accounting department 30 and a personnel department 32. For explanatory purposes, the link 14 can provide a maximum of 200 million bits per second ("MBPS") of bandwidth. It will be appreciated that if each of the network portions 10, 12 in the network are allowed to appropriate bandwidth without restraint, some users may appropriate much more bandwidth than others, and some users may get too little or no bandwidth.
Control of overall bandwidth utilization is provided by controlling bandwidth utilization at the company, group and department levels. The company link portion 34 in the present example is limited to a maximum of 100 MBPS of bandwidth. That is, once 100 MBPS of bandwidth is utilized in the link 34 the network prevents entities within the company from obtaining additional bandwidth through the link 34. Similarly, in the present example the engineering group is limited to 70 MBPS of bandwidth 36 and the hardware department is limited to 30 MBPS of bandwidth 50. The programming department 22 is not directly limited to a bandwidth. However, it is not necessary to impose a bandwidth limitation on every department and/or group. While the programming department has no direct bandwidth limit in this example, the programming department is still limited by the engineering group bandwidth allocation 36. That is, the bandwidth use by the departments within a group cannot exceed the bandwidth allocation for the group. It should also be noted that a department is not guaranteed access to the respective maximum bandwidth associated with that department.
Further control over bandwidth allocation is provided by assigning a minimum guaranteed amount of bandwidth to be provided to the bandwidth utilizing entities such as companies, groups and departments, etc. The company in the present example has a minimum guaranteed bandwidth in the link 34 of 100 MBPS, the engineering group has a minimum guaranteed bandwidth through link 52 of 60 MBPS, the hardware department has a minimum guaranteed bandwidth through link 66 of 25 MBPS, and the programming department has a minimum guaranteed bandwidth through link 68 of 35 MBPS. In order to provide certainty of access to the minimum guaranteed bandwidth for respective entities within the network, the system may be structured such that the minimum guaranteed bandwidth provided to the company is equal to the sum of the minimum guaranteed bandwidths of the groups, and the sum of the minimum guaranteed bandwidths of the groups is equal to the sum of the minimum guaranteed bandwidths of the departments. Further, the minimum guaranteed bandwidth provided to some entities may be set to zero.
FIG. 2 illustrates resource management on N levels, where N.gtoreq.2. Twenty seven basic resource utilizing entities 100-152 exist within level 0 entities. Nine entities 154-172 exist at level 0. The level 0 entities exist within level 1 entities 174, 176, 178, which exist within a level 2 entity 180. The architecture continues to a level N entity 182 in which all lower level entities exist. For example, entity 122 exists within level 0 entity 160, which exists within level 1 entity 176, etc. Resources are allocated to the basic entities through the level N entities and intervening entities. As such, the total amount of resources which can be allocated to the basic entity is limited by the amount of resources which can be provided by the level 0 to level N entities.
Each entity in the architecture may optionally be assigned one or both of two variables which determine resource allocation to that entity: Minimum.sub.-- Guaranteed and Maximum.sub.-- Allowed. For each level illustrated, the area enclosed by dashed lines represents the Minimum.sub.-- Guaranteed resource allocation which can be supplied to the next lower level. The sum of the Minimum.sub.-- Guaranteed resource allocations at a given level is equal to the Minimum.sub.-- Guaranteed resource allocation of that level. For example, the Minimum.sub.-- Guaranteed resource allocation at level 1 for entity 176 is equal to the sum of allocations 184, 186, 188. Open area at each level represents additional bandwidth available above the Minimum.sub.-- Guaranteed allocation. The actual current amount of resource utilized ("actual use") by an entity is greater than or equal to zero, and less than or equal to Maximum.sub.-- Allowed. The basic operational rules of the management system are: (1) an entity may not utilize more resources than the assigned Maximum.sub.-- Allowed amount; and (2) each entity is assured of being provided with the Minimum.sub.-- Guaranteed amount, although a Minimum.sub.-- Guaranteed amount of "0" may be specified.
The system advantageously allocates resources to a requesting entity based on availability of resources at the intervening levels. If the resource utilizing entity is utilizing less than the assigned Minimum.sub.-- Guaranteed resource allocation, the request for resources is granted. However, when a request is made for, allocation of resources in excess of Minimum.sub.-- Guaranteed, resource availability at intervening levels 1 to N is determined based on assigned Minimum.sub.-- Guaranteed and Maximum.sub.-- Allowed variables. If a resource unit is not available at an intervening level because the entity at that level has reached the assigned Maximum.sub.-- Allowed resource allocation, the request for a resource unit at level 0 is denied. If a resource unit is available at each intervening level, the request is granted.
FIG. 3 is a flow chart which illustrates enqueuing under the resource management system. When a level 0 entity first requests 300 a resource unit, a query 302 is made to determine whether actual use by the level 0 entity is less than the respective Minimum.sub.-- Guaranteed variable. If the actual use is less than Minimum.sub.-- Guaranteed variable then a resource unit is allocated 304. If the actual use is not less than the respective Minimum.sub.-- Guaranteed variable, a query 306 is made to determine whether the actual use is less than Maximum.sub.-- Allowed for that entity. If the actual use is greater than or equal to the respective Maximum.sub.-- Allowed variable then the request is denied 308 and enqueuing terminates 310. If the actual use is less than the respective Maximum.sub.-- Allowed variable then a query 312 is made to determine whether a resource unit is available to the intervening level entities through which the resource unit would be allocated to the level 0 entity. That is, query 312 determines whether actual use is less than the respective Maximum.sub.-- Allowed variable at each intervening level. If actual use is not less than the respective Maximum.sub.-- Allowed variable at each intervening level then the resource unit request is denied 308. If actual use is less than the respective Maximum.sub.-- Allowed variable at each intervening level, the resource unit is allocated 304 to the level 0 entity and variables which track resource use are updated accordingly.
In order to track resource use for allocation purposes each entity is assigned a Min.sub.-- Limit variable, an AboveMin.sub.-- Limit variable, a Min.sub.-- Counter and an AboveMin.sub.-- Counter. Min.sub.-- Limit is used to limit the total Minimum.sub.-- Guaranteed resources the entity can use. AboveMin.sub.-- limit is used to limit the total resources which the entity can use in excess of Minimum.sub.-- Guaranteed. The Maximum.sub.-- Allowed number of resources an entity can use is equal to the sum of Min.sub.-- Limit and AboveMin.sub.-- Limit. Min.sub.-- Counter counts the number of resource units in use by an entity up to Min.sub.-- Limit. AboveMin.sub.-- Counter counts the number of resource units in use by an entity in excess of Min.sub.-- Limit. In practice, Min.sub.-- Counter is incremented as resource units are taken by the entity until Min.sub.-- Limit is reached, and AboveMin.sub.-- Counter is incremented as resource units are taken by the entity between Min.sub.-- Limit and AboveMin.sub.-- Limit.
To update the counters when a resource unit has been allocated the management system first determines whether the level 0 entity Min.sub.-- Counter variable is less than the Min.sub.-- Limit variable in a query 314. If the Min.sub.-- Counter variable is less than the Min.sub.-- Limit variable, the Min.sub.-- Counter variable is incremented 316. If the Min.sub.-- Counter variable is not less than the Min.sub.-- Limit variable, the AboveMin.sub.-- Counter variable is incremented 318. The system then determines whether level N has been reached in query 320. If level N has not been reached then the system examines the next higher level 322 and updates the counters accordingly. When level N is reached the program terminates 310.
Referring now to FIG. 4, resources which are no longer in use are dequeued. In dequeuing, a resource unit is returned in step 326 and the counters are updated. The counters are updated by first decrementing the respective AboveMin.sub.-- Counter variable to zero and then decrementing the respective Min.sub.-- Counter variable. More particularly, when a resource unit is returned the management system first determines if the respective AboveMin.sub.-- Counter variable is greater than zero in query 328. If the AboveMin.sub.-- Counter variable is greater than zero then AboveMin.sub.-- Counter variable is decremented 330, otherwise the respective Min.sub.-- Counter variable is decremented 332. The system then determines whether level N has been reached in query 334. If level N has not been reached then the system examines the next higher level 327 and updates the counters accordingly. When level N is reached the program terminates 336.
FIG. 5 illustrates a memory managed in a hierarchical manner. The actual memory 500 is shown at right. The memory is level 2 of a three level hierarchy. The requesters 502, at level 0 of the hierarchy, are shown at the left. Requesters are grouped by departments 504 at level 1 of the hierarchy. There may be many requesters per department, but for clarity only two are shown. Further, when necessary for clarity some levels are denoted by "n" and requesters are denoted by "c", as in Min.sub.-- Limit (n,c), which for n=1 and c=6 denotes the Min.sub.-- Limit value for department number 6.
The hierarchical memory management system includes Control Storage 506 to store all the Min.sub.-- Limit values, AboveMin.sub.-- Limit values and counters corresponding to all the Min.sub.-- Counter and AboveMin.sub.-- Counter values that are required. In each case the number of these values required is defined by the number of levels and the number of peer requesters at each level.
The actual memory 500 contains a fixed number of memory locations. These locations are divided into two pools of memory based on the Min.sub.-- Limit(2). Memory elements below the Min.sub.-- Limit represent the minimum guaranteed memory resource available at level 2. The total memory elements shown including those above the Min.sub.-- Limit represent the maximum allowable memory resource available. Memory elements corresponding to the minimum guaranteed resource available are depicted as being adjacent to one another with the dividing line for the Min.sub.-- Limit being a horizontal line, but there is no requirement that this be the case. Memory elements corresponding to the minimum guaranteed resource may be distributed throughout the actual memory 500, with the number of memory elements used in both categories held in corresponding counters and the Min.sub.-- Limit value held in a corresponding control storage location. The maximum allowable memory resource, depicted by the outline of the rectangle designated Memory, is equal to the number of memory elements available, which is a fixed predetermined number based on the physical memory itself. The value for Min.sub.-- Limit may be assigned, changed and managed, based upon the requirements of the controlling system.
At level 1 the minimum guaranteed memory resource available and the maximum allowable memory resource available for each of the requesters are similarly depicted. Again these values are held in counters and control storage locations. However, at level 1 all values may be assigned, changed and managed, based upon the requirements of the controlling system.
At level 0 the minimum guaranteed memory resource available and the maximum allowable memory resource available for each of the requestors are similarly depicted. Again these values are held in counters and control storage locations. At level 0 all values may be assigned, changed and managed, based upon the requirements of the controlling system.
As depicted, the memory elements granted to a requester are shown as a contiguous block of memory at level 0, level 1 and level 2, with the Min.sub.-- Limits aligning with the horizontal line depicting the Min.sub.-- Limit at the next higher level. However, in actual usage the memory elements granted could be spread throughout the memory. In all cases, counters and control storage control and track the actual assignment of memory elements to a requester. The AboveMin.sub.-- Limit at level 2, AboveMin.sub.-- Limit(2), exactly correspond to the total actual memory available less the guaranteed minimum, Min.sub.-- Limit(2). At levels 0 and 1 each of the values of the maximum memory resources available, AboveMin.sub.-- Limit(n,c), are set based both upon the actual memory resources available at level 2 and the requirements of the controlling system.
A request for a memory resource by a requester at level 0 is processed based upon the flow provided in FIG. 3, with the current usage at level 0, level 1 and level 2 and the flow of FIG. 3 determining whether the request will be granted or rejected. if granted, the requester will be allowed to use the requested memory in the actual memory at level 2. The return of the requested memory to available status is controlled by the flow shown in FIG. 4.
The sum of the minimum guaranteed memory resources for the requesters at level 0 belonging to one of the departments may exactly equal the minimum guaranteed memory resource for that department at level 1. Similarly, the sum of the minimum guaranteed memory resources at level 1 for all of the departments may exactly equal the minimum guaranteed memory resource at level 2.
The sum of the maximum allowable memory resources for requesters at level 0 belonging to one of the departments may be greater than the maximum allowable memory resource for that department at level 1. Similarly, the sum of the maximum allowable memory resources at level 1 for all of the departments may be greater than the maximum allowable memory resource at level 2.
The following pseudo code implements a system for managing resources, such as buffers or a memory as described above. This pseudo code requires a policy where the minimum resource is guaranteed.
Level 0: connection with "i" connections competing at this level
Level 1: service classes with "j" service classes competing level . . .
Level N: physical link with no competition above this level
In the basic scenario, an entity requests resources and competes against other entities within a level for resources. In the following definitions, the "n" variable refers to the level, and the "c" variable refers to which peer entity at the level is requesting resources at the next higher level.
Min.sub.-- Limit (n,c): Limits the number of "minimum" type resource the entity can use. It is used to assure that every entity gets its Minimum.sub.-- Guaranteed amount of the resource.
AboveMin.sub.-- Limit (n,c): Limits the number of "above-minimum" type resource the entity can use. The Maximum.sub.-- Allowed number of resource an entity can use is equal to the sum of the Min.sub.-- Limit and the AboveMin.sub.-- Limit.
Min.sub.-- Counter (n,c): Counts the number of resources in use by an entity until the Min.sub.-- Limit has been reached.
AboveMin.sub.-- Counter (n,c): Counts the number of resources in-use by an entity after the Min.sub.-- Limit has been reached.
Enqueuing (consume resource)
______________________________________IF (((Min.sub.-- Counter(0,i) < Min.sub.-- Limit(0,i)) OR((AboveMin.sub.-- Counter (0,i) < Above-Min.sub.-- Limit (0,i)) AND(AboveMin.sub.-- Counter (1,j) < AboveMin.sub.-- Limit (1,j) AND. . .(AboveMin.sub.-- Counter (N) < AboveMin.sub.-- Limit(N)))THEN{Take ResourceIF (Min.sub.-- Counter(0,i) < Min.sub.-- Limit(0,i))THEN {Increment Min-Counter(0, i)} ELSE {Increment AboveMin.sub.-- Counter(0,i)Increment AboveMin.sub.-- Counter(1,j). . .Increment AboveMin.sub.-- Counter (N)} ELSE {No.sub.-- Action (resource denied)}______________________________________
Free Resource:
______________________________________Return ResourceIF (AboveMin.sub.-- Counter (0,i) > 0)THEN Decrement AboveMin.sub.-- Counter(0,i)Decrement AboveMin.sub.-- Counter(i,j). . .Decrement Min.sub.-- Counter(N)} ELSE {Decrement Min.sub.-- Counter(0,i)______________________________________
The pseudo code provides management of resources such as buffers, bandwidth, channels and memory according to the method illustrated in FIGS. 3 and 4. Referring now to buffers for purposes of illustration, in enqueuing the first IF statement (IF (((Min.sub.-- Counter(0,i)<Min.sub.-- Limit(0,i)) OR ) determines if the buffer usage by the level 0 entity is less than Min.sub.-- Provided. If buffer usage is below Minimum.sub.-- Guaranteed then a buffer is granted, otherwise a buffer is granted only if buffer usage at each intervening level is less than Maximum.sub.-- Allowed. The IF statements following the logical OR (IF((AboveMin.sub.-- Counter (0,i)<AboveMin.sub.-- Limit (0,i)) AND (AboveMin.sub.-- Counter (1,j)<AboveMin.sub.-- Limit (1,j) AND . . . (AboveMin.sub.-- Counter (N)<AboveMin.sub.-- Limit(N)))) determine whether buffer usage by each intervening level is less than Maximum.sub.-- Allowed. If buffer usage by an intervening level is not less than Maximum.sub.-- Allowed then the request for a buffer is denied. When a buffer is granted the counters at level 0 and each intervening level are updated accordingly by the IF-THEN-ELSE statements. In dequeuing the AboveMin.sub.-- Counter is tested at each level and is decremented if greater than zero, otherwise the Min.sub.-- Counter is decremented.
The Minimum.sub.-- Guaranteed and Maximum.sub.-- Allowed variables may be assigned to entities in accordance with specified management policies. For example, an entity could be charged for resource use based on the values of the assigned Minimum.sub.-- Guaranteed and Maximum.sub.-- Allowed variables. Further, variables may be assigned in such a way as to allow a theoretical possibility of over-allocation of resources and thereby violate the basic rules of the management system. For example, if the total Minimum.sub.-- Guaranteed resources at level 0 for all level 0 entities exceeds the Minimum.sub.-- Guaranteed resources for the level 1 entity, a level 0 entity might possibly be unable to obtain its Minimum.sub.-- Guaranteed resource level, e.g., if each level 0 entity demands Minimum.sub.-- Guaranteed and the level 1 entity is reduced to Minimum.sub.-- Guaranteed due to competition. Such over allocation may be done by the user of the management system in accordance with a policy decision or according to statistical data indicating the unlikely occurrence of each level 0 entity simultaneously demanding Minimum.sub.-- Guaranteed.
The pseudo code above is directed toward a resource management system wherein a predetermined policy assures that the total Minimum.sub.-- Guaranteed available at one level is equal to the sum of all the Minimum.sub.-- Guaranteed resources at the level below. The pseudo code thus requires a Min.sub.-- Counter/Limit only at level 0. If statistical resource sharing is used then the intervening levels may also be assigned a Min.sub.-- Counter/Limit. Further, the resource management system may be operated without assigning any Minimum.sub.-- Guaranteed variables. For example, Maximum.sub.-- Allowed variables could be assigned to one or more entities such that the system operates to deny a request for a resource unit if any entity from level 0 to level N is not below Maximum.sub.-- Allowed, without regard for assuring minimum resource allocation.
The following table shows an exemplary resource management policy:
TABLE 1______________________________________ Sum of Num- Min.sub.-- Limit Sum of AboveMin.sub.-- AboveMinLevel Entities /entity Min.sub.-- Limits Limit/entity .sub.-- Limits______________________________________0 10 2 20 10 1001 10 20 200 60 6002 10 200 2000 500 50003 1 2000 n/a 4000 n/a______________________________________
The difference between the "Sum of Min.sub.-- Limits" at a lower level and the Min.sub.-- Limit/entity of the next level illustrates the policy which ensures that each level 0 entity is provided with the assigned Min.sub.-- Provided allocation, i.e., the sum of Min.sub.-- Limits at a lower level is less than or equal to the Min.sub.-- Limit of the entity at the next higher level.
It should be noted however that Table 1 is a simplified example because the Min.sub.-- Limit and the AboveMin.sub.-- Limit are the same for all entities at a given level, and the number of entities are the same at each level. Typically, the limits would be different for each entity at each level, and the number of entities at each level would decrease at each increasing level of the hierarchy. A lower cost implementation could employ a restriction such that all of the limits of a particular entity would have the same value. Such a restriction would obviate the requirement of a per-entity limit storage location. The limits could then be kept on a per level basis for example.
Under some circumstances it may be desirable to statistically share the Minimum Guaranteed resources. For example, statistical sharing would be advantageous in a system having numerous resource users which will not simultaneously request resources. Table 2 illustrates an exemplary resource management policy where statistical sharing of the minimum guaranteed resource is employed. It will be appreciated that if every level 0 entity were to simultaneously request resources, the sum of the requested minimum resources (200) could not be provided by a level 1 entity (180).
TABLE 2______________________________________ Sum of Num- Min.sub.-- Limit Sum of AboveMin.sub.-- AboveMinLevel Entities /entity Min.sub.-- Limits Limit/entity .sub.-- Limits______________________________________0 10 2 20 10 1001 10 18 180 60 6002 10 144 1440 500 50003 1 864 n/a 4000 n/a______________________________________
When the Minimum.sub.-- Guaranteed portion is going to be statistically shared, then the pseudocode is modified as follows:
Consume Resource
______________________________________IF (((Min.sub.-- Counter(0,i) < Min.sub.-- Limit(0,i)) AND((Min.sub.-- Counter(1,j) < Min.sub.-- Limit(1,j)) OR(AboveMin.sub.-- Counter (1,j) < AboveMin.sub.-- Limit(1,j)))AND. . .((Min.sub.-- Counter(N) < Min.sub.-- Limit(N)) OR(AboveMin.sub.-- Counter(N) < AboveMin.sub.-- Limit(N))))THEN {Take ResourceIF (Min.sub.-- Counter(0,i) < Min.sub.-- Limit(0,i)THEN { Increment Min.sub.-- Counter(0,i)IF (Min.sub.-- Counter(1,j) < Min.sub.-- Limit(1,j)THEN { Increment Min.sub.-- Counter(1,j)} ELSE { Increment Min.sub.-- Counter(1,j)}. . . . .IF(Min.sub.-- Counter (N) < Min.sub.-- Limit(N))THEN { Increment Min.sub.-- Counter(N)} ELSE { Increment AboveMin.sub.-- Counter(N)}}ELSE IF ((AboveMin.sub.-- Counter(0,i) < AboveMin.sub.-- Limit(0,i)) AND((Min.sub.-- Counter(1,j) < Min.sub.-- Limit(1,j)) ANDAboveMin.sub.-- Counter(1,j) < AboveMin.sub.-- Limit(1,j))) AND. . . .((Min.sub.-- Counter(N) < Min.sub.-- Limit(N)) AND(AboveMin.sub.-- Counter(N) < AboveMin Limit (N))))THEN {Take ResourceIncrement AboveMin.sub.-- Counter(0,i)Increment AboveMin.sub.-- Counter(1,j). . . . .Increment AboveMin.sub.-- Counter(N)}ELSE {No.sub.-- Action (resource denied)}Free Resource:Return ResourceIF (AboveMin.sub.-- Counter(0,i)>0)THEN {Decrement AboveMin.sub.-- Counter(0,i)} ELSE {Decrement Min.sub.-- Counter(0,i)}IF (AboveMin.sub.-- Counter(1,j)>0)THEN {Decrement AboveMin.sub.-- Counter(1,j)} ELSE {Decrement Min.sub.-- Counter(1,j)}. . . . .IF (AboveMin.sub.-- Counter(N) >0)THEN {Decrement AboveMin.sub.-- Counter(N)} ELSE {Decrement Min.sub.-- Counter(N)}______________________________________
The pseudocode allows statistical sharing policies for resource usage below the minimum guaranteed. Unlike the earlier described pseudocode, this pseudocode requires a Min.sub.-- Limit and Min.sub.-- Counter for each entity at each level. If the level 0 entity is below the associated Minimum.sub.-- Guaranteed resource allocation, then the level 0 entity may use resources above the Minimum.sub.-- Guaranteed allocations at higher levels. This policy allows entities to continue using resources even if the Minimum.sub.-- Guaranteed pool at a higher level in the hierarchy has been consumed. Additionally, if actual use by the level 0 entity is above the Minimum.sub.-- Guaranteed allocation, the resource will only be granted if there are "Minimum.sub.-- Guaranteed" resources at all higher levels, i.e., the Min.sub.-- Counter must be less than the Min.sub.-- Limit for each higher level entity. This policy leaves resources available for other entities that have not consumed their minimum.
The Min.sub.-- Limit at level N is optional. Min.sub.-- Limit(N) controls the point at which requests for resources above the "Minimum.sub.-- Guaranteed" pool are denied. Without control at level N, the "Maximum.sub.-- Allowed" denial point is equal to the sum of the Min.sub.-- Limits at the level below. The change to the pseudocode for such an implementation is to remove the (Min.sub.-- Counter(N)<Min.sub.-- Limit(N)) comparisons in the conditional statements.
The techniques described above are equally applicable to management of hierarchical resources other than bandwidth, and may be generalized with entities other than departments, groups, companies, etc. It should therefore be understood that the invention is not limited to the particular embodiments shown and described herein, and that various changes and modifications may be made without departing from the spirit and scope of this novel concept as defined by the following claims.
Claims
  • 1. A method of hierarchical resource management for allocation of resource units to a level 0 entity through level 1 to level N entities including intervening levels, where N is greater than or equal to two, comprising:
  • assigning a first variable for at least one entity specifying a maximum resource allocation for the respective entity;
  • receiving a request from a level 0 entity for a resource unit; and
  • in response to a request from a level 0 entity for a resource unit, allocating said resource unit to the level 0 entity if:
  • a) actual resource utilization by said level 0 entity is less than the maximum resource allocation specified by the respective first variable; and
  • b) actual resource utilization by each of said level 1 through level N-1 entities is less than the maximum resource allocation specified by the respective first variable for each of said entities.
  • 2. The method of hierarchical resource management of claim 1 wherein said method further includes the step of assigning a second variable for at least one entity, said second variable specifying a minimum resource allocation for the respective entity to which the second variable is assigned.
  • 3. The method of hierarchical resource management of claim 1 including the further step of employing the method to manage a storage resource.
  • 4. The method of hierarchical resource management of claim 3 including the further step of employing the method to manage memory.
  • 5. The method of hierarchical resource management of claim 4 including the further step of employing the method to manage buffers in a telecommunications network.
  • 6. The method of hierarchical resource management of claim 4 including the further step of employing the method to manage memory for storing ATM cells.
  • 7. The method of hierarchical resource management of claim 1 including the further step of employing the method to manage channels.
  • 8. A method of hierarchical resource management for allocating resource units to a level 0 entity through entities at levels 1 to N including intervening levels, where N is greater than or equal to two, comprising the steps of:
  • assigning a first variable for at least one entity specifying a maximum resource allocation for the respective entity;
  • assigning a second variable for at least one entity specifying a minimum resource allocation for the respective entity;
  • receiving a request from a level 0 entity for a resource unit; and
  • in response to a request from a level 0 entity for a resource unit, allocating said resource unit to the level 0 entity if actual resource utilization is less than the resource allocation specified by the respective second variable, and allocating said resource unit to the level 0 entity if:
  • a) actual resource utilization by said level 0 entity is less than the maximum resource allocation specified by the respective first variable; and
  • b) actual resource utilization by each of said level 1 through said level N-1 entities is less than the maximum resource allocation specified by the respective first variable for each of said entities.
  • 9. A method of hierarchical resource management for allocation of resource units to a level 0 entity through level 1 to level N entities including intervening levels, where N is greater than or equal to two, comprising the steps of:
  • assigning a first variable for at least one entity specifying a maximum resource allocation for the respective entity;
  • assigning a second variable for at least one entity specifying a minimum resource allocation for the respective entity;
  • receiving a request from a level 0 entity for a resource unit; and
  • in response to a request from a level 0 entity for a resource unit:
  • a) allocating said resource unit to the level 0 entity provided actual resource utilization by the level 0 entity is less than the minimum resource utilization specified by the second variable, said resource unit being granted without regard to any resultant resource utilization in excess of that specified by the respective second variable at levels 1 through N; and
  • b) allocating said resource unit to the level 0 entity provided actual resource utilization by the level 0 entity is greater than the minimum allocation specified by the second variable, and provided resultant actual resource utilization at each of levels 1 through N will be less than the minimum resource utilization specified by the respective second variable.
  • 10. The method of hierarchical resource management of claim 9 including the further step of employing the method to manage a storage resource.
  • 11. The method of hierarchical resource management of claim 10 including the further step of employing the method to manage memory.
  • 12. The method of hierarchical resource management of claim 11 including the further step of employing the method to manage buffers in a telecommunications network.
  • 13. The method of hierarchical resource management of claim 11 including the further step of employing the method to manage memory for storing ATM cells.
US Referenced Citations (268)
Number Name Date Kind
3804991 Hammond et al. Apr 1974
3974343 Cheney et al. Aug 1976
4069399 Barrett et al. Jan 1978
4084228 Dufond et al. Apr 1978
4240143 Besemer et al. Dec 1980
4603382 Cole et al. Jul 1986
4715030 Koch et al. Dec 1987
4727537 Nichols Feb 1988
4737953 Koch et al. Apr 1988
4748658 Gopal et al. May 1988
4797881 Ben-Artzi Jan 1989
4821034 Anderson et al. Apr 1989
4837761 Isono et al. Jun 1989
4849968 Turner Jul 1989
4870641 Pattavina Sep 1989
4872157 Hemmady et al. Oct 1989
4872159 Hemmady et al. Oct 1989
4872160 Hemmady et al. Oct 1989
4872197 Pemmaraju Oct 1989
4878216 Yunoki Oct 1989
4893302 Hemmady et al. Jan 1990
4893307 McKay et al. Jan 1990
4894824 Hemmady et al. Jan 1990
4897833 Kent et al. Jan 1990
4897841 Gang, Jr. Jan 1990
4899333 Roediger Feb 1990
4920531 Isono et al. Apr 1990
4922503 Leone May 1990
4933938 Sheehy Jun 1990
4942574 Zelle Jul 1990
4947390 Sheehy Aug 1990
4953157 Franklin et al. Aug 1990
4956839 Torii et al. Sep 1990
4958341 Hemmady et al. Sep 1990
4979100 Makris et al. Dec 1990
4993018 Hajikano et al. Feb 1991
5014192 Mansfield et al. May 1991
5021949 Morten et al. Jun 1991
5029164 Goldstein et al. Jul 1991
5060228 Tsutsui et al. Oct 1991
5067123 Hyodo et al. Nov 1991
5070498 Kakuma et al. Dec 1991
5083269 Syobatake et al. Jan 1992
5084867 Tachibana et al. Jan 1992
5084871 Carn et al. Jan 1992
5090011 Fukuta et al. Feb 1992
5090024 Mey et al. Feb 1992
5093827 Franklin et al. Mar 1992
5093912 Dong et al. Mar 1992
5115429 Hluchyj et al. May 1992
5119369 Tanabe et al. Jun 1992
5119372 Verbeek Jun 1992
5128932 Li Jul 1992
5130975 Akata Jul 1992
5130982 Ash et al. Jul 1992
5132966 Hayano et al. Jul 1992
5146474 Nagler et al. Sep 1992
5146560 Goldberg et al. Sep 1992
5150358 Punj et al. Sep 1992
5151897 Suzuki Sep 1992
5157657 Potter et al. Oct 1992
5163045 Caram et al. Nov 1992
5163046 Hahne et al. Nov 1992
5179556 Turner Jan 1993
5179558 Thacker et al. Jan 1993
5185743 Murayama et al. Feb 1993
5191582 Upp Mar 1993
5191652 Dias et al. Mar 1993
5193151 Jain Mar 1993
5197067 Fujimoto et al. Mar 1993
5198808 Kudo Mar 1993
5199027 Barri Mar 1993
5239539 Uchida et al. Aug 1993
5253247 Hirose et al. Oct 1993
5253248 Dravida et al. Oct 1993
5255264 Cotton et al. Oct 1993
5255266 Watanabe et al. Oct 1993
5257311 Naito et al. Oct 1993
5258979 Oomuro et al. Nov 1993
5265088 Takigawa et al. Nov 1993
5267232 Katsube et al. Nov 1993
5268897 Komine et al. Dec 1993
5271010 Miyake et al. Dec 1993
5272697 Fraser et al. Dec 1993
5274641 Shobatake et al. Dec 1993
5274768 Traw et al. Dec 1993
5280469 Taniguchi et al. Jan 1994
5280470 Buhrke et al. Jan 1994
5282201 Frank et al. Jan 1994
5283788 Morta et al. Feb 1994
5285446 Yonehara Feb 1994
5287349 Hyodo et al. Feb 1994
5287535 Sakagawa et al. Feb 1994
5289462 Ahmadi et al. Feb 1994
5289463 Mobasser Feb 1994
5289470 Chang et al. Feb 1994
5291481 Doshi et al. Mar 1994
5291482 McHarg et al. Mar 1994
5295134 Yoshimura et al. Mar 1994
5301055 Bagchi et al. Apr 1994
5301184 Uriu et al. Apr 1994
5301190 Tsukuda et al. Apr 1994
5301193 Toyofuku et al. Apr 1994
5303232 Faulk, Jr. Apr 1994
5305311 Lyles Apr 1994
5309431 Tominaga et al. May 1994
5309438 Nakajima May 1994
5311586 Bogart et al. May 1994
5313454 Bustini et al. May 1994
5313458 Suzuki May 1994
5315586 Charvillat May 1994
5319638 Lin Jun 1994
5321605 Chapman et al. Jun 1994
5321695 Proctor et al. Jun 1994
5323389 Bitz et al. Jun 1994
5333131 Tanabe et al. Jul 1994
5333134 Ishibashi et al. Jul 1994
5335222 Kamoi et al. Aug 1994
5335325 Frank et al. Aug 1994
5339310 Taniguchi Aug 1994
5339317 Tanaka et al. Aug 1994
5339318 Tanaka et al. Aug 1994
5341366 Soumiya et al. Aug 1994
5341373 Ishibashi et al. Aug 1994
5341376 Yamashita Aug 1994
5341483 Frank et al. Aug 1994
5345229 Olnowich et al. Sep 1994
5350906 Brody et al. Sep 1994
5355372 Sengupta et al. Oct 1994
5357506 Sugawara Oct 1994
5357507 Hughes et al. Oct 1994
5357508 Le Bouded et al. Oct 1994
5357510 Norizuki et al. Oct 1994
5359600 Ueda et al. Oct 1994
5361251 Aihara et al. Nov 1994
5361372 Rege et al. Nov 1994
5363433 Isono Nov 1994
5363497 Baker et al. Nov 1994
5369570 Parad Nov 1994
5371893 Price et al. Dec 1994
5373504 Tanaka et al. Dec 1994
5375117 Morita et al. Dec 1994
5377262 Bales et al. Dec 1994
5377327 Jain et al. Dec 1994
5379297 Glover et al. Jan 1995
5379418 Shimazaki et al. Jan 1995
5390170 Sawant et al. Feb 1995
5390174 Jugel Feb 1995
5390175 Hiller et al. Feb 1995
5392280 Zheng Feb 1995
5392402 Robrock, II Feb 1995
5394396 Yoshimura et al. Feb 1995
5394397 Yanagi et al. Feb 1995
5398235 Tsuzuki et al. Mar 1995
5400337 Munter Mar 1995
5402415 Turner Mar 1995
5412648 Fan May 1995
5414703 Sakaue et al. May 1995
5418942 Krawchuk et al. May 1995
5420858 Marshall et al. May 1995
5420988 Elliott May 1995
5422879 Parsons et al. Jun 1995
5425021 Derby et al. Jun 1995
5425026 Mori Jun 1995
5426635 Mitra et al. Jun 1995
5432713 Takeo et al. Jul 1995
5432784 Ozveren Jul 1995
5432785 Ahmed et al. Jul 1995
5432908 Heddes et al. Jul 1995
5436886 McGill Jul 1995
5436893 Barnett Jul 1995
5440547 Easki et al. Aug 1995
5444702 Burnett et al. Aug 1995
5446733 Tsuruoka Aug 1995
5446737 Cidon et al. Aug 1995
5446738 Kim et al. Aug 1995
5448559 Hayter et al. Sep 1995
5448621 Knudsen Sep 1995
5450406 Esaki et al. Sep 1995
5452296 Shimizu Sep 1995
5454299 Thessin et al. Oct 1995
5455820 Yamada Oct 1995
5455825 Lauer et al. Oct 1995
5457687 Newman Oct 1995
5459743 Fukuda et al. Oct 1995
5461611 Drake Jr. et al. Oct 1995
5463620 Sriram Oct 1995
5463629 Ko Oct 1995
5463775 DeWitt et al. Oct 1995
5465331 Yang et al. Nov 1995
5465365 Winterbottom Nov 1995
5469003 Kean Nov 1995
5473608 Gagne et al. Dec 1995
5475679 Munter Dec 1995
5479401 Bitz et al. Dec 1995
5479402 Hata et al. Dec 1995
5483526 Ben-Nun et al. Jan 1996
5485453 Wahlman et al. Jan 1996
5485455 Dobbins et al. Jan 1996
5487063 Kakuma et al. Jan 1996
5488606 Kakuma et al. Jan 1996
5491691 Shtayer et al. Feb 1996
5491694 Oliver et al. Feb 1996
5493566 Ljungberg et al. Feb 1996
5497369 Wainwright Mar 1996
5499238 Shon Mar 1996
5504741 Yamanaka et al. Apr 1996
5504742 Kakuma et al. Apr 1996
5506834 Sekihata et al. Apr 1996
5506839 Hatta Apr 1996
5506956 Cohen Apr 1996
5509001 Tachibana et al. Apr 1996
5509007 Takashima et al. Apr 1996
5513134 Cooperman et al. Apr 1996
5513178 Tanaka Apr 1996
5513180 Miyake et al. Apr 1996
5515359 Zheng May 1996
5517495 Lund et al. May 1996
5519690 Suzuka et al. May 1996
5521905 Oda et al. May 1996
5521915 Dieudonne et al. May 1996
5521916 Choudhury et al. May 1996
5521917 Watanabe et al. May 1996
5521923 Willmann et al. May 1996
5523999 Takano et al. Jun 1996
5524113 Gaddis Jun 1996
5526344 Diaz et al. Jun 1996
5528588 Bennett et al. Jun 1996
5528590 Iidaka et al. Jun 1996
5528591 Lauer Jun 1996
5530695 Dighe et al. Jun 1996
5533009 Chen Jul 1996
5533020 Byrn et al. Jul 1996
5535196 Aihara et al. Jul 1996
5535197 Cotton Jul 1996
5537394 Abe et al. Jul 1996
5541912 Choudhury et al. Jul 1996
5544168 Jeffrey et al. Aug 1996
5544169 Norizuki et al. Aug 1996
5544170 Kasahara Aug 1996
5546389 Wippenbeck et al. Aug 1996
5546391 Hochschild et al. Aug 1996
5546392 Boal et al. Aug 1996
5550821 Akiyoshi Aug 1996
5550823 Irie et al. Aug 1996
5553057 Nakayama Sep 1996
5553068 Aso et al. Sep 1996
5555243 Kakuma et al. Sep 1996
5555265 Kakuma et al. Sep 1996
5557607 Holden Sep 1996
5568479 Watanabe et al. Oct 1996
5570361 Norizuki et al. Oct 1996
5570362 Nishimura Oct 1996
5572522 Calamvokia et al. Nov 1996
5577032 Sone et al. Nov 1996
5577035 Hayter et al. Nov 1996
5583857 Soumiya et al. Dec 1996
5583858 Hanaoka Dec 1996
5583861 Holden Dec 1996
5590132 Ishibashi et al. Dec 1996
5602829 Nie et al. Feb 1997
5610913 Tomonaga et al. Mar 1997
5623405 Isono Apr 1997
5625846 Kobayakawa et al. Apr 1997
5633861 Hanson et al. May 1997
5640569 Miller et al. Jun 1997
5682530 Shimamura Oct 1997
5719854 Choudhury et al. Feb 1998
Non-Patent Literature Citations (17)
Entry
An Ascom Timeplex White Paper, Meeting Critical Requirements with Scalable Enterprise Networking Solutions Based on a Unified ATM Foundation, pp. 1-12, Apr. 1994.
Douglas H. Hunt, ATM Traffic Management -Another Perpective, Business Communications Review, Jul. 1994.
Richard Bubenik et al., Leaf Initiated Join Extensions, Technical Committee, Signalling Subworking Group, ATM Forum/94-0325R1, Jul. 1, 1994.
Douglas H. Hunt et al., Flow Controlled Virtual Connections Proposal for ATM Traffic Management (Revision R2), Traffic Management Subworking Group, ATM Forum/94-0632R2, Aug. 1994.
Flavio Bonomi et al., The Rate-Based Flow Control Framework for the Available Bit Rate ATM Service, IEEE Network, Mar./Apr. 1995, pp. 25-39.
R. Jain, Myths About Congestion Management in High Speed Networks, Internetworking Research and Experience, vol. 3, 101-113 (1992).
Douglas H. Hunt et al., Credit-Based FCVC Proposal for ATM Traffic Management (Revision R1), ATM Forum Technical Committee Traffic Management Subworking Group, ATM Forum/94-0168R1, Apr. 28, 1994.
Douglas H. Hunt et al., Action Item Status for Credit-Based FCVC Proposal, ATM Forum Technical Committee Traffic Management Subworking Group, ATM Forum/94-0439, Apr. 28, 1994.
Timothy P. Donahue et al., Arguments in Favor of Continuing Phase 1 as the Initial ATM Forum P-NNI Routing Protocol Implementation, ATM Forum Technical Committee, ATM Forum/94-0460, Apr. 28,1994.
Richard Bubenick et al., Leaf Initiated Join Extensions, Technical Committee, Signalling Subworking Group, ATM Forum/94-0325, Apr. 28, 1994.
Rob Coltun et al., PRP: A P-NNI Routing Protocol Proposal, ATM Forum Technical Committee, ATM Forum/94-0492, Apr. 28, 1994.
Richard Bubenik et al., Leaf Initiated Join Extensions, ATM Forum Technical Committee, Signalling Subworking Group, ATM Forum 94-0325, Apr. 28, 1994.
Richard Bubenik et al., Requirements For Phase 2 Signaling Protocol, ATM Forum Technical Committee, Signalling Subworking Group, ATM Forum 94-1078, Jan. 1, 1994.
SITA, ATM RFP: C-Overall Technical Requirements, Sep. 1994.
H.T. Kung and K. Chang, Receiver-Oriented Adaptive Buffer Allocation in Credit-Based Flow Control for ATM Networks, Proceedings of INFOCOM '95, Apr. 2-6, 1995, pp. 1-14.
H.T. Kung, et al., Credit-Based Flow Control for ATM Networks: Credit Update Protocol, Adaptive Credit Alloca and Statistical Multiplexing, Proceedings of ACM SIGCOMM '94 Symposium on Communications Architectures, Protocols and Applications, Aug. 31-Sep. 2, 1994, pp. 1-14.
Head of Line Arbitration in ATM Switches with Input-Output Buffering and Backpressure Control. By Hosein F. Badran and H.T. Mouftah, Globecom '91, pp. 0347-0351.