The present invention relates to data processing systems and computer networks in general, and, more particularly, to techniques for caching resources in cache.
When a user of the World Wide Web requests a Web page, the user must wait until the page is available on his or her data processing system (e.g., computer, etc.) for viewing. In general, this wait occurs because the request for the Web page must traverse the Internet from the user's data processing system to the data processing system that is the source of the page, the request must be fulfilled, and the requested page must travel back to the user's system. If the Internet is congested or the data processing system that is the source of the page is overwhelmed with many concurrent requests for pages, the wait can be considerably long.
To shorten this wait, special data processing systems are deployed throughout the Internet that expedite the delivery of some Web pages. Some of these data processing systems expedite the delivery of Web pages by functioning as cache memories, which are also known as “caches.” For the purpose of this specification, a “cache” is defined as a cache memory. For example, a cache stores commonly requested Web pages and thereafter enables requests for those pages to be intercepted and fulfilled from the cache without retrieval from the principal memory. This expedites the delivery of the Web page in two ways. First, a cache eliminates the need for the request to travel all of the way to the system that is the ultimate source of the page, and, therefore, eliminates some of the wait associated with the transit. Second, a cache also reduces the number of Web page requests that must be fulfilled by the system that is the ultimate source of the page, and, therefore, the wait associated with contention for the system is eliminated.
From the perspective of node 121 and node 111, node 101 actually or apparently comprises a vast amount of information arranged in bundles, called “resources.” For the purposes of this specification, a “resource” is defined as an individually addressable bundle of information that can be requested by a node. For example, a resource might be an individual computer file (e.g., a World Wide Web page, a .gif file, a Java script, etc.) or a database record, etc. Although node 101 can actually comprise a vast amount of information if, for example, it is a disk farm, it can also apparently comprise the information if it acts as a gateway to a data network, such as the Internet.
When node 101 is bombarded with a large number of concurrent requests for resources from node 121, node 101 might not be able to instantaneously respond to all of the requests. Therefore, to reduce the average delay between when node 121 requests a resource from node 101 and when it receives the resource, node 111 functions as a cache for node 101.
At step 301, node 121 receives a resource identifier and a request for the resource. This request and resource identifier might, for example, originate with a user of node 121 as part of a World Wide Web browsing session (e.g., http://www.amazon.com/mccullers.htm, etc.).
At step 302, node 121 transmits: (i) the resource identifier, and (ii) a request for the resource to node 111, and at step 303, node 111 receives: (i) the resource identifier, and (ii) a request for the resource.
At step 305, node 111 determines if, in fact, the requested resource is in its cache data structure. If it is (i.e., a cache “hit”), then control passes to step 309; otherwise (i.e., a cache “miss”) control passes to step 306.
At step 306, node 111 transmits the resource identifier and request for the resource identifier to node 101, and at step 307 node 111 receives the requested resource.
At step 308, node 111 populates its cache with the received resource so that the next time the resource is requested, node 111 can fulfill the request itself. If the cache does not have enough empty storage available for the resource, node 111 can delete other resources in the cache, in accordance with any of many well-known cache replacement algorithms, to make room for the most recently requested resource.
At step 309, node 111 transmits the resource to node 121, as requested, whether the requested resource was in node 111's cache data structure or not.
The increasing size and complexity of the Internet, and its increasing use for transmitting multimedia resources has created the need for improved caching techniques.
The present invention is a technique for efficiently populating a cache with resources that avoids some of the costs and disadvantages associated with caching techniques in the prior art. In particular, a node in accordance with the illustrative embodiment of the present invention defers, at least occasionally, populating its cache with a resource until at least two requests for the resource have been received. This is advantageous because it prevents the cache from being populated with infrequently requested resources.
Furthermore, the illustrative embodiment of the present invention populates a cache with a resource only when:
Embodiments of the present invention are particularly advantageous in computer networks that comprise a logical hierarchical topology, but are useful an any computer network, and in individual data processing systems and routers that comprise a cache memory.
The illustrative embodiment of the present invention comprises populating a cache with a resource only when at least i requests for said resource have been received, wherein i is an integer greater than one.
Although the illustrative embodiment comprises 12 data processing nodes in one particular hierarchy, it will be clear to those skilled in the art how to make and use embodiments of the present invention that comprise any number of nodes that are interconnected in any hierarchy. Furthermore, it will be clear to those skilled in the art how the inventions described herein are useful in any computer network with any logical topology—including those that are not hierarchical—and also to individual data processing systems and routers that comprise a cache memory.
In accordance with the illustrative embodiment of the present invention, each pair of interconnected nodes communicate with each other, either directly or indirectly, via one or more physical wireline or wireless telecommunications links or both (not shown in
Because the illustrative embodiment has a hierarchical topology, several terms relating to hierarchies are defined so as to facilitate an unambiguous description of the illustrative embodiment. Therefore, for the purpose of this specification:
In accordance with the illustrative embodiment, root node 401 actually or apparently comprises a vast amount of information, arranged in bundles called “resources,” that are individually addressable and that can be individually requested by some or all of the nodes in hierarchical network 400. For example, root node 401 can be a disk farm or a gateway to a data network (not shown in
For the purposes of this specification, a “resource identifier” is defined as the indicium of a resource. In accordance with the illustrative embodiment, each resource identifier is a uniform resource locator (e.g., www.amazon.com/books/102-8956393, etc.), which is commonly called a “URL.” It will be clear to those skilled in the art how to make and use embodiments of the present invention in which a resource identifier is something other than a URL.
In accordance with the illustrative embodiment of the present invention, some or all of the nodes in the illustrative embodiment generate requests for resources that are originally available via root node 401. Some of these requests might be instigated by a user associated with a node and some of the requests might be instigated by a node itself. Typically, the leaf nodes are the nodes that originally generate the requests because the leaf nodes are typically those that interact most often with end-users.
Because root node 401 might be bombarded with many concurrent requests for resources, it is typically not able to instantaneously provide a requested resource. And because any delay between the time when a node requests a resource and when the node receives the resource is generally undesirable, the illustrative embodiment advantageously incorporates caches for reducing the average delay. In accordance with the illustrative embodiment of the present invention, each node advantageously acts as a cache for its lineal nodes.
Processor 501 is advantageously a general-purpose processor or a special-purpose processor that performs the functionality described herein and with respect to FIG. 6. Memory 502 holds programs and data for processor 501, and cache 503. It will be clear to those skilled in the art that memory 502 can utilize any storage technology (e.g., semiconductor RAM, magnetic hard disk, optical disk, etc.) or combination of storage technologies, and it will also be clear to those skilled in the art that memory 502 can comprise a plurality of memories, each of which has different memory spaces.
All nodes, including root node 401 if it is a gateway to a data network, comprise: transmitter 513 for transmitting data to its parental node (or to the data network in the case of the root node) and receiver 514 for receiving data from its parental node (or from the data network in the case of the root node). It will be clear to those skilled in the art how to make and use transmitter 513 and receiver 514.
All nodes, except the leaves, comprise: one or more receivers 510-i and one or more transmitters 511-i for communicating with each of the node's n filial nodes, where i=1 to n. It will be clear to those skilled in the art how to make and use receivers 510-1 through 510-n and transmitters 511-1 through 511-n.
At step 601, the Given Node receives a resource identifier and a request for the resource. This request and resource identifier might, for example, originate with a user of the Given Node as part of a World Wide Web browsing session (e.g., http://www.amazon.com/mccullers.htm, etc.). As another example, the request and resource identifier can originate with a lineal node of the Given Node, in which case the Given Node might retrieve the resource and store it and its resource identifier in its own cache.
At step 602, the Given Node transmits:
At step 603, the Parental Node receives:
At step 604, the Parental Node uses the resource identifier as an index into cache 503 to determine if the resource is contained in the Parental Node's cache. Alternatively, as taught in applicants' co-pending U.S. patent application Ser. No. 09/______, entitled “Distributed Caching Architecture For Computer Networks,” the Parental Node can use a hashed function of the resource identifier as the index into cache 503. In either case, if the requested resource is in cache 503 (i.e., a cache hit), then control passes to step 610; otherwise (i.e., a cache “miss”) control passes to step 605.
At step 605, the Parental Node begins the process, which is completed in step 606, of retrieving the requested resource from its parental node, the Grandparental Node. Advantageously, the Parental Node retrieves the requested resource from its parental node in the same manner that the Given Node did from the Parental Node. In other words, step 605 for the Parental Node is identical to step 602 for the Given Node in that the Parental Node advantageously transmits:
At step 606, the Parental Node receives the requested resource from the Grandparental Node.
At step 607 the Parental Node records:
The purpose of recording this information is to enable the Parental Node, at step 608, to determine when and if the resource received in step 606 should be stored in cache 503.
At step 608, the Parental Node determines whether the resource received in step 606, which is not currently in the Parental Node's cache, should be stored in cache 503. There are several factors that the Parental Node considers.
First, the Parental Node only populates cache 503 with the resource when at least i requests for the resource have been received, wherein i is a positive integer. In other words, the illustrative embodiment won't store the resource in the Parental Node's cache unless at least i requests for the resource have been received within an elapsed time interval, Δt. In some cases, the value of i is one, and in other cases the value of i is an integer greater than one.
In some cases, the value of i is invariant (i.e., it does not change over time or as a function of circumstance). Alternatively, the value of i varies and is based on:
i. the calendrical time, or
ii. the elapsed time interval, Δt, or
iii. the number, m, of filial nodes of the Parental Node, or
iv. any combination of i, ii, and iii.
For the purposes of this specification, the phrase “calendrical time” is defined as the time with respect to the calendar. For example, the value of i can vary with the time of day, the day of the week, the day of the month, the day of the year, the month of the year, the season of the year, the year itself, etc. Table 2 depicts an illustrative embodiment of the present invention in which the value of i varies as a function of the day of the week.
The Parental Node can determine how many total requests there have been for a resource by using the data the Parental Node stored in step 607. As will be clear to those skilled in the art, varying the value of i as a function of the calendrical time has several effects on the operation of the illustrative embodiment. First, as shown in
Second, the Parental Node only populates cache 503 with the resource when at least i requests for the resource have been received within an elapsed time interval, Δt. In other words, the illustrative embodiment won't store the resource in the Parental Node's cache unless at least a plurality of requests for the resource have been received within some time interval, Δt. In some cases, the value of Δt can be invariant.
Alternatively, the value of Δt can vary and can be based on:
i. the value of i, or
ii. the calendrical time, or
iii. the number, m, of filial nodes of the Parental Node, or
iv. any combination of i, ii, and iii.
Table 3 depicts an illustrative embodiment of the present invention in which the value Δt varies as a function of the value of i.
Furthermore, both i and Δt can vary and can be based on the calendrical time. Table 4 depicts an illustrative embodiment of the present invention in which the values of i and Δt vary as a function of the time of day.
The Parental Node can determine when each request for a resource have been made, and therefore if the requisite number of requests have been made in the elapsed time interval, Δt, by using the using the data the Parental Node has stored in step 607.
Third, the Parental Node only populates cache 503 with the resource when at least one request for the resource has been received from at least n of the Parental Node's m filial nodes. This is advantageous because it prevents the cache from being populated with resources that are only being used by a few of the Parental Node's filial nodes. In some cases, the value of n can be invariant.
Alternatively, the value of n can vary based on:
i. the value of m, or
ii. the value of i, or
iii. the elapsed time interval, Δt, or
iv. the calendrical time,
v. any combination of i, ii, iii, and iv.
The Parental Node can determine how many of the Parental Node's m filial nodes have requested the resource by using the using the data the Parental Node has stored in step 607.
As part of step 608, if the Parental Node determines that the resource should be stored in cache 503, then control passes to step 609; otherwise, control passes to step 610.
At step 609, the Parental Node populates its cache with the resource with the resource identifier (or a hash function of the resource identifier) as the index.
At step 610, the Parental Node transmits the resource to the Given Node, and at step 611, the Given Node receives the resource.
It is to be understood that the above-described embodiments are merely illustrative of the present invention and that many variations of the above-described embodiments can be devised by those skilled in the art without departing from the scope of the invention. It is therefore intended that such variations be included within the scope of the following claims and their equivalents.
This application is a continuation of U.S. patent application Ser. No. 09/725,797 (Attorney Docket 500-002us). This application is related to U.S. Pat. No. 7,225,219 B2, entitled “Distributed Caching Architecture For Computer Networks,” (Attorney Docket “500-001us”), which was filed on the same date as the parent of this application and which is incorporated by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 09725737 | Nov 2000 | US |
Child | 12467000 | US |