Dynamic recommendations taken over time for reservations of information technology resources

Information

  • Patent Grant
  • 10937036
  • Patent Number
    10,937,036
  • Date Filed
    Thursday, June 13, 2013
    11 years ago
  • Date Issued
    Tuesday, March 2, 2021
    3 years ago
Abstract
Embodiments are directed towards providing dynamic recommendations of reserving information technology resources over time that may be visually displayed over that time. In one embodiment, the recommendations may be determined based on an analysis of actual usage data obtained over a prior time period and used to predict future resource demands. The subject innovations enable a user to dynamically perform various ‘what-if’ analysis to determine optimum purchase times, and configurations. In some embodiments, the user is further provided information about currently purchased resource under-utilizations to enable the user to redistribute work, release resources, or take other actions directed towards improving management of their IT budget. While subject innovations are may be directed towards managing IT resources obtained through one or more cloud computing service providers, some embodiments further allow the user to perform make/buy decisions such as when to use in-house resources versus using cloud-based resources.
Description
TECHNICAL FIELD

The present invention relates generally to computer automated activity based budgeting and forecasting, and more particularly, but not exclusively to providing an interactive mechanism for managing reservations of Information Technology (IT) resources, such as cloud based IT resources.


BACKGROUND

Cloud-based computing may be defined as the use of computing resources (hardware and software) that are delivered as a service over a network, such as the Internet. Cloud-based computing is often argued to provide numerous benefits to a business, including rapid scalability, availability, and cost savings. Some providers of cloud-based computing services allow users to buy access to their resources from the cloud on a pay-per-use basis; other providers further provide an ability of a user to pay to reserve resources for an extended period of time. Other providers provide still different plans for use of their services that might include variable rate plans, reservations based on differing costing models over different time periods.


While cloud-based services may provide cost savings, it remains up to the user purchasing the services to determine when and how to make use of the services, so that cost benefits may be obtained. However, there appears to be little solutions designed to help IT managers, and/or other users of cloud-based services, to manage or communicate costs of their IT resource consumption. IT managers, and other users, are often required to predict with little assistance as to when to purchase a cloud-based service, or even how much to purchase. This lack of adequate tools for the IT manager even extends to a lack of tools usable to assess when to purchase IT resources for in-house versus when to use cloud-based services. With an ever increasing number of cloud service providers becoming available, and an ever growing number of different purchase plans being made available, making such IT evaluations is becoming more and more complex. Thus, it is with respect to these considerations and others that the invention has been made.





BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following drawings. In the drawings, like reference numerals refer to like parts throughout the various figures unless otherwise specified. For a better understanding of the present invention, reference will be made to the following Detailed Description, which is to be read in association with the accompanying drawings, wherein:



FIG. 1 is a system diagram showing components of an environment in which at least one of the various embodiments may be practiced;



FIG. 2 shows one embodiment of a client device that may be included in a system in accordance with the embodiments;



FIG. 3 shows one embodiment of a network device that may be included in a system implementing at least one of the various embodiments;



FIG. 4 illustrates a logical flow diagram showing one embodiment of a process usable to manage and display recommendations for reserving IT resources, such as cloud IT resources; FIGS. 5-6 illustrate non-limiting, non-exhaustive examples of interfaces for managing and displaying IT resource reservations; and



FIG. 7 illustrates one non-limiting, non-exhaustive example of data analysis useable to calculate reserved resources for a single resource type.





DESCRIPTION OF THE VARIOUS EMBODIMENTS

The present invention now will be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific embodiments by which the invention may be practiced. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Among other things, the present invention may be embodied as methods or devices. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. The following detailed description is, therefore, not to be taken in a limiting sense.


Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The phrase “in one embodiment” as used herein does not necessarily refer to the same embodiment, though it may. Furthermore, the phrase “in another embodiment” as used herein does not necessarily refer to a different embodiment, although it may. Thus, as described below, various embodiments of the invention may be readily combined, without departing from the scope or spirit of the invention.


In addition, as used herein, the term “or” is an inclusive “or” operator, and is equivalent to the term “and/or,” unless the context clearly dictates otherwise. The term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of “a,” “an,” and “the” include plural references. The meaning of “in” includes “in” and “on.”


As used herein, the term “instance” refers to a configuration of a computing resource, including hardware and software. In one embodiment, an instance is further defined based on a geographic location in which the computing device physically resides. Thus, for example, an instance might be defined based on its hardware, the software made available for use, and where the computing device resides. In some embodiments, an instance is further definable based on a network connection to the hardware device. Moreover, the term “resource” may be used interchangeably with the term “instance,” where a resource is a definable configuration of a computing device, including its hardware, software, and physical location.


Typically, where a resource resides within a cloud-computing environment, the resource may be leased, or otherwise purchased, for various time periods. For example, the resource may be purchased for use at once, and purchased based on a pay per use plan. In other cases, a resource might be leased by reserving use of the resource for some time period, such as six months, one year, two years, three years, or the like. Further, resources may be purchased based on a combination of fixed fees, and variable fees. The fixed fees may be based on the lease/purchase time, while the variable fees may be based on actual usage of the resource. For example, a user might purchase for three years, and use the resource on an average over the three years at 60% of the time. Thus, an effective fee rate may be determined that varies over some time period based on a combination of the fixed fees and the variable usage rate fees.


The following briefly describes the embodiments of the invention in order to provide a basic understanding of some aspects of the invention. This brief description is not intended as an extensive overview. It is not intended to identify key or critical elements, or to delineate or otherwise narrow the scope. Its purpose is merely to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.


Briefly stated, the subject innovations are directed towards providing dynamic recommendations for reserving information technology resources over time that may be visually displayed over that time frame. In one embodiment, the recommendations may be determined based on an analysis of actual usage data obtained over a prior time period that is then used to predict future resource demands. The subject innovations enable a user to perform various ‘what-if’ analysis to determine optimum purchase times, and configurations. In some embodiments, the user is further provided information about currently purchased resource under-utilizations to enable the user to redistribute work, release resources, or take other actions directed towards improving management of their IT budget. While subject innovations may be directed towards managing IT resources obtained through one or more cloud computing service providers, some embodiments further allow the user to perform make/buy decisions such as when to use in-house resources versus using cloud-based resources, or to use on-demand resources versus using reserved resources.


In some embodiments, the recommendation analysis may assume that a future usage of resources will be substantially the same as a previous time period usage. Substantially the same might be based on using some statistical parameter describing the historical usage, including a mean, mode, median value, or the like. However, other more complex algorithms may be employed, including usage of a machine learning model, linear prediction models, non-linear prediction models, a covariance estimation approach, a time-varying estimation model, or any of a variety of other models. For example, in some embodiments, a model that accounts for trends or varying use of resources might be employed. Moreover, the analysis may be configured to account for various service level agreement cost implications, as well as various purchase/lease options provided by a given resource provider. In fact, virtually any parameter that might affect a cost of the resource to the user may be used to provide to the user a visual cost recommendation over time. In some embodiments, a user might select a particular time period, such as a current date, and be provided with a table reflecting a recommended purchase list based on an optimum costing forecast model. In other embodiments, a table might be displayed indicating resources that have been reserved, but are currently going unused. In some embodiments, a recommendation of how to reallocate the unused resources may be provided.


Illustrative Operating Environment



FIG. 1 shows components of one embodiment of an environment in which at least one of the various embodiments may be practiced. Not all the components may be required to practice various embodiments, and variations in the arrangement and type of the components may be made. As shown, system 100 of FIG. 1 includes local area networks (“LANs”)/wide area networks (“WANs”)-(network) 111, wireless network 110, client devices 101-104, Budgeting and Finance System (BFS) 107, and cloud services 120 and 130. Within cloud service 120 are illustrated instances (or resources) 121-124; while within cloud service 130 are illustrated instances (or resources) 121-133.


Generally, client devices 102-104 may include virtually any portable computing device capable of receiving and sending a message over a network, such as network 111, wireless network 110, or the like. Client devices 102-104 may also be described generally as client devices that are configured to be portable. Thus, client devices 102-104 may include virtually any portable computing device capable of connecting to another computing device and receiving information. Such devices include portable devices such as, cellular telephones, smart phones, display pagers, radio frequency (RF) devices, infrared (IR) devices, Personal Digital Assistants (PDA's), handheld computers, laptop computers, wearable computers, tablet computers, integrated devices combining one or more of the preceding devices, or the like. As such, client devices 102-104 typically range widely in terms of capabilities and features. For example, a cell phone may have a numeric keypad and a few lines of monochrome Liquid Crystal Display (LCD) on which only text may be displayed. In another example, a web-enabled mobile device may have a touch sensitive screen, a stylus, and several lines of color LCD in which both text and graphics may be displayed.


Client device 101 may include virtually any computing device capable of communicating over a network to send and receive information, including messaging, performing various online actions, or the like. The set of such devices may include devices that typically connect using a wired or wireless communications medium such as personal computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, network Personal Computers (PCs), or the like. In one embodiment, at least some of client devices 102-104 may operate over wired and/or wireless network. Today, many of these devices include a capability to access and/or otherwise communicate over a network such as network 111 and/or even wireless network 110. Moreover, client devices 102-104 may access various computing applications, including a browser, or other web-based application.


In one embodiment, one or more of client devices 101-104 may be configured to operate within a business or other entity to perform a variety of services for the business or other entity. For example, client devices 101-104 may be configured to operate as a web server, an accounting server, a production server, an inventory server, or the like. However, client devices 101-104 are not constrained to these services and may also be employed, for example, as an end-user computing node, in other embodiments. Further, it should be recognized that more or less client devices may be included within a system such as described herein, and embodiments are therefore not constrained by the number or type of client devices employed. In any event, one or more of client devices 101-104 may be considered as in-house resources, or more generally on-demand resource. As used herein, the term “on-demand resource,” refers to resources in which a user may pay for compute capacity by the hour, or some other time period, with no long-term commitments. Further, as used herein, the term “reserved resource,” refers to resources in which the user is provided the option to make an initial payment for each resource to be reserved for use in some future time period.


In another embodiment, one or more of client devices 101-104 may be configured to access various services from one or more of the resources within various cloud-based services, such as cloud services 120 and/or 130.


A web-enabled client device may include a browser application that is configured to receive and to send web pages, web-based messages, or the like. The browser application may be configured to receive and display graphics, text, multimedia, or the like, employing virtually any web-based language, including a wireless application protocol messages (WAP), or the like. In one embodiment, the browser application is enabled to employ any of a variety of scripting languages, including for example, JavaScript, as well as any of a variety of markup languages, including for example Standard Generalized Markup Language (SGML), HyperText Markup Language (HTML), eXtensible Markup Language (XML), HTML5, or the like, to display and send a message. In one embodiment, a user of the client device may employ the browser application to perform various actions over a network.


Client devices 101-104 also may include at least one other client application that is configured to receive and/or send data, including resource recommendation information, between another computing device. The client application may include a capability to provide requests and/or receive data relating to resource recommendations. In other embodiments, BFS 107 may be configured to provide to the client devices 101-104 visual representations of resource recommendations usable to enable a user to make IT decisions for allocating budget to IT resources and for reserving use of one or more resources, such as cloud-based resources.


Wireless network 110 is configured to couple client devices 102-104 and its components with network 111. Wireless network 110 may include any of a variety of wireless sub-networks that may further overlay stand-alone ad-hoc networks, or the like, to provide an infrastructure-oriented connection for client devices 102-104. Such sub-networks may include mesh networks, Wireless LAN (WLAN) networks, cellular networks, or the like.


Wireless network 110 may further include an autonomous system of terminals, gateways, routers, or the like connected by wireless radio links, or the like. These connectors may be configured to move freely and randomly and organize themselves arbitrarily, such that the topology of wireless network 110 may change rapidly.


Wireless network 110 may further employ a plurality of access technologies including 2nd (2G), 3rd (3G), 4th (4G), 5th (5G) generation radio access for cellular systems, WLAN, Wireless Router (WR) mesh, or the like. Access technologies such as 2G, 3G, 4G, and future access networks may enable wide area coverage for mobile devices, such as client devices 102-104 with various degrees of mobility. For example, wireless network 110 may enable a radio connection through a radio network access such as Global System for Mobil communication (GSM), General Packet Radio Services (GPRS), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), or the like. In essence, wireless network 110 may include virtually any wireless communication mechanism by which information may travel between client devices 102-104 and another computing device, network, or the like.


Network 111 is configured to couple network devices with other computing devices, including, BFS 107, client device(s) 101, and through wireless network 110 to client devices 102-104. Network 111 is enabled to employ any form of computer readable media for communicating information from one electronic device to another. Also, network 111 can include the Internet in addition to local area networks (LANs), wide area networks (WANs), direct connections, such as through a universal serial bus (USB) port, other forms of computer-readable media, or any combination thereof. On an interconnected set of LANs, including those based on differing architectures and protocols, a router acts as a link between LANs, enabling messages to be sent from one to another. In addition, communication links within LANs typically include twisted wire pair or coaxial cable, while communication links between networks may utilize analog telephone lines, full or fractional dedicated digital lines including T1, T2, T3, and T4, Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines (DSLs), wireless links including satellite links, or other communications links known to those skilled in the art. For example, various Internet Protocols (IP), Open Systems Interconnection (OSI) architectures, and/or other communication protocols, architectures, models, and/or standards, may also be employed within network 111 and wireless network 110. Furthermore, remote computers and other related electronic devices could be remotely connected to either LANs or WANs via a modem and temporary telephone link. In essence, network 111 includes any communication method by which information may travel between computing devices.


Additionally, communication media typically embodies computer-readable instructions, data structures, program modules, or other transport mechanism and includes any information delivery media. By way of example, communication media includes wired media such as twisted pair, coaxial cable, fiber optics, wave guides, and other wired media and wireless media such as acoustic, RF, infrared, and other wireless media. Such communication media is distinct from, however, processor-readable storage devices described in more detail below.


BFS 107 may include virtually any network device usable to provide resource recommendation services, such as network device 200 of FIG. 2. In one embodiment, BFS 107 employs various techniques to create and display resource recommendations. BFS 107 may include applications for generating cost traces, and predications within a resource recommendation model. Furthermore, BFS 107 may include applications for visualizing the generated costs and recommendations. BFS 107 may also enable the user to perform various what-if analysis and dynamically view differing resource recommendations based in part on user input parameters, and historical usage data.


Devices that may operate as BFS 107 include various network devices, including, but not limited to personal computers, desktop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, server devices, network appliances, or the like. It should be noted that while BFS 107 is illustrated as a single network device, the invention is not so limited. Thus, in another embodiment, BFS 107 may represent a plurality of network devices. For example, in one embodiment, BFS 107 may be distributed over a plurality of network devices and/or implemented using a cloud architecture.


Moreover, BFS 107 is not limited to a particular configuration. Thus, BFS 107 may operate using a master/slave approach over a plurality of network devices, within a cluster, a peer-to-peer architecture, and/or any of a variety of other architectures. Thus, BFS 107 is not to be construed as being limited to a single environment, and other configurations, and architectures are also envisaged. BFS 107 may employ processes such as described below in conjunction with FIGS. 4-22 to perform at least some of its actions.


Cloud services 120 and 130 represent cloud-based service providers that provider for use by a user various instances or resources. While cloud services 120 is illustrated to include resources 121-124, and cloud services 130 is illustrated to include resources 131-133, other implementations are not so constrained. Thus, it should be understood that cloud services 120 and 130 may include many more or less resources than illustrated in FIG. 1. Further resources 121-124 and 131-133 are intended to be representative and not actual reflections of configurations. Thus, a cloud service might include resources that are configured into clusters, are rack components, a virtual machine, a plurality of different computing devices, reside in differing geographic locations around the United States, or other locations, or the like. As noted above, each resource or instance may be defined based on its hardware, software, and physical location. However, other parameters may also be used, including its service level agreement, lease/purchase rate plans, or the like. In some embodiments, a resource might be purchased at once, and billed to the user based on usage plus the purchase fees, while others might be reserved for a period of time, such as one or three years, and billed out based on an effective fee rate that in turn is based on a usage rate and a fixed fee rate schedule.


In one embodiment, cloud services 120 and 130 might be considered to be managed by different service providers; however, in other embodiments, cloud services 120 and 130 might represent services provided over different locations, different arrangements of services, or the like. For example, in one embodiment cloud services might be partitioned based on different service level agreements, different locations, different types of architectures, different security levels, and/or any of a variety of other criteria.


In any event, cloud services 120 and 130, may be configured to provide information about actual usage of resources, as well as various information about the resource, including its configuration and fee rate plans, or the like, to BFS 107, which may then employ the information in part to determine a resource recommendation. In some embodiments, configurations of client devices or other in-house resources, their costs schedules, and the like, might also be sent to BFS 107, so that BFS 107 might consider in-house (or more generally, on-demand) resources as well as outsourced (or more generally, reserved) resource usages (e.g., cloud services) in determining resource recommendations. It should be noted that while FIG. 1 illustrates cloud services, other forms of outsourced services may also be considered, and thus, subject innovations are not constrained to merely considering cloud services.


Moreover, BFS 107 might employ a process such as described below in conjunction with FIG. 4 to perform and provide resource recommendations. Further, BFS 107 might provide graphical interfaces such as described below in conjunction with FIGS. 5-6 for use in managing a set of recommendations for reserving resources, highlighting underused reserved resources, and even providing recommendations for reallocating resources.


Illustrative Client Device



FIG. 2 shows one embodiment of client device 200 that may be included in a system implementing at least one of the various embodiments. Client device 200 may include many more or less components than those shown in FIG. 2. However, the components shown are sufficient to disclose an illustrative embodiment for practicing the present invention. Client device 200 may represent, for example, one embodiment of at least one of client devices 101-104 of FIG. 1. It should be recognized that, as discussed above, client devices may operate as an interface mechanism into a cloud-based service, and/or as a resource that may be managed along with the resources obtained through the cloud-based service.


As shown in the figure, client device 200 includes a central processing unit (“CPU”) 202 in communication with a mass memory 226 via a bus 234. Client device 200 also includes a power supply 228, one or more network interfaces 236, an audio interface 238, a display 240, a keypad 242, and an input/output interface 248. Power supply 228 provides power to client device 200. A rechargeable or non-rechargeable battery may be used to provide power. The power may also be provided by an external power source, such as an AC adapter or a powered docking cradle that supplements and/or recharges a battery.


Client device 200 may optionally communicate with a base station (not shown), or directly with another computing device. Network interface 236 includes circuitry for coupling client device 200 to one or more networks, and is constructed for use with one or more communication protocols and technologies including, but not limited to, global system for mobile communication (“GSM”), code division multiple access (“CDMA”), time division multiple access (“TDMA”), user datagram protocol (“UDP”), transmission control protocol/Internet protocol (“TCP/IP”), short message service (“SMS”), general packet radio service (“GPRS”), WAP, ultra wide band (“UWB”), IEEE 802.16 Worldwide Interoperability for Microwave Access (“WiMax”), session initiated protocol/real-time transport protocol (“SIP/RTP”), or any of a variety of other wireless communication protocols. Network interface 236 is sometimes known as a transceiver, transceiving device, or network interface card (“NIC”).


Audio interface 238 is arranged to produce and receive audio signals such as the sound of a human voice. For example, audio interface 238 may be coupled to a speaker and microphone (not shown) to enable telecommunication with others and/or generate an audio acknowledgement for some action. Display 240 may be a liquid crystal display (“LCD”), gas plasma, light emitting diode (“LED”), or any other type of display used with a computing device. Display 240 may also include a touch sensitive screen arranged to receive input from an object such as a stylus or a digit from a human hand.


Keypad 242 may comprise any input device arranged to receive input from a user. For example, keypad 242 may include a push button numeric dial, or a keyboard. Keypad 242 may also include command buttons that are associated with selecting and sending images.


Client device 200 also comprises input/output interface 248 for communicating with external devices, such as a headset, or other input or output devices not shown in FIG. 2. Input/output interface 248 can utilize one or more communication technologies, such as USB, infrared, Bluetooth™, or the like.


Mass memory 226 includes a Random Access Memory (“RAM”) 204, a Read-only Memory (“ROM”) 222, and other storage means. Mass memory 226 illustrates an example of computer readable storage media (devices) for storage of information such as computer readable instructions, data structures, program modules or other data. Mass memory 226 stores a basic input/output system (“BIOS”) 224 for controlling low-level operation of client device 200. The mass memory also stores an operating system 206 for controlling the operation of client device 200. It will be appreciated that this component may include a general-purpose operating system such as a version of UNIX, or LINUX™, or a specialized client communication operating system such as Windows Mobile™, Google Android™, Apple iOS™, or the Symbian® operating system. The operating system may include, or interface with a Java virtual machine module that enables control of hardware components and/or operating system operations via Java application programs.


Mass memory 226 further includes one or more data storage 208, which can be utilized by client device 200 to store, among other things, applications 214 and/or other data. For example, data storage 208 may also be employed to store information that describes various capabilities of client device 200. The information may then be provided to another device based on any of a variety of events, including being sent as part of a header during a communication, sent upon request, or the like. At least a portion of the information may also be stored on a disk drive or other computer-readable storage device (not shown) within client device 200. Data storage 208 may also store various financial data, including reservation data, usage data, and the like, that may reside within a database, text, spreadsheet, folder, file, or the like. Such financial data may also be stored within any of a variety of other computer-readable storage devices, including, but not limited to a hard drive, a portable storage device, or the like, such as illustrated by non-transitory computer-readable storage device 230.


Applications 214 may include computer executable instructions which, when executed by client device 200, transmit, receive, and/or otherwise process network data. Examples of application programs include, but are not limited to calendars, search programs, email clients, IM applications, SMS applications, voice over Internet Protocol (“VoIP”) applications, contact managers, task managers, transcoders, database programs, word processing programs, security applications, spreadsheet programs, games, search programs, and so forth. Applications 214 may include, for example, browser 218 and resource recommendation interface (I/F) 219.


Browser 218 may include virtually any application configured to receive and display graphics, text, multimedia, and the like, employing virtually any web based language. In one embodiment, the browser application is enabled to employ HDML, WML, WMLScript, JavaScript, SGML, HTML, XML, and the like, to display and send a message. However, any of a variety of other web-based languages may be employed. In one embodiment, browser 218 may enable a user of client device 200 to communicate with another network device, such as BFS 107 of FIG. 1. In one embodiment, browser 218 may enable a user to view and/or manipulate resource data, including creating resource recommendations, adding/purchasing/reallocating resources, modifying resource reservation models, rendering visualizations of resource recommendations and related what-ifs, or the like.


In at least one of the various embodiments, a user may employ client device 200 to create and manage IT resource recommendations and to access information stored or otherwise managed through BFS 107. In at least one of the various embodiments, a user may enter various types of data into a resource recommendation system accessible through BFS 107. Also, in at least one of the various embodiments, the user may be enabled to perform a variety of actions on the data, including, queries, comparisons, summations, analysis, or the like. In some embodiments, a user may employ client 200 to create one more resource reservation models.


Resource recommendation I/F (RRI) 219 provides another mechanism for interacting with BFS 107. RRI 219 may operate as a separate application providing and managing communications with BFS 107 over a network and providing for display of user interfaces, including, but not limited to those described below. Thus, in some embodiments, the user might employ browser 218 or RRI 219 to communicate with BFS 107, provide data to BFS 107, and otherwise manage IT resource reservations. It should be noted that while the subject innovations are directed towards managing It resource reservations, other actions might also be performed, including, managing other aspects of IT resources, including budgeting, tracking work flow, up/down times of resources, application usages, back-up management, recovery management, and any of a variety of other IT management activities.


Illustrative Network Device



FIG. 3 shows one embodiment of network device 300 that may be included in a system implementing at least one of the various embodiments. Network device 300 may include many more or less components than those shown. The components shown, however, are sufficient to disclose an illustrative embodiment for practicing the invention. Network device 300 may represent, for example, BFS 107 of FIG. 1.


Network device 300 includes processing unit 312, video display adapter 314, and a mass memory, all in communication with each other via bus 322. The mass memory generally includes RAM 316, ROM 332, and one or more permanent mass storage devices, such as hard disk drive 328, tape drive, optical drive, flash drive, and/or floppy disk drive. The mass memory stores operating system 320 for controlling the operation of network device 300. Any general-purpose operating system may be employed. Basic input/output system (“BIOS”) 318 is also provided for controlling the low-level operation of network device 300. As illustrated in FIG. 3, network device 300 also can communicate with the Internet, or some other communications network, via network interface unit 310, which is constructed for use with various communication protocols including the TCP/IP protocol. Network interface unit 310 is sometimes known as a transceiver, transceiving device, or network interface card (NIC). Network device 300 also includes input/output interface 324 for communicating with external devices, such as a headset, or other input or output devices not shown in FIG. 3. Input/output interface 324 can utilize one or more communication technologies, such as USB, infrared, Bluetooth™, or the like.


The mass memory as described above illustrates another type of processor-readable storage media. Processor-readable storage media (devices) may include volatile, nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer readable storage media include RAM, ROM, Electronically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), digital versatile disks (DVD), Blu-Ray, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other physical medium which can be used to store the desired information and which can be accessed by any computing device.


As shown, data stores 354 may include a database, text, spreadsheet, folder, file, or the like, that may be configured to maintain and store various resource recommendation models, resource data, resource usage logs, resource configuration data, service level agreements, cloud-service provider contract data, or the like. Data stores 354 may further include program code, data, algorithms, or the like, for use by a processor, such as central processing unit (CPU) 312 to execute and perform actions. In one embodiment, at least some of data and/or instructions stored in data stores 354 might also be stored on another device of network device 300, including, but not limited to cd-rom/dvd-rom 326, hard disk drive 328, or other computer-readable storage device resident on network device 300 or accessible by network device 300 over, for example, network interface unit 310.


The mass memory also stores program code and data. One or more applications 350 are loaded into mass memory and run on operating system 320. Examples of application programs may include transcoders, schedulers, calendars, database programs, word processing programs, Hypertext Transfer Protocol (HTTP) programs, customizable user interface programs, IPSec applications, encryption programs, security programs, SMS message servers, IM message servers, email servers, account managers, and so forth. Mass memory may also include web services 356, and resource recommender 357.


Web services 356 represent any of a variety of services that are configured to provide content, over a network to another computing device. Thus, web services 356 include for example, a web server, a File Transfer Protocol (FTP) server, a database server, a content server, or the like. Web services 356 may provide the content over the network using any of a variety of formats, including, but not limited to WAP, HDML, WML, SGML, HTML, XML, compact HTML (cHTML), extensible (xHTML), or the like.


In one embodiment, web services 356 may provide an interface for accessing and manipulating data in a data store, such as data stores 354, or the like. In another embodiment, web services 356 may provide for interacting with resource recommender 357 that may enable a user to access and/or otherwise manage resource reservations, and/or other IT management related actions.


In at least one of the various embodiments, resource recommender 357 may enable users to generate financial resource recommendation models, establish what-if scenarios, display graphic plots for reserving resources, determine underused, including unused, resources, or the like. Resource recommender 357 may be configured in one embodiment, to employ a process such as described below in conjunction with FIG. 4 to perform at least some of its actions. Further resource recommender 357 may provide various user interfaces including those discussed below in conjunction with FIGS. 5-6.


Generalized Operation


The operation of certain aspects of the invention will now be described with respect to FIGS. 4-6. The operations of the processes described below may, in one embodiment, be performed within BFS 107 of FIG. 1, and/or displayed at one or more screens within one or more client devices 101-104 of FIG. 1.



FIG. 4 illustrates a logical flow diagram showing one embodiment of a process 400 usable to manage and display recommendations for reserving IT resources, such as cloud IT resources, in-house resources, on-demand resources, reserved resourced, and the like.


In one embodiment, a user may select to initially arrange for usage of various IT resources directly through one or more cloud service providers. During this initial stage, the user might contract for initial use of various resources, reserve the resources based on various terms of use, including agreements over a six month, one year, two year, three year, or other time period. Moreover, it should be understood, that the user may select to manage IT resources from a plurality of different cloud service providers, manage in-house (or on-demand) resources, and/or manage IT resources obtained from other than cloud service providers and/or other reservation type service providers.


While such actions may be performed prior to beginning process 400, in other embodiments, initial contracting for services might be performed within process 400, such as at step 402. Thus, process 400 begins, after a start block, at block 402, where the user may negotiate initial IT resource (instance) purchases, reservations, contracts, and the like. In one embodiment, block 402 might be performed using interfaces provided by BFS 107; however, as noted above, block 402 might be performed independent of use of BFS 107, as suggested by dashed block 402.


In any event, proceeding to block 404, the user may establish an account for reservation management services through BFS 107, and provide information about resources (instances) to be managed, including, cloud services, in-house services, and the like. In one embodiment, resource information might be input automatically through a request by BFS 107 to a cloud service provider, on behalf of the user. In other embodiments, an interface might be established with a user's computing device to enable access to various data about the resources. In still another embodiment, the user might directly input data about resources to be tracked and managed by BFS 107.


Initially, the user may be provided with an interface illustrating a list of resources that are to be managed by BFS 107, where the list might provide an ability for the user to view details about a configuration of the resource, a location of the resource, a service level agreement for the resource, uptime/downtime information about the resource, and a variety of other information about the resource.


Proceeding to block 406, resources identified within BFS 107 are then tracked for usage by the user and/or the user's business. In one embodiment, the usage data might be directly sent by the in-house (or on-demand), cloud services (or reservation services), or the like, using any of a variety of mechanisms. For example, an agreement might be established that the tracked resources automatically provide usage data to BFS 107 based on regular schedules, based on queries by BFS 107, and/or a combination of events, conditions, or the like. Such usage data may include any changes in a configuration of a resource, as well as loads on the resource, up/down times of the resources, a cost of the resource, and any of a variety of other related information.


At least some of the tracked data may be used to make predictions on future usage of a resource, determine when to move an in-house resource usage to a cloud resource (or the reverse), determine when to purchase additional resources, release a resource, re-allocate a resource usage, or the like. In some embodiments, tracked data for a defined prior period of time may be used to make recommendations. For example, a three month prior period of time might be used to determine recommendations. However, in other embodiments, tracked data may be used based on various models, including, error covariance models, learning models, or the like, where historical data may be consolidated into various parameters of the model over virtually any time period.


At any time that the user wishes to perform recommendations analysis, process 400 moves to block 408. This may occur, for example, when the user selects an interface into resource recommender 357 of network device 300 of FIG. 3.


Processing then flows to block 410, where recommendations are dynamically displayed to the user. One such non-limiting, non-exhaustive interface is discussed in more detail below in conjunction with FIG. 5. Briefly, however, the user may be provided with various strategy interfaces.


At decision block 412, the user might select to modify various user input parameters or assumptions useable to determine resource recommendations. For example, the user might be able to vary a time frame in which the resources are to be committed to by the user, vary an upfront cost parameter, and/or vary a number of recommended resources to be reserved, or the like. The user may further update various resources to be considered, their configurations, contracts, or the like.


If the user selects to modify any assumptions, input parameters, or the like, processing flows to block 414, where, based on the tracked data, and the user input assumptions, the changes may be used to perform an updated analysis. Then, flowing to block 410, the results are dynamically displayed to the user to indicate changes in the recommendations for reserving resources. As the user varies input parameters, the user may automatically and dynamically view how the changes affect the resulting recommendations (by cycling through blocks 410, 412, and 414). Thus, at least in part, the user may dynamically perform a variety of ‘what-if’ analysis to determine an optimum resource recommendation given the constraints provided by the user, the resource providers, and/or the tracked data.


When the user selects to accept the recommendations (at decision block 412, by no longer modifying inputs) processing flows to block 416. At block 416, the user may take one or more actions based on the recommendations. In one embodiment, this might include selecting a time period within the forecast display of FIG. 5. This action may then result in a display of actions to be taken. One non-limiting, non-exhaustive embodiment of such a display is described in more detail below in conjunction with FIG. 6. The user may then make purchases, release resources, renegotiate contracts for resources, reallocate resource usages, or the like. In one embodiment, such actions might be performed outside of process 400 (as indicated by the dashed block 416), or be performed through another interface provided by BFS 107.


Processing continues to decision block 418, where a determination is made whether to continue to manage tracking and recommending resource reservations, or to terminate the process. If the process is to continue, then the flow may return to block 406. However, in other embodiments, the flow might return to block 404, where additional changes to the resources might be performed by the user, including adding new resources, deleting one or more resources, or the like. Should it be determined that process 400 is to terminate, then flow may return to a calling process to perform other actions.


It will be understood that each component of the illustrations, and combinations of components in these illustrations, can be implemented by computer program instructions. These program instructions may be provided to a processor to produce a machine, such that the instructions, which execute on the processor, create means for implementing the actions specified in the flow component or components. The computer program instructions may be executed by a processor to cause a series of operational steps to be performed by the processor to produce a computer-implemented process such that the instructions, which execute on the processor to provide steps for implementing the actions specified in the flow component or components. The computer program instructions may also cause at least some of the operational steps shown in the components of the flows to be performed in parallel. Moreover, some of the steps may also be performed across more than one processor, such as might arise in a multiprocessor computer system. In addition, one or more components or combinations of components in the flow illustrations may also be performed concurrently with other components or combinations of components, or even in a different sequence than illustrated.


Accordingly, components of the flow illustrations support combinations of means for performing the specified actions, combinations of steps for performing the specified actions and program instruction means for performing the specified actions. It will also be understood that each component of the flow illustrations, and combinations of components in the flow illustrations, can be implemented by special purpose hardware based systems, which perform the specified actions or steps, or combinations of special purpose hardware and computer instructions.


Non-Limiting, Non-Exhaustive Example User Interfaces


The following provides examples of user interfaces usable in conjunction with process 400 of FIG. 4 to enable a user to manage resource recommendations. It should be noted that other interfaces may also be provided. Moreover, it should be noted that the interfaces illustrated in FIGS. 5-6 discussed below may include more or less components that shown. In addition, not all the components may be required to practice various embodiments, and variations in the arrangement and type of the components may be made.


As shown, however, FIGS. 5-6 illustrate non-limiting, non-exhaustive examples of interfaces for managing and displaying IT resource reservations. As shown in FIG. 5, for example, is a display 500 that provides one portion 502 usable for the user to modify various assumptions, and/or parameters used to determine resource reservations.


For example, the user might be able to modify what time frame for which the user might wish to commit to for reserving resources, as shown in portion 502, labeled 1. The displayed time frame might be specific to a given cloud service provider. Thus, while illustrated in FIG. 5 to provide up to three years, other time frames might be provided instead. Briefly, this time frame allows the user to determine the time period over which they wish to make purchases, allowing the user to select immediate purchases, all at once, or to make purchases over the course of the contract. Also illustrated, labeled 2, the user might provide an input indicating how much the user wishes to pay upfront, or at a beginning of a contract period. In one embodiment, the user might be able to input a specific value, or a generic parameter, such as low/medium/high, or the like. Input, labeled 3 in FIG. 5, allows the user to modify resource recommendations from an initial computed recommendation, to some value less than the recommended amount of resources. As illustrated, the recommended value is 15 resources, for which the user might select the recommended value or some value less.


As shown in the lower portion 503, of display 500 provides an immediate and dynamically changing resource recommendation chart over time. Portion 503 may dynamically change based on changes in inputs, assumptions, or the like, by the user. Thus, should the user change any of the input assumptions in portion 502, they may immediately (or as quickly as reasonable given network connections, or the like) view the impact to the recommendations.


As shown in portion 503, are lines 510, 512, 514, and 516. Line 510 is directed towards illustrating over time a costing forecast based on no reserved instances. Line 512 is directed towards illustrating over time a costing forecast that is based on the recommended number of instances. Line 514 illustrates a costing flow over time should the user select to take no actions, including purchases, re-allocations, or the like, while line 516 illustrates costing flow over time should the user follow the recommendations for reserving and/or otherwise managing resources provided by process 400 of FIG. 4. As illustrated in the lower portion 502, the user may quickly see a cost difference between the actions the user may take or not take. By varying assumptions, and/or other inputs, including taking none, some, or all of the recommendations, the lower portion 503 can dynamically update and reflect changes to the recommendations.


As discussed above, the recommendations are in part based on historical usage data, service level agreements, contracts, resource configurations, and the like. Thus, when the user selects to turn on/off a resource, use a resource for some time, allow the resource to sit idle or off, and perform a variety of other actions, such factors are considered in determining the changes to the recommendations.


In one embodiment, the historical data used might be constrained to using a particular time window of prior time. For example, in some embodiments, the time window of prior time might be between one to four months. However, other time periods might be used. In some determinations of recommendations, it might be determined that a prior usage reflects a predicted future usage over a remaining portion of the contract for a resource. Thus, in one embodiment, if it is determined that historical usage of the resource is at 80% over the prior time period, then it might be assumed that the future usage will also be at 80% for the remaining time period on the contract. However, other models might be used, including, but not limited to using a mode, median, or other statistical parameter from the prior time period, to predict future usage.


Further, in some embodiments, a comparison might be made between a predicted usage and an actual usage of a resource. The comparison might then be used to generate various errors covariance values, or the like, usable to improve future estimates for recommendations. For example, various machine learning models, or the like, might be used that take into account trends in usage, peak usages, or the like.


Moreover, various recommendation models take into account fixed fees and variable fees for a resource, to improve recommendations. Recommendations may be provided for a given cloud service provider's resources, or be determined across a plurality of cloud service providers. For example, recommendations may be provided on how to manage resources for a given cloud service provider, independent of resources of other providers. However, in other embodiments, recommendations may be provided to the user across a plurality of sources of resources, including recommendations that take into account costing differences between providers, and outsourced resources and in-house resources, as well as to assist in deciding when an actual trade-off between paying as one goes and reservations might occur. In this manner, the user may have an improved visibility of total costs for IT resource reservations. In some embodiments, the user might conduct what-if analysis for comparing using in-house resources versus outsourcing, by inputting different assumptions in portion 502 of FIG. 5, in addition to those illustrated.


In one embodiment, the user may further select a time within lower portion 503 in which to expand display of the recommendations. As illustrated in FIG. 5, a selector bar 520 might be provided to enable the user to select a time period to expand upon. However, other mechanisms might be provided, including a button input sequence or combination, an icon selection, a window for inputting a time period, or the like.


In any event, selection of a time period to expand recommendation display may result in a display interface, such as illustrated in FIG. 6. FIG. 6 is one non-limiting, non-exhaustive example, of a user interface 600 usable for providing a purchase listing for a selected time period. As shown, interface 600 may include purchase list 602 and non-usage list 604.


Purchase list 602 provides a recommendation listing of resources that the user might select to purchase or otherwise reserve for the given time period based on recommendations provided through FIG. 5. Also illustrated, in non-usage list 604 that provides to the user a listing, when such condition exists, of resources that are determined to be unused, or otherwise used at a level below some threshold value. The user may then select to release these resources, or otherwise reallocate usage to these resources. The non-usage list 604 is generated in part based on purchase list 602. Thus, reallocation of a non-used or underutilized resource is not expected to immediately change the purchase list contents. However, the list and the recommendations may change over time for future recommendations. In any event, the user is provided with numerous integrated and dynamic recommendations, and interfaces for managing their IT resources, and thereby enabling the user to improve usage of their IT budgets.


For example, using the above subject innovations, the user will be able to determine when to employ on-demand resources versus reserved resources, or on-demand resources versus an in-house service that might, for example, have been purchases (with upfront fees) and incurs on-going fees from such as maintenance, power, network, physical space over time, or the like.


As an additional example, FIG. 7 illustrates one non-limiting, non-exhaustive example of data analysis useable to calculate reserved resources for a single resource type. In a complete analysis as performed in block 408 of FIG. 4 many resources types would be analyzed in parallel and summarized for use as in FIG. 5 and FIG. 6. It should be understood that, as discussed above, other approaches may be used, as well as other algorithms. Thus, the following example is not to be construed as limiting or otherwise constraining the subject innovations discussed herein.


In some embodiments, the approach discussed in conjunction with FIG. 7 may be performed within at least block 408 of FIG. 4. However, prior to performing calculations, tracking and collecting historical usage data over some time period is performed, such as is described above at block 406 of FIG. 4. Then, a 4 phase calculation would take place to determine the optimal recommendation. In phase 1, a calculation is performed for on-demand, or cloud usage per unit of time. Referring briefly to FIG. 7, chart 700-1 represents one embodiment for displaying of tracked usage data over time. The unit of time may be defined as virtually any time unit, including hours, other portions of a day, days, weeks, or so forth. The units of resources used may represent discrete resources, or bundled resources. Each unit of time is treated as a bucket and 1 unit is added to the bucket for any resource used during the time period. The result seen in chart 700-1 is a histogram of units used in each time. Once data has been collected for a given previous period of time, the results are used to predict future resource usage. In some embodiments, the data is used to project future resource usage in a time period by assuming it is identical to that of the previous time period data has been collected for. However, as discussed above, other prediction algorithms may be used, including those that take into consideration trends in resource usage, identifying temporary peak usages, or the like.


In chart 700-2, each bucket from chart 700-1 is then sorted from the largest bucket to smallest by unit of time (e.g. hour). Chart 700-2 illustrates one possible display showing rank sorted units for each time period.


In phase two, a calculation is performed to determine a “trade-off percentage” of a given reservation period. This represents the percentage of a time in a period when it costs less to purchase a reservation when cost is amortized over the entire period rather than when using on-demand resources. While a variety of equations may be employed, one such example equation might be:

Trade-off percentage=[U+(RR*TL)]/(ODR*TL)

where U represents an upfront cost; RR represents a reserved rate; TL represents a time of reservation; and ODR represents an on-demand rate. Once we have the trade-off percentage, it can be used to calculate a specific number of hours a resource is to be allocated in a time period before it costs less to purchase a reservation when the cost is amortized over the same period. This can be calculated, in one embodiment, using the following equation:

Trade-off Hours=Trade-off percentage*TL


In phase 3, using the results from phases 1 and 2, optimal resources to purchase for a given time may be determined. Chart 700-3 illustrates one embodiment of a display showing a result for determining a trade-off point (e.g., trade-off hour). Once the time bucket is selected using the calculated trade-off hour, that bucket's unit value is then definable, in one embodiment, as the optimal number of resources of the type in question to purchase in the next time period. This is because the number of resources in that bucket represents the number of resources where it is more cost effective to purchase reserved resources over using on-demand resources.


Chart 700-4 shows how once the optimal number of resource reservations has been selected one could allocate resources over a time period. The number of reserved resources would cover all resources during normal operation while on-demand resources would handle resources above the optimal number of resource reservations.


By using phases 1-4 on every type of resource a list of optimal reservation purchases can be obtained and summarized. FIG. 5 and FIG. 6 display how one may visualize the spread of these purchases across a time period. Purchasing resources over time then may reduce the upfront costs associated with purchasing a large number of reservations at once. This also may lower the risk of purchasing many long term reservations by temporarily using on-demand resources for a time even if it's cost-optimal to buy reserved resources. One algorithm to achieve this would be evenly spreading the number of reservations purchased over an entire time period. Others may also be used, as discussed above. In FIG. 5 the recommended costs line is the sum of on-demand rates, upfront-fees and reserved rates for all current resources given recommended purchases and the time the purchase has been recommended in.


The above specification, examples, and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.

Claims
  • 1. A computer implemented method for modifying a recommendation of an optimal reservation number of computing resources, wherein a server computer having one or more processors, a memory, and a network interface executes logic to perform actions, comprising: instantiating a resource recommender program in the memory that performs actions, including: tracking historical usage of each of a plurality of the computing resources over time buckets that represent units of time, wherein the tracked historical usage includes: changes in a configuration of the computing resources, up time of the computing resources, down time of the computing resources, and loads placed on the computing resources;identifying, by machine learning algorithms temporary peaks in the usage of the computing resources using the tracked historical usage, to predict future computing resource usage over the time buckets, wherein the prediction include determinations of: when to release the computing resources, when to reallocate the computing resources, when to move an in-house resource to a cloud resource, and when to move the cloud resource back to the in-house resource;providing, a trade-off percentage of a reservation for each of the computing resources based on a sum of an upfront cost and a reserved rate multiplied by a time of the reservation, the sum further divided by a mathematical product between an on-demand rate and the time of the reservation;providing, by using the trade-off percentage, a reservation period of a computing resource, of the plurality of computing resources, wherein the reservation period represents a time before the computing resource costs less to purchase as the reservation when its cost is amortized over the reservation period;providing, based on the time buckets further ranked from largest to smallest and the trade-off percentage, a trade-off point representing a number of the computing resources where it is cost effective to purchase reserved computing resources over using on-demand computing resources; anddetermining a recommendation for an optimum forecast model based on the ranked time buckets, the trade-off percentage, the trade-off point, the optimal reservation number of the computing resources over the reservation period, service level agreements, contracts, and the historical usage, wherein the reserved computing resources cover the computing resources during normal operation and the on-demand computing resources cover the computing resources above the optimal reservation number of the computing resources; andinstantiating a user interface on a client computer to display the recommendation and also display selectable control elements; andin response to an input provided by a user, immediately and dynamically modifying assumptions of the optimum forecast model for the recommendation to generate a new recommendation that is displayed to the user, wherein the new recommendation is based on the modified assumptions of the optimum forecast model that provide a new estimate of corresponding savings or additional costs when fewer of the computing resources are provided than a previously recommended optimal reservation number of the computing resources over the reservation period from one or more different types of providers for one or more portions of the computing resources to improve an understanding of the new recommendation.
  • 2. The method of claim 1, wherein the recommendation further comprises an indication of how to manage the computing resources across a plurality of different resource providers.
  • 3. The method of claim 1, further comprising generating at least one error covariance value for use in determining other recommendations based on a comparison of a predicted usage of the computing resources with an actual usage of the computing resources.
  • 4. The method of claim 1, further comprising: receiving at least one user input regarding at least one of a time frame in which the resources are to be committed to by the user, a variation in a parameter of the upfront cost, or the recommended optimal reservation number of computing resources; andas the at least one user input varies, dynamically modifying the recommendation based in part on the at least one user input.
  • 5. The method of claim 1, further comprising: determining the predicted future usage of the computing resource during a contract; andmodifying the recommendation for a remaining time period of the contract based on the predicted future usage of the computing resource.
  • 6. A system for modifying a recommendation of an optimal reservation number of computing resources, the system comprising: a server computer that includes: a network interface for communicating over one or more of a wired network or a wireless network with a client computer;a memory for storing a resource recommender software program; and one or more hardware processors that instantiate and execute the resource recommender software program that is programmed to perform actions, including: tracking historical usage of each of a plurality of the computing resources over time buckets that represent units of time, wherein the tracked historical usage includes: changes in a configuration of the computing resources, up time of the computing resources, down time of the computing resources, and loads placed on the computing resources;identifying, by machine learning algorithms temporary peaks in the usage of the computing resources using the tracked historical usage, to predict future computing resource usage over the time buckets, wherein the prediction include determinations of: when to release the computing resources, when to reallocate the computing resources, when to move an in-house resource to a cloud resource, and when to move the cloud resource back to the in-house resource;providing a trade-off percentage of a reservation for each of the computing resources based on a sum of an upfront cost and a reserved rate multiplied by a time of the reservation, the sum further divided by a mathematical product between an on-demand rate and the time of the reservation;providing, by using the trade-off percentage, a reservation period of a computing resource, of the plurality of computing resources, wherein the reservation period represents a time before the computing resource costs less to purchase as the reservation when its cost is amortized over the reservation period;providing, based on the time buckets further ranked from largest to smallest and the trade-off percentage, a trade-off point representing a number of the computing resources where it is cost effective to purchase reserved computing resources over using on-demand computing resources; anddetermining a recommendation for an optimum forecast model based on the ranked time buckets, the trade-off percentage, the trade-off point, the optimal reservation number of the computing resources over the reservation period, service level agreements, contracts, and the historical usage, wherein the reserved computing resources cover the computing resources during normal operation and the on-demand computing resources cover the computing resources above the optimal reservation number of the computing resources; andinstantiating a user interface on a client computer to display the recommendation and also display selectable control elements; andin response to an input provided by a user, immediately and dynamically modifying assumptions of the optimum forecast model for the recommendation to generate a new recommendation that is displayed to the user, wherein the new recommendation is based on the modified assumptions of the optimum forecast model that provide a new estimate of corresponding savings or additional costs when fewer of the computing resources are provided than a previously recommended optimal reservation number of the computing resources over the reservation period from one or more different types of providers for one or more portions of the computing resources to improve an understanding of the new recommendation.
  • 7. The system of claim 6, wherein the recommendation further comprises an indication of how to manage the computing resources across a plurality of different resource providers.
  • 8. The system of claim 6, further comprising generating at least one error covariance value for use in determining other recommendations based on a comparison of a predicted usage of the computing resources with an actual usage of the computing resources.
  • 9. The system of claim 6, further comprising: receiving at least one user input regarding at least one of a time frame in which the resources are to be committed to by the user, a variation in a parameter of the upfront cost, or the recommended optimal reservation number of computing resources; andas the at least one user input varies, dynamically modifying the recommendation based in part on the at least one input.
  • 10. The system of claim 6, further comprising: determining the predicted future usage of the computing resource during a contract; andmodifying the recommendation for a remaining time period of the contract based on the predicted future usage of the computing resource.
  • 11. A non-transitory computer readable storage medium for modifying a recommendation of an optimal reservation number of computing resources, on which is recorded computer executable instructions that, when executed by a server computer that includes a memory, a network interface, and one or more processors, cause the one or more processors to instantiate and execute a resource recommender software program that performs actions, comprising: tracking historical usage of each of a plurality of the computing resources over time buckets that represent units of time, wherein the tracked historical usage includes: changes in a configuration of the computing resources, up time of the computing resources, down time of the computing resources, and loads placed on the computing resources;identifying, by machine learning algorithms temporary peaks in the usage of the computing resources using the tracked historical usage to predict future computing resource usage over the time buckets, wherein the prediction include determinations of: when to release the computing resources, when to reallocate the computing resources, when to move an in-house resource to a cloud resource, and when to move the cloud resource back to the in-house resource;providing, by the one or more processors, a trade-off percentage of a reservation for each of the computing resources based on a sum of an upfront cost and a reserved rate multiplied by a time of the reservation, the sum further divided by a mathematical product between an on-demand rate and the time of the reservation;providing, by using the trade-off percentage, a reservation period of a computing resource, of the plurality of computing resources, wherein the reservation period represents a time before the computing resource costs less to purchase as the reservation when its cost is amortized over the reservation period;providing, based on the time buckets further ranked from largest to smallest and the trade-off percentage, a trade-off point representing a number of the computing resources where it is cost effective to purchase reserved computing resources over using on-demand computing resources; anddetermining a recommendation for an optimum forecast model based on the ranked time buckets, the trade-off percentage, the trade-off point, the optimal reservation number of the computing resources over the reservation period, service level agreements, contracts, and the historical usage, wherein the reserved computing resources cover the computing resources during normal operation and the on-demand computing resources cover the computing resources above the optimal reservation number of the computing resources; andinstantiating a user interface on a client computer to display the recommendation and also display selectable control elements; andin response to an input provided by a user, immediately and dynamically modifying assumptions of the optimum forecast model for the recommendation to generate a new recommendation that is displayed to the user, wherein the new recommendation is based on the modified assumptions of the optimum forecast model that provide a new estimate of corresponding savings or additional costs when fewer of the computing resources are provided than a previously recommended optimal reservation number of the computing resources over the reservation period from one or more different types of providers for one or more portions of the computing resources to improve an understanding of the new recommendation.
  • 12. The non-transitory computer readable storage medium of claim 11, further comprising: receiving at least one user input regarding at least one of a time frame in which the resources are to be committed to by the user, a variation in an upfront cost parameter, or a number of recommended resources to be reserved; andas the at least one user input varies, dynamically modifying the recommendation for reserving the plurality of resources based in part on the at least one user input.
  • 13. The non-transitory computer readable storage medium of claim 11, further comprising: determining the predicted future usage of the computing resource during a contract; andmodifying the recommendation for a remaining time period of the contract based on the predicted future usage of the computing resource.
  • 14. The non-transitory computer readable storage medium of claim 11, further comprising: generating at least one error covariance value for use in determining other recommendations based on a comparison of a predicted usage of the computing resources with an actual usage of the computing resources.
  • 15. The non-transitory computer readable storage medium of claim 11, wherein the recommendation further comprises an indication of how to manage the computing resources across a plurality of different resource providers.
CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of U.S. application Ser. No. 13/675,837 filed Nov. 13, 2012, entitled “DYNAMIC RECOMMENDATIONS TAKEN OVER TIME FOR RESERVATIONS OF INFORMATION TECHNOLOGY RESOURCES,” the benefit of the earlier filing date of which is hereby claimed under 35 U.S.C. § 120 and 37 C.F.R. § 1.78, and which is further incorporated by reference.

US Referenced Citations (389)
Number Name Date Kind
4744026 Vanderbei May 1988 A
5249120 Foley Sep 1993 A
5615121 Babayev et al. Mar 1997 A
5619211 Horkin et al. Apr 1997 A
5721919 Morel et al. Feb 1998 A
5758327 Gardner et al. May 1998 A
5799286 Morgan Aug 1998 A
5802508 Morgenstern Sep 1998 A
5903453 Stoddard, II May 1999 A
5970476 Fahey Oct 1999 A
5991741 Speakman et al. Nov 1999 A
6014640 Bent Jan 2000 A
6032123 Jameson Feb 2000 A
6047290 Kennedy et al. Apr 2000 A
6208993 Shadmon Mar 2001 B1
6249769 Ruffin et al. Jun 2001 B1
6253192 Corlett et al. Jun 2001 B1
6308166 Breuker et al. Oct 2001 B1
6321207 Ye Nov 2001 B1
6330552 Farrar et al. Dec 2001 B1
6424969 Gruenwald Jul 2002 B1
6336138 Caswell et al. Dec 2002 B1
6507825 Suh Jan 2003 B2
6578005 Lesaint Jun 2003 B1
6594672 Lampson et al. Jul 2003 B1
6647370 Fu et al. Nov 2003 B1
6738736 Bond May 2004 B1
6789252 Burke et al. Sep 2004 B1
6832212 Zenner et al. Dec 2004 B1
6839719 Wallace Jan 2005 B2
6877034 Machin et al. Apr 2005 B1
6882630 Seaman Apr 2005 B1
6965867 Jameson Nov 2005 B1
6983321 Trinon et al. Jan 2006 B2
7050997 Wood, Jr. May 2006 B1
7130822 Their et al. Oct 2006 B1
7149700 Munoz et al. Dec 2006 B1
7177850 Argenton et al. Feb 2007 B2
7263527 Malcom Aug 2007 B1
7305491 Miller et al. Dec 2007 B2
7308427 Hood Dec 2007 B1
7321869 Phibbs, Jr. Jan 2008 B1
7386535 Kalucha et al. Jun 2008 B1
7418438 Gould et al. Sep 2008 B2
7505888 Legault et al. Mar 2009 B2
7590937 Jacobus et al. Sep 2009 B2
7634431 Stratton Dec 2009 B2
7653449 Hunter et al. Jan 2010 B2
7664729 Klein et al. Feb 2010 B2
7703003 Payne et al. Apr 2010 B2
7725343 Johanson et al. May 2010 B2
7742961 Aaron et al. Jun 2010 B2
7752077 Holden et al. Jul 2010 B2
7761548 Snyder et al. Jul 2010 B2
7769654 Hurewitz Aug 2010 B1
7774458 Trinon et al. Aug 2010 B2
7783759 Eilam Aug 2010 B2
7801755 Doherty et al. Sep 2010 B2
7805400 Teh et al. Sep 2010 B2
7813948 Ratzloff Oct 2010 B2
7852711 Fitzgerald et al. Dec 2010 B1
7870051 En et al. Jan 2011 B1
7877742 Duale et al. Jan 2011 B2
7899235 Williams et al. Mar 2011 B1
7917555 Gottumukkala et al. Mar 2011 B2
7930396 Trinon et al. Apr 2011 B2
7933861 Zadorozhny Apr 2011 B2
7945489 Weiss et al. May 2011 B2
7966235 Capelli et al. Jun 2011 B1
7966266 Delvat Jun 2011 B2
8010584 Craver et al. Aug 2011 B1
8024241 Bailey Sep 2011 B2
8073724 Harthcryde et al. Dec 2011 B2
8121959 Delvat Feb 2012 B2
8175863 Ostermeyer May 2012 B1
8195524 Sandholm et al. Jun 2012 B2
8195785 Snyder et al. Jun 2012 B2
8200518 Bailey et al. Jun 2012 B2
8200561 Scott et al. Jun 2012 B1
8209218 Basu et al. Jun 2012 B1
8214829 Neogi et al. Jul 2012 B2
8260959 Rudkin Sep 2012 B2
8370243 Cernyar Feb 2013 B1
8396775 Mindlin Mar 2013 B1
8423428 Grendel et al. Apr 2013 B2
8484355 Lochhead Jul 2013 B1
8533904 Conrad Sep 2013 B2
8543438 Fleiss Sep 2013 B1
8600830 Hoffberg Dec 2013 B2
8601263 Shankar et al. Dec 2013 B1
8606827 Williamson Dec 2013 B2
8655714 Weir et al. Feb 2014 B2
8667385 Mui et al. Mar 2014 B1
8766981 McLachlan et al. Jul 2014 B2
8768976 McLachlan et al. Jul 2014 B2
8826230 Michelsen Sep 2014 B1
8935301 Chmiel et al. Jan 2015 B2
8937618 Erez et al. Jan 2015 B2
8970476 Chan Mar 2015 B2
8993552 Munkes et al. Mar 2015 B2
9015692 Clavel Apr 2015 B1
9020830 Purpus et al. Apr 2015 B2
9104661 Evans Aug 2015 B1
9213573 French et al. Dec 2015 B2
9268964 Schepis et al. Feb 2016 B1
9281012 Hedges Mar 2016 B2
9384511 Purpus Jul 2016 B1
9529863 Gindin et al. Dec 2016 B1
9805311 Mohler Oct 2017 B1
10152722 Heath Dec 2018 B2
20020002557 Straube et al. Jan 2002 A1
20020016752 Suh Feb 2002 A1
20020056004 Smith May 2002 A1
20020069102 Vellante et al. Jun 2002 A1
20020082966 O'Brien et al. Jun 2002 A1
20020087441 Wagner, Jr. et al. Jul 2002 A1
20020107914 Charisius et al. Aug 2002 A1
20020123945 Booth et al. Sep 2002 A1
20020129342 Kil Sep 2002 A1
20020145040 Grabski Oct 2002 A1
20020154173 Etgen Oct 2002 A1
20020156710 Ryder Oct 2002 A1
20020174006 Rugge et al. Nov 2002 A1
20020174049 Kitahara Nov 2002 A1
20020178198 Steele Nov 2002 A1
20020194329 Alling Dec 2002 A1
20030019350 Khosla Jan 2003 A1
20030033191 Davies et al. Mar 2003 A1
20030074269 Viswanath Apr 2003 A1
20030083388 L'Alloret May 2003 A1
20030083888 Argenton et al. May 2003 A1
20030083912 Covington et al. May 2003 A1
20030093310 Macrae May 2003 A1
20030110113 Martin Jun 2003 A1
20030139960 Nishikawa et al. Jul 2003 A1
20030139986 Roberts, Jr. Jul 2003 A1
20030158724 Uchida Aug 2003 A1
20030158766 Mital et al. Aug 2003 A1
20030172018 Chen et al. Sep 2003 A1
20030172368 Alumbaugh et al. Sep 2003 A1
20030195780 Arora et al. Oct 2003 A1
20030208493 Hall et al. Nov 2003 A1
20030217033 Sandler et al. Nov 2003 A1
20030233301 Chen et al. Dec 2003 A1
20030236721 Plumer et al. Dec 2003 A1
20040030628 Takamoto et al. Feb 2004 A1
20040039685 Hambrecht et al. Feb 2004 A1
20040059611 Kananghinis et al. Mar 2004 A1
20040059679 Mizumachi et al. Mar 2004 A1
20040073477 Heyns et al. Apr 2004 A1
20040093344 Berger et al. May 2004 A1
20040111509 Eilam Jun 2004 A1
20040133876 Sproule Jul 2004 A1
20040138942 Pearson et al. Jul 2004 A1
20040186762 Beaven et al. Sep 2004 A1
20040243438 Mintz Dec 2004 A1
20040249737 Tofte Dec 2004 A1
20050004856 Brose et al. Jan 2005 A1
20050033631 Wefers et al. Feb 2005 A1
20050037326 Kuntz et al. Feb 2005 A1
20050038788 Dettinger et al. Feb 2005 A1
20050044224 Jun et al. Feb 2005 A1
20050060298 Agapi et al. Mar 2005 A1
20050060317 Lott et al. Mar 2005 A1
20050071285 Laicher et al. Mar 2005 A1
20050091102 Retsina Apr 2005 A1
20050120032 Liebich et al. Jun 2005 A1
20050131870 Krishnaswarny et al. Jun 2005 A1
20050131929 Bailey Jun 2005 A1
20050144110 Chen et al. Jun 2005 A1
20050171918 Eden et al. Aug 2005 A1
20050235020 Gabelmann et al. Oct 2005 A1
20050246482 Gabelmann et al. Nov 2005 A1
20060010156 Netz et al. Jan 2006 A1
20060010294 Pasumansky et al. Jan 2006 A1
20060041458 Ringrose et al. Feb 2006 A1
20060041501 Tabata et al. Feb 2006 A1
20060059032 Wong et al. Mar 2006 A1
20060074980 Sarkar Apr 2006 A1
20060080264 Zhang et al. Apr 2006 A1
20060085302 Weiss et al. Apr 2006 A1
20060085465 Nori et al. Apr 2006 A1
20060106658 Johanson et al. May 2006 A1
20060116859 Legault et al. Jun 2006 A1
20060116975 Gould et al. Jun 2006 A1
20060126552 Lee et al. Jun 2006 A1
20060136281 Peters et al. Jun 2006 A1
20060143219 Smith et al. Jun 2006 A1
20060161879 Lubrecht et al. Jul 2006 A1
20060167703 Yakov Jul 2006 A1
20060178960 Lepman Aug 2006 A1
20060179012 Jacobs Aug 2006 A1
20060190497 Inturi et al. Aug 2006 A1
20060200400 Hunter et al. Sep 2006 A1
20060200477 Barrenechea Sep 2006 A1
20060212146 Johnson et al. Sep 2006 A1
20060212334 Jackson Sep 2006 A1
20060224740 Sievers-Tostes Oct 2006 A1
20060224946 Barrett et al. Oct 2006 A1
20060228654 Sanjar et al. Oct 2006 A1
20060235785 Chait et al. Oct 2006 A1
20060259468 Brooks et al. Nov 2006 A1
20060277074 Einav et al. Dec 2006 A1
20060282429 Hernandez-Sherrington et al. Dec 2006 A1
20070038494 Kreitzberg et al. Feb 2007 A1
20070088641 Aaron et al. Apr 2007 A1
20070113289 Blumenau May 2007 A1
20070118516 Li et al. May 2007 A1
20070124162 Mekyska May 2007 A1
20070129892 Smartt et al. Jun 2007 A1
20070185785 Carlson et al. Aug 2007 A1
20070198390 Lazear et al. Aug 2007 A1
20070198982 Bolan et al. Aug 2007 A1
20070179975 Teh et al. Sep 2007 A1
20070198317 Harthcryde et al. Sep 2007 A1
20070214413 Boeckenhauer Sep 2007 A1
20070226064 Yu et al. Sep 2007 A1
20070226090 Stratton Sep 2007 A1
20070233439 Carroll et al. Oct 2007 A1
20070260532 Blake, III Nov 2007 A1
20070265896 Smith Nov 2007 A1
20070271203 Delvat Nov 2007 A1
20070276755 Rapp Nov 2007 A1
20070282626 Zhang et al. Dec 2007 A1
20080027957 Bruckner et al. Jan 2008 A1
20080033774 Kimbrel Feb 2008 A1
20080059945 Sauer et al. Mar 2008 A1
20080060058 Shea et al. Mar 2008 A1
20080065435 Ratzloff Mar 2008 A1
20080071844 Gopal Mar 2008 A1
20080082186 Hood et al. Apr 2008 A1
20080082435 O'Brien et al. Apr 2008 A1
20080120122 Olenski et al. May 2008 A1
20080201269 Hollins et al. Aug 2008 A1
20080201297 Choi et al. Aug 2008 A1
20080208647 Hawley et al. Aug 2008 A1
20080208667 Lymbery et al. Sep 2008 A1
20080221949 Delurgio et al. Sep 2008 A1
20080222638 Beaty et al. Sep 2008 A1
20080239393 Navon Oct 2008 A1
20080255912 Christiansen et al. Oct 2008 A1
20080295096 Beaty Nov 2008 A1
20080312979 Lee et al. Dec 2008 A1
20080319811 Casey Dec 2008 A1
20090012986 Arazi et al. Jan 2009 A1
20090013325 Kobayashi et al. Jan 2009 A1
20090018880 Bailey et al. Jan 2009 A1
20090018996 Hunt et al. Jan 2009 A1
20090063251 Rangarajan et al. Mar 2009 A1
20090063540 Mattox et al. Mar 2009 A1
20090100017 Graves et al. Apr 2009 A1
20090100406 Greenfield et al. Apr 2009 A1
20090144120 Ramachandran Jun 2009 A1
20090150396 Elisha et al. Jun 2009 A1
20090195350 Tsern et al. Aug 2009 A1
20090198535 Brown et al. Aug 2009 A1
20090199192 Laithwaite et al. Aug 2009 A1
20090210275 Andreev et al. Aug 2009 A1
20090216580 Bailey et al. Aug 2009 A1
20090222742 Pelton et al. Sep 2009 A1
20090234892 Anglin et al. Sep 2009 A1
20090300173 Bakman Dec 2009 A1
20090307597 Bakman Dec 2009 A1
20090319316 Westerfeld et al. Dec 2009 A1
20100005014 Castle et al. Jan 2010 A1
20100005173 Baskaran Jan 2010 A1
20100017344 Hambrecht et al. Jan 2010 A1
20100042455 Liu et al. Feb 2010 A1
20100049494 Radibratovic et al. Feb 2010 A1
20100082380 Merrifield, Jr. et al. Apr 2010 A1
20100094740 Richter Apr 2010 A1
20100125473 Tung May 2010 A1
20100153282 Graham Jun 2010 A1
20100161371 Cantor et al. Jun 2010 A1
20100161634 Caceres Jun 2010 A1
20100169477 Stienhans Jul 2010 A1
20100185557 Hunter et al. Jul 2010 A1
20100198750 Ron et al. Aug 2010 A1
20100211667 O'Connell, Jr. Aug 2010 A1
20100250419 Ariff et al. Sep 2010 A1
20100250421 Ariff et al. Sep 2010 A1
20100250642 Yellin Sep 2010 A1
20100293163 McLachlan et al. Nov 2010 A1
20100299233 Licardi et al. Nov 2010 A1
20100306382 Cardosa Dec 2010 A1
20100323754 Nakagawa Dec 2010 A1
20100325506 Cai et al. Dec 2010 A1
20100325606 Sundararajan et al. Dec 2010 A1
20100332262 Horvitz Dec 2010 A1
20100333109 Milnor Dec 2010 A1
20110016214 Jackson Jan 2011 A1
20110016448 Bauder et al. Jan 2011 A1
20110022861 Agneeswaran Jan 2011 A1
20110066472 Scheider Mar 2011 A1
20110066628 Jayaraman Mar 2011 A1
20110072340 Miller Mar 2011 A1
20110106691 Clark et al. May 2011 A1
20110107254 Moroze May 2011 A1
20110167034 Knight et al. Jul 2011 A1
20110196795 Pointer Aug 2011 A1
20110225277 Freimuth Sep 2011 A1
20110238608 Sathish Sep 2011 A1
20110261049 Cardno et al. Oct 2011 A1
20110295766 Tompkins Dec 2011 A1
20110313947 Grohavaz Dec 2011 A1
20120016811 Jones Jan 2012 A1
20120023170 Matignon et al. Jan 2012 A1
20120066020 Moon Mar 2012 A1
20120116990 Huang May 2012 A1
20120131591 Moorthi May 2012 A1
20120150736 Dickerson et al. Jun 2012 A1
20120232947 McLachlan Sep 2012 A1
20120233217 Purpus et al. Sep 2012 A1
20120233547 McLachlan Sep 2012 A1
20120239739 Manglik et al. Sep 2012 A1
20120246046 Hirsch Sep 2012 A1
20120272234 Kaiser et al. Oct 2012 A1
20120330869 Durham Dec 2012 A1
20130014057 Reinpoldt Jan 2013 A1
20130028537 Miyake et al. Jan 2013 A1
20130041792 King et al. Feb 2013 A1
20130041819 Khasho Feb 2013 A1
20130060595 Bailey Mar 2013 A1
20130066866 Chan et al. Mar 2013 A1
20130091456 Sherman et al. Apr 2013 A1
20130091465 Kikin-Gil et al. Apr 2013 A1
20130103369 Huynh et al. Apr 2013 A1
20130103654 McLachlan et al. Apr 2013 A1
20130124454 Bhide et al. May 2013 A1
20130124459 Iwase et al. May 2013 A1
20130138470 Goyal et al. May 2013 A1
20130159926 Vainer et al. Jun 2013 A1
20130173159 Trum et al. Jul 2013 A1
20130179371 Jain Jul 2013 A1
20130201193 McLachlan Aug 2013 A1
20130227584 Greene et al. Aug 2013 A1
20130268307 Li et al. Oct 2013 A1
20130282537 Snider Oct 2013 A1
20130290470 CaraDonna et al. Oct 2013 A1
20130293551 Erez et al. Nov 2013 A1
20130339274 Willis et al. Dec 2013 A1
20130346390 Jerzak Dec 2013 A1
20140006222 Hericks et al. Jan 2014 A1
20140067632 Curtis Mar 2014 A1
20140075004 Van Dusen et al. Mar 2014 A1
20140089509 Akolkar et al. Mar 2014 A1
20140108295 Renshaw Apr 2014 A1
20140122374 Hacigumus et al. May 2014 A1
20140129583 Munkes et al. May 2014 A1
20140136295 Wasser May 2014 A1
20140143175 Greenshields et al. May 2014 A1
20140172918 Kornmann et al. Jun 2014 A1
20140229212 MacElheron et al. Aug 2014 A1
20140244364 Evers Aug 2014 A1
20140252095 Kikin Sep 2014 A1
20140257928 Chen et al. Sep 2014 A1
20140278459 Morris Sep 2014 A1
20140279121 George et al. Sep 2014 A1
20140279201 Iyoob et al. Sep 2014 A1
20140279676 Schafer et al. Sep 2014 A1
20140279947 Chachra et al. Sep 2014 A1
20140288987 Liu Sep 2014 A1
20140310233 Catalano et al. Oct 2014 A1
20140006085 McLachlan et al. Nov 2014 A1
20140337007 Waibel et al. Nov 2014 A1
20140351166 Schlossberg Nov 2014 A1
20140365503 Franceschini et al. Dec 2014 A1
20140365504 Franceschini et al. Dec 2014 A1
20150006552 Lord Jan 2015 A1
20150012328 McLachlan et al. Jan 2015 A1
20150046363 McNamara et al. Feb 2015 A1
20150066808 Legare et al. Mar 2015 A1
20150074075 Alexander Mar 2015 A1
20150088584 Santiago, III et al. Mar 2015 A1
20150120370 Agrawal et al. Apr 2015 A1
20150149257 Bielat et al. May 2015 A1
20150227991 Yu Aug 2015 A1
20150278024 Barman et al. Oct 2015 A1
20150294273 Barraci et al. Oct 2015 A1
20150302303 Hakim Oct 2015 A1
20150341230 Dave et al. Nov 2015 A1
20150363725 Andersson et al. Dec 2015 A1
20150379061 Paraschivescu Dec 2015 A1
20160063577 Yellin et al. Mar 2016 A1
20160098234 Weaver et al. Apr 2016 A1
20160266594 Kauffman et al. Sep 2016 A1
20170102246 Yang Apr 2017 A1
20170154088 Sherman Jun 2017 A1
20180068246 Crivat et al. Mar 2018 A1
Foreign Referenced Citations (1)
Number Date Country
2011134268 Jul 2011 JP
Non-Patent Literature Citations (299)
Entry
US 5,649,211 A, 04/1997, Horkin et al. (withdrawn)
Vizard Michael, Free Service from Apptio Tracks Cloud Service Provider Pricing, IT business edge, Dec. 12, 2012 http://www.itbusinessedge.com/blogs/it-unmasked/free-service-from-apptio-tracks-cloud-service-provider-pricing.html.
Talbot Chris, Apptio Cloud Express Provides Free Usage Tracking Service, talkincloud, Dec. 12, 2012 http://talkincloud.com/cloud-computing-management/apptio-cloud-express-provides-free-usage-tracking-service.
Morgan Timothy, Apptio puffs up freebie cost control freak for public clouds, The Register, Dec. 12, 2012 http://www.theregister.co.uk/2012/12/12/apptio_cloud_express/.
Riknas Mikael, Apptio unveils tool to keep track of cloud costs, Computerworld Dec. 12, 2012 http://www.computerworld.com/s/article/9234630/Apptio_unveils_tool_to_keep_track_of_cloud_costs.
Accenture Sustainability Cloud Computing the Environmental Benefits of Moving to the Cloud, archives org, Aug. 13, 2011 http://web.archive.org/web/20110813022626/http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture_Sustainability_Cloud_Computing_TheEnvironmentalBenefitsofMovingtotheCloud.pdf.
Amazon Elastic Compute Cloud, Amazon EC2, archives org, Oct. 21, 2011 http://web.archive.org/web/20111029130914/http://aws.amazon.com/ec2/#pricing.
Automated Cost Transparency, Apptio 2008 http://www.cio.com/documents/whitepapers/AutomatedCostTransparency.pdf.
Apptio Optimizes Enterprise IT Costs Utilizing Amazon Web Services Cloud Computing , Apprio, Apr. 7, 2009 http://www.apptio.com/news/apptio-optimizes-enterprise-it-costs-utilizing-amazon-web-services-cloud-computing#.Ukm5XsX7Lco.
Apptio Extends Leadership in Cloud Business Management with Launch of Apptio Cloud Express, Apptio, Dec. 12, 2012 http://www.apptio.com/news/apptio-extends-leadership-cloud-business-management-launch-apptio-cloud-express#.Ukm4r8X7Lco.
Visualization for Production Management Treemap and Fisheye Table Browser open-video organization webpages 2001 http://www.open-video.org/details.php?videoid=4547.
Robinson Glen, Cloud Economics—Cost Optimization (selected slides), Amazon Web Services AWS, Slideshare webpages Feb. 28, 2012 http://www.slideshare.net/AmazonWebServices/whats-new-with-aws-london.
Amazon Reserved Instances, Amazon Web Services , archives org, Jan. 14, 2012 http://web.archive.org/web/20120114153849/http://aws.amazon.com/rds/reserved-instances/?.
Cost Optimisation with Amazon Web Services, extracted slides, Slideshare Jan. 30, 2012 http://www.slideshare.net/AmazonWebServices/cost-optimisation-with-amazon-web-services?from_search=1.
Deciding an Approach to the Cloud AWS Reserved Instances, Cloudyn webpages, Feb. 28, 2012 https://www.cloudyn.com/blog/deciding-an-approach-to-the-cloud-aws-reserved-instances/.
Ganesan Harish, Auto Scaling using AWS, Amazon Web Services AWS (selected slides), Apr. 20, 2011 http://www.slideshare.net/harishganesan/auto-scaling-using-amazon-web-services-aws.
Robinson Glen, Cloud Economics—Cost Optimization (selected slides), Amazon Web Services AWS, Slideshare, Feb. 28, 2012 http://www.slideshare.net/AmazonWebServices/whats-new-with-aws-london.
Skilton et al, Building Return on Investment from Cloud Computing, The Open Group Whitepaper, mladina webpages, Apr. 2010 http://www.mladina.si/media/objave/dokumenti/2010/5/31/31_5_2010_open_group_building_return_on_investment_from_cloud_computing.pdf.
Ward Miles, Optimizing for Cost in the Cloud (selection), AWS Summit, Slideshare Apr. 20, 2012 http://www.slideshare.net/AmazonWebServices/optimizing-your-infrastructure-costs-on-aws.
Efficient frontier—Wikipedia, Efficient frontier, Wikipedia webpages, Oct. 30, 2016 https://en.wikipedia.org/wiki/Efficient_frontier.
“Notice from the European Patent Office dated Oct. 1, 2007 concerning business methods,” Official Journal EPO, Nov. 2007, pp. 592-593.
“Program Evaluation and Review Technique,” Wikipedia, the free encyclopedia, accessed Mar. 13, 2012, 10 pages http://en.wikipedia.org/wiki/Program_Evaluation_and_Review_Technique—last modified Mar. 12, 2012.
“Project Management,” Wikipedia, the free encyclopedia, accessed Mar. 13, 2012, 14 pages http://en.wikipedia.org/wiki/Project_management—last modified Mar. 7, 2012.
Extended European Search Report in EP Application No. 13151967.0-1955, dated Apr. 19, 2013.
International Search Report and Written Opinion for International Patent Application No. PCT/US2010/035021 dated Jul. 14, 2010.
International Preliminary Report on Patentability for International Patent Application No. PCT/US2010/035021 dated Nov. 24, 2011.
International Search Report and Written Opinion for International Patent Application No. PCT/US2012/028353 dated Oct. 31, 2012.
International Search Report and Written Opinion for International Patent Application No. PCT/US2012/028378 dated Sep. 12, 2012.
Official Communication for U.S. Appl. No. 12/467,120 dated Oct. 4, 2011.
Official Communication for U.S. Appl. No. 12/467,120 dated Jun. 20, 2012.
Official Communication for U.S. Appl. No. 12/467,120 dated Aug. 29, 2012.
Official Communication for U.S. Appl. No. 12/467,120 dated Mar. 26, 2013.
Official Communication for U.S. Appl. No. 13/324,253 dated Sep. 25, 2012.
Official Communication for U.S. Appl. No. 13/324,253 dated Jan. 10, 2013.
Official Communication for U.S. Appl. No. 13/324,253 dated Mar. 19, 2013.
Official Communication for U.S. Appl. No. 13/452,628 dated Apr. 22, 2013.
“Activity Based Costing is the best allocation methodology,” APPTIO, Community for Technology Business Management, Mar. 16, 2010, 2 pages.
“IT Cost Transparency and Apptio,” Dec. 4, 2008, 2 pages http://web.archive.org/web/20081204012158/http://www.apptio.com/solutions.
Busch, J., “Six Strategies for IT Cost Allocation,” Spend Matters, Jan. 5, 2011, 3 pages http://spendmatters.com/2011/01/05/six-strategies-for-it-cost-allocation/.
International Preliminary Report on Patentability for International Patent Application No. PCT/US2012/028353 dated Sep. 19, 2013.
International Preliminary Report on Patentability for International Patent Application No. PCT/US2012/028378 dated Sep. 19, 2013.
Official Communication for U.S. Appl. No. 12/467,120 dated Oct. 23, 2013.
Official Communication for U.S. Appl. No. 13/324,253 dated Sep. 6, 2013.
Official Communication for U.S. Appl. No. 13/415,797 dated Oct. 3, 2013.
Official Communication for U.S. Appl. No. 13/452,628 dated Nov. 18, 2013.
Official Communication for U.S. Appl. No. 13/837,815 dated Oct. 23, 2013.
Official Communication for U.S. Appl. No. 13/917,478 dated Nov. 7, 2013.
Official Communication for U.S. Appl. No. 13/935,147 dated Oct. 22, 2013.
Official Communication for U.S. Appl. No. 13/675,837 dated Oct. 10, 2013.
Official Communication for U.S. Appl. No. 13/452,628 dated Mar. 13, 2014.
Office Communication for U.S. Appl. No. 13/452,628 dated Mar. 30, 2015 (18 pages).
European Search Report for Application No. 12755613.2 dated Jan. 26. 2015 (6 pages).
Office Communication for U.S. Appl. No. 14/180,308 dated Jan. 30, 2015.
Office Communication for U.S. Appl. No. 13/837,815 dated Sep. 25, 2014.
Official Communication for U.S. Appl. No. 13/324,253 dated Jan. 23, 2014.
Official Communication for U.S. Appl. No. 13/675,837 dated Jan. 31, 2014.
Extended European Search Report in EP Application No. 14159413.5 dated Jul. 4, 2014.
International Search Report and Written Opinion for Application PCT/US2012/028353 dated Mar. 8, 2012.
Official Communication for U.S. Appl. No. 14/180,308 dated Apr. 8, 2014.
Official Communication for U.S. Appl. No. 14/180,308 dated Sep. 2, 2014.
Office Communication for U.S. Appl. No. 14/033,130 dated Aug. 5, 2014.
Office Communication for U.S. Appl. No. 14/033,130 dated May 27, 2014.
Office Communication for U.S. Appl. No. 13/935,147 dated Jun. 16, 2014.
Office Communication for U.S. Appl. No. 13/935,147 dated Apr. 11, 2014.
Office Communication for U.S. Appl. No. 13/675,837 dated Jan. 31, 2014.
SAS Activity-Based Management, 2010, Fact Sheet, 4 pages.
Office Communication for U.S. Appl. No. 13/415,797 dated Apr. 9, 2014.
Office Communication for U.S. Appl. No. 13/324.253 dated Apr. 9, 2014.
Office Communication for U.S. Appl. No. 13/324,253 dated Oct. 24, 2014.
Office Communication for U.S. Appl. No. 13/365,150 dated Dec. 3, 2014.
Office Communication for U.S. Appl. No. 13/452,628 dated Oct. 1, 2014.
Office Communication for U.S. Appl. No. 13/415,797 dated Jan. 12, 2015.
Office Communication for U.S. Appl. No. 13/837,815 dated Apr. 7, 2014.
Office Communication for U.S. Appl. No. 13/675,837 dated Apr. 2, 2014.
Office Communication for U.S. Appl. No. 13/949,019 dated Feb. 10, 2015.
Office Communication for U.S. Appl. No. 13/324,253 dated Feb. 19, 2015.
Henriet et al. “Traffic-Based Cost Allocation in a Network.” The Rand Journal of Economics, 1996, pp. 332-345.
Rudnick et al., “Marginal Pricing and Supplement Cost Allocation in Transmission Open Access.” Power Systems, IEEE Transactions on 10.2, 1995, pp. 1125-1132.
Official Communication for U.S. Appl. No. 14/033,130 dated Dec. 16, 2013.
Office Communication for U.S. Appl. No. 14/180,308 dated Apr. 17, 2015.
Office Communication for U.S. Appl. No. 13/675,837 dated Apr. 16, 2015.
Office Communication for U.S. Appl. No. 13/837,815 dated Apr. 27, 2015.
Office Communication for U.S. Appl. No. 13/452,628 dated Jun. 23, 2015.
Office Communication for U.S. Appl. No. 13/649,019 dated Sep. 23, 2015.
Office Communication for U.S. Appl. No. 13/365,150 dated Sep. 24, 2015.
Office Communication for U.S. Appl. No. 14/033,130 dated Sep. 15, 2015.
Office Communication for U.S. Appl. No. 13/415,797 dated Jul. 23, 2015.
International Search Report and Written Opinion for PCT/US2015/015486 dated Jun. 29, 2015.
Office Communication for U.S. Appl. No. 13/935,147 dated Jul. 9, 2015.
European Examination Report for Application No. 14159413.5 dated Jul. 15, 2015, 9 pages.
Office Communication for U.S. Appl. No. 13/415,701 dated Oct. 27, 2015, 16 pages.
David B. Stewart et al., “Rapid Development of Robotic Applications Using Component-Based Real-Time Software,” Intelligent Robots and Systems 1995, Human Robot Interaction and Cooperative Robots Proceedings, 1995, IEEE International Conference on vol. 1, pp. 465-470 (6 pages).
Official Communication for U.S. Appl. No. 14/846,349 dated Jul. 1, 2016, 24 pages.
Official Communication for U.S. Appl. No. 14/981,747 dated Jul. 14, 2016, 29 pages.
Office Communication for U.S. Appl. No. 14/722,663 dated Dec. 1, 2015, 37 pages.
Chien-Liana Fok et al., “Rapid Development and Flexible Deployment of Adaptive Wireless Sensor Network Applications,” Proceedings of the 25th IEEE International Conference on Distributed Computing Systems, 2005, pp. 653-662 (10 pages).
Frans Flippo et al., “A Framework for Rapid Development of Multimodal Interfaces,” Proceedings of the 5th International Conference on Multimodal Interfaces, 2003, pp. 109-116 (8 pages).
Official Communication for U.S. Appl. No. 14/846,349 dated Dec. 17, 2015, 23 pages.
Official Communication for U.S. Appl. No. 13/649,019 dated Jan. 4, 2016, 8 pages.
Official Communication for U.S. Appl. No. 13/452,628 dated Jan. 12, 2016, 21 pages.
Official Communication for U.S. Appl. No. 14/869,721 dated Jan. 13, 2016, 57 pages.
Official Communication for U.S. Appl. No. 14/033,130 dated Feb. 18, 2016, 22 pages.
Official Communication for U.S. Appl. No. 13/935,147 dated Mar. 9, 2016, 10 pages.
International Search Report and Written Opinion for PCT/US2015/048697 dated Mar. 31, 2016, 9 pages.
Office Communication for U.S. Appl. No. 13/365,150, dated Apr. 6, 2016, 11 pages.
Office Communication for U.S. Appl. No. 14/722,663, dated Mar. 31, 2016, 7 pages.
Van Diessen et al., “Component Business Model for Digital Repositories: A Framework for Analysis,” AAAI, 2008, 7 pages.
Melcher et al., “Visualization and Clustering of Business Process Collections Based on Process Metric Values,” IEEE Computer Society, 2008, 4 pages.
Lee et al., “Value-Centric, Model-Driven Business Transformation,” IEEE, 2008, 8 pages.
Lee et al., “Business Transformation Workbench: A Practitioner's Tool for Business Transformation,” IEEE International Conference on Services Computing, 2008, 8 pages.
Risch et al., “Interactive Information Visualization for Exploratory Intelligence Data Analysis,” IEEE Proceedings of VRAIS, 1996, 10 pages.
Office Communication for U.S. Appl. No. 13/415,797, dated Apr. 4, 2016, 24 pages.
Office Communication for U.S. Appl. No. 13/837,815, dated Apr. 13, 2016, 22 pages.
Office Communication for U.S. Appl. No. 14/867,552, dated Apr. 25, 2016, 12 pages.
Office Communication for U.S. Appl. No. 14/033,130, dated Apr. 25, 2016, 4 pages.
Office Communication for U.S. Appl. No. 14/971,944, dated May 19, 2016, 17 pages.
Stephen Muller and Hasso Platner, “An IN-Depth Analysis of Data Aggregation Cost Factors in a Columnar In-Memory Database”, ACM DOLAP'12, Nov. 2, 2012, Maui, Hawaii, USA, pp. 65-72.
Official Communication for U.S. Appl. No. 14/869,721 dated Jun. 1, 2016, 35 pages.
Official Communication for U.S. Appl. No. 14/977,368 dated Jun. 7, 2016, 11 pages.
Official Communication for U.S. Appl. No. 13/837,815 dated Jun. 23, 2016, 3 pages.
Office Communication for U.S. Appl. No. 13/675,837 dated Oct. 26, 2015, 20 pages.
Official Communication for U.S. Appl. No. 13/637,815 dated Nov. 9, 2016, 11 pages.
Official Communication for U.S. Appl. No. 15/260,221 dated Dec. 20, 2016, 21 pages.
Official Communication for U.S. Appl. No. 15/271,013 dated Dec. 15, 2016, 50 pages.
Office Communication for U.S. Appl. No. 13/415,797 dated Oct. 19, 2015.
Office Communication for U.S. Appl. No. 13/837,815 dated Sep. 28, 2015.
Official Communication for U.S. Appl. No. 14/869,721 dated Aug. 3, 2016, 5 pages.
Official Communication for U.S. Appl. No. 13/452,628 dated Aug. 18, 2016, 22 pages.
Official Communication for U.S. Appl. No. 14/867,552 dated Oct. 3, 2016, 19 pages.
Official Communication for U.S. Appl. No. 14/180,308 dated Oct. 19, 2016, 22 pages.
Official Communication for U.S. Appl. No. 14/977,368 dated Oct. 19, 2016, 5 pages.
Official Communication for U.S. Appl. No. 13/365,150 dated Oct. 24, 2016, 19 pages.
Official Communication for U.S. Appl. No. 13/415,797 dated Jan. 11, 2017, 25 pages.
Official Communication for U.S. Appl. No. 13/675,837 dated Jan. 11, 2017, 29 pages.
Official Communication for U.S. Appl. No. 15/351,313 dated Jan. 12, 2017, 7 pages.
Official Communication for U.S. Appl. No. 14/867,552 dated Jan. 9, 2017, 3 pages.
Official Communication for U.S. Appl. No. 14/033,130 dated Jan. 11, 2017, 12 pages.
Official Communication for U.S. Appl. No. 14/180,308 dated Feb. 8, 2017, 3 pages.
Official Communication for U.S. Appl. No. 14/846,349 dated Mar. 1, 2017, 27 pages.
Official Communication for U.S. Appl. No. 13/935,147 dated Mar. 7, 2017, 12 pages.
Official Communication for U.S. Appl. No. 13/365,150 dated Mar. 15, 2017, 19 pages.
Official Communication for U.S. Appl. No. 13/452,628 dated Mar. 9, 2017, 24 pages.
Official Communication for U.S. Appl. No. 15/379,267 dated Mar. 10, 2017, 11 pages.
Official Communication for U.S. Appl. No. 13/415,797 dated Apr. 14, 2017, 3 pages.
Official Communication for U.S. Appl. No. 13/365,150 dated May 22, 2017, 3 pages.
Official Communication for U.S. Appl. No. 13/365,150 dated Aug. 23, 2017, 30 pages.
Official Communication for U.S. Appl. No. 14/869,721 dated May 5, 2017, 49 pages.
Official Communication for U.S. Appl. No. 14/981,747 dated May 19, 2017, 43 pages.
Official Communication for U.S. Appl. No. 15/271,013 dated May 24, 2017, 37 pages.
Official Communication for U.S. Appl. No. 14/180,308 dated May 25, 2017, 21 pages.
Official Communication for U.S. Appl. No. 15/379,267 dated Jun. 30, 2017, 16 pages.
Official Communication for U.S. Appl. No. 14/867,552 dated Jun. 29, 2017, 31 pages.
Official Communication for U.S. Appl. No. 14/033,130 dated Jun. 29, 2017, 18 pages.
Official Communication for U.S. Appl. No. 13/837,815 dated Jun. 12, 2017, 12 pages.
Official Communication for U.S. Appl. No. 15/351,313 dated Jul. 18, 2017, 15 pages.
Official Communication for U.S. Appl. No. 15/260,221 dated Aug. 15, 2017, 21 pages.
Official Communication for European Application No. 13151967.0 dated Aug. 18, 2017, 7 pages.
European Search Report for European Application No. 10775648.8 dated Mar. 10, 2017, 6 pages.
Official Communication for European Application No. 12755613.2 dated Aug. 17, 2017, 7 pages.
Official Communication for U.S. Appl. No. 14/033,130 dated Sep. 7, 2017, 3 pages.
Official Communication for U.S. Appl. No. 14/846,349 dated Sep. 8, 2017, 25 pages.
Official Communication for U.S. Appl. No. 13/452,628 dated Sep. 28, 2017, 26 pages.
Official Communication for U.S. Appl. No. 13/837,815 dated Sep. 28, 2017, 9 pages.
Official Communication for U.S. Appl. No. 13/415,797 dated Sep. 7, 2017, 26 pages.
Official Communication for U.S. Appl. No. 14/869,721 dated Oct. 17, 2017, 30 pages.
Official Communication for U.S. Appl. No. 15/379,267 dated Oct. 6, 2017, 3 pages.
Official Communication for U.S. Appl. No. 13/935,147 dated Nov. 3, 2017, 11 pages.
Official Communication for U.S. Appl. No. 14/846,349 dated Nov. 20, 2017, 3 pages.
Official Communication for U.S. Appl. No. 13/365,150 dated May 22, 2017.
Official Communication for U.S. Appl. No. 13/365,150 dated Aug. 23, 2017.
Official Communication for U.S. Appl. No. 14/869,721 dated May 5, 2017.
Official Communication for U.S. Appl. No. 14/981,747 dated May 19, 2017.
Official Communication for U.S. Appl. No. 15/271,013 dated May 24, 2017.
Official Communication for U.S. Appl. No. 14/180,308 dated May 25, 2017.
Official Communication for U.S. Appl. No. 15/379,267 dated Jun. 30, 2017.
Official Communication for U.S. Appl. No. 14/867,552 dated Jun. 29, 2017.
Official Communication for U.S. Appl. No. 13/837,815 dated Jun. 12, 2017.
Official Communication for U.S. Appl. No. 15/351,313 dated Jul. 18, 2017.
Official Communication for U.S. Appl. No. 15/260,221 dated Aug. 15, 2017.
Official Communication for European Application No. 13151967.0 dated Aug. 18, 2017.
European Search Report for European Application No. 10775648.8 dated Mar. 10, 2017.
Official Communication for European Application No. 12755613.2 dated Aug. 17, 2017.
Official Communication for U.S. Appl. No. 14/033,130 dated Sep. 7, 2017.
Official Communication for U.S. Appl. No. 14/846,349 dated Sep. 8, 2017.
Official Communication for U.S. Appl. No. 13/452,628 dated Sep. 28, 2017.
Official Communication for U.S. Appl. No. 13/837,815 dated Sep. 28, 2017.
Official Communication for U.S. Appl. No. 13/415,797 dated Sep. 7, 2017.
Official Communication for U.S. Appl. No. 14/869,721 dated Oct. 17, 2017.
Official Communication for U.S. Appl. No. 15/379,267 dated Oct. 6, 2017.
Official Communication for U.S. Appl. No. 13/935,147 dated Nov. 3, 2017.
Official Communication for U.S. Appl. No. 14/846,349 dated Nov. 20, 2017.
Official Communication for U.S. Appl. No. 14/180,308 dated Dec. 22, 2017.
Official Communication for U.S. Appl. No. 15/271,013 dated Dec. 27, 2017.
Official Communication for U.S. Appl. No. 15/260,221 dated Jan. 9, 2018.
Official Communication for U.S. Appl. No. 15/379,267 dated Jan. 2, 2018.
Official Communication for U.S. Appl. No. 15/351,313 dated Jan. 8, 2018.
Official Communication for U.S. Appl. No. 14/846,349 dated Jan. 18, 2018.
Official Communication for U.S. Appl. No. 14/180,308 dated Dec. 22, 2017, 18 pages.
Official Communication for U.S. Appl. No. 15/271,013 dated Dec. 27, 2017, 35 pages.
Official Communication for U.S. Appl. No. 15/260,221 dated Jan. 9, 2018, 21 pages.
Official Communication for U.S. Appl. No. 15/379,267 dated Jan. 2, 2018, 15 pages.
Official Communication for U.S. Appl. No. 15/351,313 dated Jan. 8, 2018, 11 pages.
Official Communication for U.S. Appl. No. 14/846,349 dated Jan. 18, 2018, 29 pages.
Official Communication for U.S. Appl. No. 13/837,815 dated Jan. 26, 2018, 12 pages.
Official Communication for U.S. Appl. No. 14/869,721 dated Jan. 19, 2018, 3 pages.
Official Communication for U.S. Appl. No. 14/667,552 dated Feb. 13, 2018, 3 pages.
Official Communication for U.S. Appl. No. 15/859,008 dated Mar. 5, 2018, 20 pages.
Official Communication for U.S. Appl. No. 13/935,147 dated Jan. 17, 2018, 3 pages.
Official Communication for U.S. Appl. No. 14/867,552 dated Nov. 29, 2017, 12 pages.
Official Communication for U.S. Appl. No. 14/981,747 dated Dec. 12, 2017, 44 pages.
Official Communication for U.S. Appl. No. 14/033,130 dated Dec. 20, 2017, 12 pages.
Official Communication for U.S. Appl. No. 14/846,349 dated Jul. 20, 2018, pp. 1-40.
Official Communication for U.S. Appl. No. 14/981,747 dated Jul. 5, 2018, pp. 1-62.
Official Communication for U.S. Appl. No. 15/271,013 dated Jul. 6, 2018, pp. 1-49.
Official Communication for U.S. Appl. No. 15/379,267 dated Jul. 19, 2018, pp. 1-34.
Official Communication for U.S. Appl. No. 13/935,147 dated Aug. 10, 2018, pp. 1-25.
Official Communication for U.S. Appl. No. 14/033,130 dated Aug. 9, 2018, pp. 1-47.
Official Communication for U.S. Appl. No. 14/180,308 dated Aug. 6, 2018, pp. 1-23.
Official Communication for U.S. Appl. No. 15/858,945 dated Sep. 10, 2018, pp. 1-25.
Official Communication for U.S. Appl. No. 13/837,815 dated Apr. 5, 2018, pp. 1-4.
Official Communication for U.S. Appl. No. 14/867,552 dated May 31, 2018, pp. 1-22.
Official Communication for U.S. Appl. No. 14/869,721 dated May 11, 2018, pp. 1-33.
Official Communication for U.S. Appl. No. 15/351,313 dated Jun. 4, 2018, pp. 1-9.
Official Communication for U.S. Appl. No. 15/858,945 dated Apr. 4, 2018, pp. 1-74.
Official Communication for U.S. Appl. No. 15/859,058 dated May 14, 2018, pp. 1-76.
Official Communication for U.S. Appl. No. 14/846,349 dated Jul. 20, 2018.
Official Communication for U.S. Appl. No. 14/981,747 dated Jul. 5, 2018.
Official Communication for U.S. Appl. No. 15/271,013 dated Jul. 6, 2018.
Official Communication for U.S. Appl. No. 15/379,267 dated Jul. 19, 2018.
Official Communication for U.S. Appl. No. 13/935,147 dated Aug. 10, 2018.
Official Communication for U.S. Appl. No. 14/033,130 dated Aug. 9, 2018.
Official Communication for U.S. Appl. No. 14/180,308 dated Aug. 6, 2018.
Official Communication for U.S. Appl. No. 15/858,945 dated Sep. 10, 2018.
Official Communication for U.S. Appl. No. 13/837,815 dated Apr. 5, 2018.
Official Communication for U.S. Appl. No. 14/867,552 dated May 31, 2018.
Official Communication for U.S. Appl. No. 14/869,721 dated May 11, 2018.
Official Communication for U.S. Appl. No. 15/351,313 dated Jun. 4, 2018.
Official Communication for U.S. Appl. No. 15/858,945 dated Apr. 4, 2018.
Official Communication for U.S. Appl. No. 15/859,058 dated May 14, 2018.
Official Communication for U.S. Appl. No. 14/869,721 dated Oct. 11, 2018, pp. 1-73.
Official Communication for U.S. Appl. No. 14/867,552 dated Nov. 21, 2018, pp. 1-37.
Official Communication for U.S. Appl. No. 15/260,221 dated Oct. 5, 2018, pp. 1-40.
Official Communication for U.S. Appl. No. 15/379,267 dated Oct. 18, 2018, pp. 1-9.
Official Communication for U.S. Appl. No. 15/859,058 dated Dec. 5, 2018, pp. 1-20.
Official Communication for U.S. Appl. No. 14/033,130 dated Dec. 18, 2018, pp. 1-11.
Official Communication for U.S. Appl. No. 15/271,013 dated Dec. 18, 2018, pp. 1-47.
Official Communication for U.S. Appl. No. 14/846,349 dated Oct. 18, 2019.
Official Communication for U.S. Appl. No. 14/981,747 dated Oct. 24, 2019.
Official Communication for U.S. Appl. No. 115/271,013 dated Nov. 21, 2019.
Official Communication for U.S. Appl. No. 15/859,008 dated Oct. 24, 2019.
Official Communication for U.S. Appl. No. 14/180,308 dated Dec. 10, 2019.
Official Communication for U.S. Appl. No. 14/646,349 dated Oct. 18, 2019, pp. 1-52.
Official Communication for U.S. Appl. No. 14/981,747 dated Oct. 24, 2019, pp. 1-62.
Official Communication for U.S. Appl. No. 15/271,013 dated Nov. 21, 2019, pp. 1-108.
Official Communication for U.S. Appl. No. 15/859,008 dated Oct. 24, 2019, pp. 1-22.
Daytime vs Night display on Garrnin GPS , POI Factory, Jun. 2008, http://www.poi-factory.com/node/14562 (Year: 2008), pp. 1-3.
Official Communication for U.S. Appl. No. 14/180,308 dated Dec. 10, 2019, pp. 1-29.
Official Communication for U.S. Appl. No. 14/981,747 dated Aug. 1, 2019.
Official Communication for U.S. Appl. No. 15/260,221 dated Sep. 3, 2019.
Official Communication for U.S. Appl. No. 15/351,313 dated Aug. 28, 2019.
Official Communication for U.S. Appl. No. 13/935,147 dated Mar. 28, 2019, pp. 1-80.
Official Communication for U.S. Appl. No. 14/033,130 dated Apr. 10, 2019, pp. 1-80.
Official Communication for U.S. Appl. No. 14/180,308 dated Feb. 26, 2019, pp. 1-26.
Official Communication for U.S. Appl. No. 14/846,349 dated Apr. 11, 2019, pp. 1-57.
Official Communication for U.S. Appl. No. 14/867,552 dated Feb. 11, 2019, pp. 1-76.
Official Communication for U.S. Appl. No. 14/981,747 dated Dec. 26, 2018, pp. 1-63.
Official Communication for U.S. Appl. No. 15/260,221 dated Jan. 8, 2019, pp. 1-27.
Official Communication for U.S. Appl. No. 15/271,013 dated Mar. 28, 2019, pp. 1-109.
Official Communication for U.S. Appl. No. 15/351,313 dated Apr. 1, 2019, pp. 1-38.
Official Communication for U.S. Appl. No. 15/858,945 dated Feb. 26, 2019, pp. 1-13.
Official Communication for U.S. Appl. No. 15/859,058 dated Mar. 25, 2019, pp. 1-57.
Official Communication for U.S. Appl. No. 15/859,008 dated Apr. 12, 2019, pp. 1-24.
Official Communication for U.S. Appl. No. 14/869,721 dated Jun. 20, 2019, pp. 1-346.
Official Communication for U.S. Appl. No. 14/180,308 dated Jun. 11, 2019, pp. 1-26.
Official Communication for U.S. Appl. No. 15/260,221 dated Jul. 11, 2019, pp. 1-40.
Official Communication for U.S. Appl. No. 15/351,313 dated Jun. 14, 2019, pp. 1-9.
Official Communication for U.S. Appl. No. 14/981,747 dated May 8, 2019, pp. 1-77.
Official Communication for U.S. Appl. No. 15/271,013 dated Jun. 14, 2019, pp. 1-8.
Official Communication for U.S. Appl. No. 14/981,747 dated Aug. 1, 2019, pp. 1-5.
Official Communication for U.S. Appl. No. 15/260,221 dated Sep. 3, 2019, pp. 1-27.
Official Communication for U.S. Appl. No. 15/351,313 dated Aug. 28, 2019, pp. 1-47.
Selen, et al. “Model-Order Selection: A review of information criterion rules,” IEEE Signal Processing Magazine, Jul. 2004, pp. 38-47.
Official Communication for U.S. Appl. No. 14/846,349 dated Jan. 21, 2020, pp. 1-6.
Official Communication for U.S. Appl. No. 15/859,008 dated Feb. 26, 2020, pp. 1-8.
Official Communication for U.S. Appl. No. 14/180,308 dated Mar. 9, 2020, pp. 1-6.
Official Communication for U.S. Appl. No. 14/846,349 dated Jan. 21, 2020.
Official Communication for U.S. Appl. No. 15/859,008 dated Feb. 26, 2020.
Official Communication for U.S. Appl. No. 14/180,308 dated Mar. 9, 2020.
Beraldi, et al., “A Clustering Approach for Scenario Tree Reduction: an Application to a Stochastic Programming Protfolio Optimization Problem,” TOP, vol. 22, No. 3, 2014, pp. 934-949.
Official Communication for U.S. Appl. No. 15/271,013 dated Mar. 23, 2020, pp. 1-45.
Official Communication for U.S. Appl. No. 14/981,747 dated Apr. 23, 2020, pp. 1-14.
Office Communication for U.S. Appl. No. 14/180,308 dated May 11, 2020, pp. 1-18.
Office Communication for U.S. Appl. No. 15/659,008 dated May 28, 2020, pp. 1-20.
Office Communication for U.S. Appl. No. 14/846,349 dated Jun. 8, 2020, pp. 1-32.
Office Communication for U.S. Appl. No. 15/271,013 dated Jun. 15, 2020, pp. 1-6.
Examination Report for UK Patent Application No. GB1617238.9 dated Sep. 24, 2020, pp. 1-7.
Office Communication for U.S. Appl. No. 15/859,008 dated Oct. 9, 2020, pp. 1-22.
Office Communication for U.S. Appl. No. 14/180,308 dated Oct. 13, 2020, pp. 1-17.
Office Communication for U.S. Appl. No. 15/859,008 dated May 28, 2020, pp. 1-20.
Related Publications (1)
Number Date Country
20140136269 A1 May 2014 US
Continuations (1)
Number Date Country
Parent 13675837 Nov 2012 US
Child 13917503 US