Cloud computing services framework

Information

  • Patent Grant
  • 10326708
  • Patent Number
    10,326,708
  • Date Filed
    Thursday, February 7, 2013
    13 years ago
  • Date Issued
    Tuesday, June 18, 2019
    6 years ago
Abstract
Methods and systems of managing computer resources in a cloud environment. Receiving, via at least one server, cloud management instructions via an API, coordinating, via the at least one server, with at least one cloud services manager via a distributed coordination and messaging service. The coordination includes instructions regarding the cloud services manager managing at least one cloud resource, and communicating with at least one database to store and retrieve the management instructions.
Description
TECHNICAL FIELD

The present invention relates to cloud computing. More particularly, the present invention relates to a framework of coordinating and automating processes in cloud computing.


BACKGROUND

Cloud computing is a way for users to store data and operate computational processes on infrastructure connected by a network. Thus, instead of having to purchase physical infrastructure, users send processes and data out to be run and stored on infrastructure owned by other entities. The user only pays for the amount of data storage or processing capability that he or she desires. This allows the user to tap computing resources that would be impossible without owning actual, physical and vast computing resources.


By its very nature, cloud computing gives developers large tool sets for executing programs and tasks along with database storage but it also creates logistical hurdles to implementing those numerous programs, tasks and storage. For instance, developers may be required to write repetitive daemons for cloud managers and object managers to program the logistics each time an object or process is run. This equates to extra work for developers who must devote great resources on cloud logistics, separate and apart from the development of the actual end product.


SUMMARY

Systems and method of managing computer resources in a cloud environment, may comprise receiving, via at least one server, cloud management instructions via an API. Also, coordinating, via the at least one server, with at least one cloud services manager via a distributed coordination and messaging service, wherein the coordination includes instructions regarding the cloud services manager managing at least one cloud resource, and communicating with at least one database to store and retrieve the management instructions.


The systems and methods may also include where the cloud services manager includes at least a cloud services framework, and an API. They may also include where the cloud services manager is further configured to establish and control at least one cloud resource.


Some example systems and methods include where the cloud services manager is located in the cloud. Also, where a controller is configured to interact with the cloud services manager.


Other examples may include where the controller is configured to communicate with the cloud services framework via the API and the cloud services framework is configured to validate the controller. Also where the controller is a cloud resource.


Further examples include where the at least one cloud resource is at least one of a Virtual Machine, an Operating System and a physical computational device. And where the controller is further configured to instantiate at least one cloud service based on parameters entered by a user.


Still other examples include where the controller instantiates at least one cloud service via a start daemon via the API. And where the cloud services framework is configured to scale the at least one cloud resource. Also, where the cloud services framework is configured to request redundant cloud resources in order to replace failed resources. And wherein the API is configured to be dynamically extended by adding services to the cloud.


Other embodiments include where the cloud resource is, or is a combination of, a virtual machine, a physical machine, an operating system, storage service, networking service, and an application service.





BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the systems and methods described in this application, reference should be made to the description below, in conjunction with the following drawings, in which:



FIG. 1 is a schematic illustrating in overview, the cloud computing system, according to some embodiments.



FIG. 2 is a schematic illustrating the CSF embodiment interacting with the cloud environment, according to some embodiments.



FIG. 3 is an illustration of Auto scaling on the cloud in a CSF environment, according to some embodiments.



FIG. 4 is an illustration of High Availability on the cloud in a CSF environment, according to some embodiments.



FIG. 5 is an illustration of High Availability on the cloud in a CSF environment, according to some embodiments.



FIG. 6 is a schematic illustrating an alternate overview of a the cloud computing system, according to some embodiments.





DETAILED DESCRIPTION

In the following detailed descriptions, numerous specific details are set forth to illustrate the subject matter presented in this document. It will, however, be apparent to one of ordinary skill in the art that the subject matter may be practiced without these exact specific details. Moreover, the descriptions are provided by way of example and should not be used to limit the scope of any later claimed inventions.


Computer networks of many kinds exist today. Closed and proprietary networks, open and public networks, and hybrid networks as well. These networks tie together computer servers and databases in ways that allow computer programs to run efficiently. Some embodiments of such networks are known as “cloud computing.” Over such a network, such as the internet, computer resources such as infrastructure, platforms and applications can all be used by client users. The physical resources of such cloud networks may be scattered around various physical locations, but by being networked together, can become a larger resource. Various systems and methods can be used to manage the resources that can be hosted and run on such a cloud network.


Cloud Services Framework (CSF) technology described herein, is a set of cloud management tools that aims to manage the resources for logistical cloud infrastructures. CSF abstracts the logistical cloud coordination in order to replicate it and template it for multiple uses.


CSF facilitates the object/process to pass through the cloud management system to the end Cloud Services Manager (CSM) that works with the cloud environment and resources found there. This system bypasses individualized processing of each object as each are loaded. Instead, CSF is able to handle the underlying logistics while maintaining ambivalence to the individual processes. CSF implements logistics such as backup, scalability, redundancy, fail-safes, placement of services across multiple clouds and security of the cloud environment, without separate developer coding for each process individually.


CSF accomplishes this by allowing developers to set Cloud Service Controllers (Controllers) that define Cloud Service Specification (CSS). These CSSs define how clients or Computational Resources (CRs) are to be managed, essentially pre-programming the logistics of the cloud management for each particular service.


CRs could be any computational resource including but not limited to Virtual Machines (VMs), Operating System (OS) containers, or actual physical computational machines.


CSSs become the Application Programming Interface (API) template for later usage and automatically become commands at the Command Line Interface (CLI) and User Interface (UI) levels. Thus, the Controller implements a simple interface with only a few APIs.


With CSSs, a developer can program coordination, networking, storage, permissions and other cloud objects with the CRs running on the cloud without having to write the actual code to do so. The result is easier implementation of the coordination and running of CRs, networking, storage, permissions and other cloud objects, and ensures that CRs run as the developer intended. Another result is that a developer can concentrate on the actual Cloud Service he or she wants to provide without having to worry about other logistical details.


Cloud Resource is inclusive but not limited to the following, virtual machine, physical machine or a OS container, any Storage services provided a private or public cloud, any Networking services provided by private or public cloud, any feature that is provided by the private or public cloud, any higher level application services like load balancers, AutoScaling, DNS, Platform As a Service (PaaS) provided by the cloud vendor or 3rd party cloud service developers.


Overview

CSF is a set of cloud management tools that can allow for the replication and automation of Cloud Services logistics. This reduces the amount of time developers have to program those logistics individually, each time a service is created.


Logistics include, but are not limited to coordination and running of the CRs, networking cloud resources, storage, permissions, backup, scalability, redundancy, fail-safes, placement of services across multiple clouds and security of the cloud environment.


CSF allows the object/process to pass through the system and handles the underlying logistics of the cloud(s) while maintaining ambivalence to the individual processes. CSF accomplishes this by allowing developers to set Cloud Service Controllers that define CSSs.


These CSSs become the API interface templates for later usage and automatically become commands at the CLI and UI levels.


CSSs define object coordination, networking, storage, permissions and other cloud objects. CSF uses APIs to implement the services/objects defined in the CSSs. The result is easier and faster implementation of the coordination and running of CRs, networking, storage, permissions and other cloud objects, and ensures that CRs run as the developer intended. Another result is that the developer can concentrate on the actual Cloud Service he or she wants to provide without having to worry about other logistical details.


Cloud Servies Framework

Cloud Services Framework aims to reduce the amount of time developers need to manage their client CRs. Through the APIs, developers create a set of Cloud Service Controllers. Cloud Service Controllers are the “software/process/daemon” that implement CSF web APIs/Interfaces. They may be written in any programming paradigm and can be added dynamically by the end user or developer to extend the cloud and add functionality.


Cloud Service Controllers define the CSSs. These CSSs become the API interface template for later implementation. Thus, the Controller implements a simple interface with only a few APIs to run complex cloud processes and coordination.



FIG. 1 depicts one example embodiment of how CSF operates to manage cloud resources.


In this example embodiment, CSF uses an API 110 to communicate with one or more Site Services Managers 116. This user API 110 could be a Restful API, an http based web language with requests similar to fetching a web page. Alternative APIs include Web APIs and standards like Simple Object Access Protocol (SOAP).


Continuing with FIG. 1, one or many Site Services Managers 116 communicate with a database 120 or distributed data store (DDS) and with one or more individual Cloud Services Managers, 130, 132, 134, over a Distributed Coordination and Messaging Service 124 (DC Messaging Service).


The individual Cloud Services Managers, 130, 132, 134, use Controllers 140, 142, 144, to carry out processes in the cloud 150.


Alternatively, The individual Cloud Services Managers 130, 132, 134, can be implemented within their own CRs in the cloud 150 as well.


Cloud Service Specifications (CSSs)


A Cloud Service is an instantiation of a CSS. Developers can customize the CSSs in any way they desire to manage and control desired CRs on the cloud. Multiple services can even be combined in one model.


Developers define CSSs as a set of Service Parameters. Users enter the Service Parameters to initiate a particular service. The Service Parameters are labeled by the developer so users can recognize what is required for each input field.


Once established, the CSS automatically populates in the API, CLI and the UI as a new feature/extension for that cloud implementation so end users can recognize the new feature.


When an end user wants to launch a service, a user calls that particular service using the API. When an end user (or another Cloud Service) uses a cloud service, it actually instantiates a cloud service for its consumption.


The user is presented with the set of defined parameter labels from the CSS and the user enters the parameter values for the desired service. The Service Parameters are entered and added to the DDS. The Service Parameters are validated according to the CSS.


Service Parameters specify how a Cloud Service should be instantiated. The CSF provides the instruments, or APIs, to coordinate cloud resources, saving information in the DDS allowing for many clouds to be utilized and auto scaling of cloud resources.


Once the Service Parameters are saved in the DDS, the DC Messaging Service is notified. The DC Messaging Service assigns the Service Parameters an identifier and puts that identifier into a registry. Each Cloud Services Manager is also assigned an identifier. The DC Messaging Service, using the respective identifiers, assigns a Service Parameter to a particular Cloud Services Manager. The Cloud Services Manager is assigned an instantiation of the Cloud Service according to the load by the Load Balancer or Site Manager according to the load.


Once the Cloud Services Manager has the Service Parameter values, it matches them to a type/class registry that define the different type/class services. The Cloud Services Manager also performs a set of validations on the fetched Service Parameters.


After validated and matched, the Manager can begin executing the service and launch CRs in the cloud, in accordance with the instantiation of CSS or Cloud Service Parameters specified by the user.


Cloud Services Manager and the Cloud


An example embodiment of how CSF is arranged and works is described with reference to FIG. 2. As shown in FIG. 2, a Cloud Services Manager 230 includes the CSF 260. A Controller 270 is depicted as being added to the cloud. The CRs 280, 282, 284, 286, are depicted in the cloud box 240.


The Controller 270 communicates with the CSF 260 through APIs 262. In some embodiments, a Controller 270 can run in its own CR 280 which includes Virtual Machines, its own Operating System (OS) container or actual physical computational device.


Once added, the Cloud Services Manager 230 validates the Controller 270.


The Controller 270 instantiates the Cloud Service based on the parameters entered by the user. To begin, a “start” daemon is communicated from the CSF 260 to the cloud. The Controller 270 can choose to start a daemon on the CR 280 via the APIs 262 provided by the CSF 260.


Once received, CRs 280 are loaded in the cloud and the Controller 270 is informed through an “add_client” notification. Handles that corresponds to the specific CR 280 are assigned. Once a handle is assigned to a CR, 280 that handle is used for later communication with the Controller 270, and to invoke the CR API 262 to manage the CRs 280.


With the CR handle provided as part of “add_client” call, the Controller 270 can set up the CR 280 to provide the service or sub-set of a service. There are many such services but examples include a webserver, memory cache, and load-balancer.


Many multiple CRs 280 can be loaded and instantiated in this way, depending on the desires of the user and the arrangement the Controller 270 requests.


Additionally, other cloud features can be similarly utilized. For example, user specified storage and public IF could be managed by the Cloud Service to make the service resilient and publically available. The CSF 260 provides the APIs 262 necessary to manage CRs 280, storage, networking, security resources which could exist in multiple private or public clouds, or both private and public clouds.


Not only multiple processes but any kind of process may be instantiated this way. For example, storage may be attached to a CR 280, persistent public IP may be assigned, config files may be loaded or any other number of APIs 262.


CSF 260 provides APIs 262 for the Controller to save statistics and information about the Cloud Service in a persistent data store so that users can access that information. The information/data can be used to make changes to the services, add services, remove services, start and stop CRs 280.


The user decides when to delete the Cloud Service that the user instantiated. When users request the Cloud Service instantiation to go away, then all the cloud resources created by the framework and the Controller 270 are cleaned up. “Stop” API implemented by the Controller 270 is called to indicate that the service instantiation is stopped.


Auto Scalability


CSF can be programmed to automatically scale cloud usage up or down depending on criteria determined by a developer. New CRs can be launched or existing ones taken down depending on the needs of the user which allows for efficient usage of cloud resources. This is depicted in FIG. 3 by showing many of the same type of CR 380 utilized by the CSF 360.


For example, a developer can initially call three CRs 380 to be web servers. Based on the load criteria determined by the developer, CSF 360can prepare more or fewer servers for future use. The Controller 370can monitor the cloud service. The CSF 360 provides APIs 362 to help with that monitoring. When a particular threshold is met, specific to the cloud service, the Controller 370 can request to scale up or down via APIs 362 provided by the CSF 360, in this example, adding or removing web servers depending on the load.


In another example, the Controller 370 monitors the cloud service. The Controller 370 requests to scale up based on some criteria by sending a notification to the Service API 362. A new CR 380 is created and launched in the cloud. The Service API 362 informs the Controller 370 of the new CR 380 by communicating an “add_client” notification. Likewise, if the Controller 370 asks the Service API 362 to scale down, the VM is removed, the Service API 362 informs the Controller 370 by passing a “remove_client” notification.


In still another example, where in a typical cloud environment utilizing CSF 360and running a load balancing/proxy, two Apache servers and a MySQL database, another server is needed. The Cloud Service application monitors the cloud. When the load on the web servers trips a threshold value, the Controller sends a Scale Up notification to the Service API. A new VM is created and launched into the cloud. The Service API 362 notifies the Controller 370 by sending an Add-Client notification. The new VM becomes a new Apache server to handle the large load and the Controller sends a config file to the new server and a reconfigure proxy to the cloud. The new server is then integrated into the cloud system.


These parameters are fully customizable by the Controller and implemented by the cloud through the CR APIs.


Cloud Services Framework Defers Processing


CSF can instantiate services/use CRs and not have to understand or process the specific CR code until late in the process. Thus, as depicted in FIG. 1, the Site Manager, DDS, and DC Messaging Service do not have to process the service code. It is not until the service or CR reaches the individual Cloud Services Controller that the actual understanding of the particular service code is necessary.


CSF coordinates the delegation of the service to a particular Cloud Services Manager by assigning an identifier to the service parameter and storing it in the DDS. Thus, the CSF parameters are initially passed all the way to the Controller, while keeping the rest of the underlying infrastructure from understanding the specifics of the service parameter.


The result is a generic front-end handling processes and a delay of handling the internal code of the service until it reaches the end Cloud Services Manager to be handled by the Controller.


Delaying of processing of code allows CSF to genericize and automate the logistics. Logistics include, but are not limited to coordination and running of the CRs, networking cloud resources, storage, permissions, backup, scalability, redundancy, fail-safes, placement of services across multiple clouds and security of the cloud environment.


High Availability


With respect to FIG. 4, CSF 460 allows for High Availability (HA) of Cloud Services. HA is the ability for cloud resources to be automatically redundant for fail-safe purposes but also for adaptability of services.


CSF 460 can be programmed using HA aspects in order to automatically assign a new CR to take over for a failed machine. It can promote CR 480 already in use to take over for failed CRs 482.


When a CR dies or becomes unavailable, the Controller 470 is notified via the “remove_client” API. Another CR is automatically arranged for and the Controller 470 is notified via the “add client” API.


In one example, HA also allows for developers to choose how redundant and fail-safe they want their processes to be. Multiple redundancies can be requested by the Controller 470. This is depicted in FIG. 4 with a newly generated CR 480 replacing a dead one 482. Because cloud computing is typically billed by usage, more redundancies cost more money, but CSF allows users to program automatic redundancies that are not used until needed.


Referring to FIG. 5, that depicts a more detailed and specific example of HA. For instance, in a cloud environment, a CR can die or become unavailable. If that happens, the Cloud Service Controller 570 is notified by passing a “remove_client” notification from the Service API 562 and a new CR is automatically launched to cover for the dead or unavailable one. Then, the Controller is notified that the new CR is up by the Service API 562 sending the Controller 570 an Add-Client notification.


In this example, the cloud environment utilizing CSF 560, is running a Load Balancer/Proxy 580, two Apache servers 582, 584 and a MySQL database 586. If one Apache server 582 dies or becomes unavailable, a “remove_client” notification is sent from the Service API 562 to the Controller 570. A Reconfigure Proxy 592 communication is sent from the client API to the cloud 540. A new CR590 is created to take the place of the failed one and it is launched. The Controller 570 is notified by an Add-Client 592 notification from the Service API 562. Another Reconfigure Proxy is communicated to the CRs.


Still another example demonstrates how existing CRs can be reconfigured to the role of a dead or unavailable CR. Redundant CRs can run simultaneously and IP and Storage is reattached if the first CR goes down and the redundant CR is promoted. These instructions are all programmed into the CSS logistical coordination.


CSF with a Load Balancer


An alternative embodiment of FIG. 1 is depicted in FIG. 6. In this embodiment, the APIs 610 are used to communicate to the cloud 650 through a distributing load balancer 612. The load balancer 612 distributes services to different CRs 630 which are each in communication with a DDS 620. A DC Messaging Service 614 coordinates the communication among the DDSs, CRs and the Load Balancer 612 on the cloud 650 and among clouds.


Multiple Clouds


References in this disclosure to a “cloud,” “clouds,” or a “cloud environment,” should be understood to encompass one cloud but also but any combination of cloud resources. For instance, CSF can utilize not only one cloud to run CRs, but numerous clouds, both public and private.


Private clouds can work in concert among one another, and also link to a public cloud. This arrangement can provide security options to developers and cloud administrators.


Cloud resources can be networked together seamlessly by CSF. This is because logistics of cloud resources including cloud resource management, can all be programmed into the CSSs. This allows users to utilize cloud resources without even being aware of what cloud they are utilizing.


Security


Permissions can be applied on each cloud service individually by the cloud administrator. Users can view, list, add, or delete permissions.


All CRs that are instantiated are completely isolated by firewalls.


Security rules can be dynamically applied to the CRs. For example, Cloud Services Controllers can selectively open ports via APIs for exposing only a specific service.


Each Controller can run inside its own isolated OS container or VM.


All API calls are encrypted using Secure Socket Layer (SSL) certificates.


Third Party Feature Addition


Cloud computing presents a vast set of resources for utilization. Therefore, it makes sense to open the management of the cloud services to as many entities as possible, in order to develop the best management tools for those resources. Therefore, CSF allows for the maximum participation by users in adding to, adapting and enhancing the cloud services available.


CSF does this by allowing users and Third Party developers to add their own cloud features or services. This effectively allows other Third Party users and developers to extend the cloud in any way they see fit.


New Cloud Services can be added via the RESTful API, RESTful API/http, CLI and UI dynamically and they will be available for immediate use.


CONCLUSION

The foregoing description, for purpose of explanation, has been described with reference to specific examples. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. This includes practicing the examples of the various subject matter described above in any combination. The examples were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the inventions with various modifications as are suited to the particular use contemplated.


As disclosed herein, features consistent with the present inventions may be implemented via computer-hardware, software and/or firmware. For example, the systems and methods disclosed herein may be embodied in various forms including, for example, a data processor, such as a computer that also includes a database, digital electronic circuitry, firmware, software, computer networks, servers, or in combinations of them. Further, while some of the disclosed implementations describe specific hardware components, systems and methods consistent with the innovations herein may be implemented with any combination of hardware, software and/or firmware. Moreover, the above-noted features and other aspects and principles of the innovations herein may be implemented in various environments. Such environments and related applications may be specially constructed for performing the various routines, processes and/or operations according to the invention or they may include a general-purpose computer or computing platform selectively activated or reconfigured by code to provide the necessary functionality. The processes disclosed herein are not inherently related to any particular computer, network, architecture, environment, or other apparatus, and may be implemented by a suitable combination of hardware, software, and/or firmware. For example, various general-purpose machines may be used with programs written in accordance with teachings of the invention, or it may be more convenient to construct a specialized apparatus or system to perform the required methods and techniques.


Aspects of the method and system described herein, such as the logic, may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (“PLDs”), such as field programmable gate arrays (“FPGAs”), programmable array logic (“PAL”) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits. Some other possibilities for implementing aspects include: memory devices, microcontrollers with memory (such as EEPROM), embedded microprocessors, firmware, software, etc. Furthermore, aspects may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. The underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (“MOSFET”) technologies like complementary metal-oxide semiconductor (“CMOS”), bipolar technologies like emitter-coupled logic (“ECL”), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, and so on.


It should also be noted that the various logic and/or functions disclosed herein may be enabled using any number of combinations of hardware, firmware, and/or as data and/or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and/or other characteristics. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) and carrier waves that may be used to transfer such formatted data and/or instructions through wireless, optical, or wired signaling media or any combination thereof. Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks via one or more data transfer protocols (e.g., HTTP, FTP, SMTP, and so on).


Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of “including, but not limited to.” Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words “herein,” “hereunder,” “above,” “below,” and words of similar import refer to this application as a whole and not to any particular portions of this application. When the word “or” is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.


Although certain presently preferred implementations of the invention have been specifically described herein, it will be apparent to those skilled in the art to which the invention pertains that variations and modifications of the various implementations shown and described herein may be made without departing from the spirit and scope of the invention. Accordingly, it is intended that the invention be limited only to the extent required by the applicable rules of law.


The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.

Claims
  • 1. A method for managing computer resources in a cloud environment, the method comprising: receiving, by a site service manager, a request from a user for a service;presenting, by the site service manager, one or more parameters to the user based on a specification for the service, wherein the specification for the service is defined by the one or more parameters that specify how the service is to be instantiated;receiving, by the site service manager, one or more parameter values from the user, the one or more parameter values corresponding to the one or more parameters of the specification for the service;assigning, by the site service manager, the one or more parameter values to a cloud service manager of a plurality of cloud service managers using an identifier for the one or more parameter values;validating, by the cloud services manager, the one or more parameters values for the one or more parameters;upon validating the one or more parameter values, executing, by the cloud services manager, the service based upon the one or more values, the cloud services manager comprising a framework providing a set of application programming interfaces (APIs) usable by a controller, wherein the executing the service comprises instantiating, by the controller using one or more APIs from the set of APIs, the service using a set of one or more computational resources (CRs) for the service according to the one or more parameter values;monitoring, by the controller using one or more APIs from the set of APIs, the service; andupon determining, by the controller, that a particular threshold is met for the service, causing, by the controller using one or more APIs from the set of APIs, a new CR to be created for the first service or a CR to be removed from the set of one or more CRs for the first service.
  • 2. The method of claim 1, wherein the one or more CRs include a physical computational device, a storage service, a networking service, a higher level application, or a combination thereof.
  • 3. The method of claim 1, wherein the instantiating the service comprises: starting, by the controller using one or more APIs from the set of APIs, a daemon on the set of one or more CRs;upon the daemon being started, loading the set of one or more CRs to the cloud environment and assigning a handle to the set of one or more CRs within the cloud environment; andconfiguring, by the controller using one or more APIs from the set of APIs and the handle, the set of one or more CRs to provide the service according to the one or more parameter values.
  • 4. The method of claim 3, wherein the one or more parameters are labeled such that the user can recognize what is required for each of the one or more parameters.
  • 5. The method of claim 3, wherein the configuring the set of one or more CRs includes populating in an API, a Command Line Interface (CLI), and a User Interface (UI) as a new feature or extension for the service so end users can recognize and use the service.
  • 6. A system for managing computer resources in a cloud environment, the system comprising: one or more processors; anda memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising: receiving a request from a user for a service;presenting one or more parameters to the user based on a specification for the service, wherein the specification for the service is defined by the one or more parameters that specify how the service is to be instantiated;receiving one or more parameter values from the user, the one or more parameter values corresponding to the one or more parameters of the specification for the service; andassigning the one or more parameter values to a cloud service manager of a plurality of cloud service managers using an identifier for the one or more parameter;validating the one or more parameters values for the one or more parameters;upon validating the one or more parameter values, executing the service based upon the one or more values in a framework that provides a set of application programming interfaces (APIs) usable by a controller, wherein the executing the service comprises instantiating, by the controller using one or more APIs from the set of APIs, the service using a set of one or more computational resources (CRs) for the service according to the one or more parameter values;monitoring the service; andupon determining that a particular threshold is met for the service, causing a new CR to be created for the first service or a CR to be removed from the set of one or more CRs for the first service.
  • 7. The system of claim 6, wherein the one or more CRs include a Virtual Machine, a physical computational device, a storage service, a networking service, a higher level application, or a combination thereof.
  • 8. The system of claim 6, wherein the instantiating the service comprises: starting a daemon on the set of one or more CRs;upon the daemon being started, loading the set of one or more CRs to the cloud environment and assigning a handle to the set of one or more CRs within the cloud environment; andconfiguring the set of one or more CRs to provide the service according to the one or more parameter values.
  • 9. The method of claim 8, wherein the one or more parameters are labeled such that the user can recognize what is required for each of the one or more parameters.
  • 10. The method of claim 8, wherein the configuring the set of one or more CRs includes populating in an API, a Command Line Interface (CLI), and a User Interface (UI) as a new feature or extension for the service so end users can recognize and use the service.
  • 11. A non-transitory computer-readable medium storing one or more instructions that, upon execution by one or more processors of a services manager in a cloud computing system, causes the one or more processors to: receive, at a site service manager, a request from a user for a service;present one or more parameters to the user based on a specification for the service, wherein the specification for the service is defined by the one or more parameters that specify how the service is to be instantiated;receive, at the site service manager, one or more parameter values from the user, the one or more parameter values corresponding to the one or more parameters of the specification for the service;assign the one or more parameter values to a cloud service manager of a plurality of cloud service managers using an identifier for the one or more parameter values;validate the one or more parameters values for the one or more parameters;upon validating the one or more parameter values, executing, at the cloud services manager, the service based upon the one or more values, the cloud services manager comprising a framework providing a set of application programming interfaces (APIs) usable by a controller, wherein the executing the service comprises instantiating, by the controller using one or more APIs from the set of APIs, the service using a set of one or more computational resources (CRs) for the service according to the one or more parameter values;monitor, using one or more APIs from the set of APIs, the service; andupon determining that a particular threshold is met for the service, cause, by the controller using one or more APIs from the set of APIs, a new CR to be created for the first service or a CR to be removed from the set of one or more CRs for the first service.
  • 12. The non-transitory computer-readable medium of claim 11 wherein the one or more CRs include includes a Virtual Machine, a physical computational device, a storage service, a networking service, a higher level application, or a combination thereof.
  • 13. The non-transitory computer-readable medium of claim 11, wherein the instantiating the service comprises: starting, by the controller using one or more APIs from the set of APIs, a daemon on the set of one or more CRs;upon the daemon being started, loading the set of one or more CRs to the cloud environment and assigning a handle to the set of one or more CRs within the cloud environment; andconfiguring, by the controller using one or more APIs from the set of APIs and the handle, the set of one or more CRs to provide the service according to the one or more parameter values.
  • 14. The non-transitory computer-readable medium of claim 13, wherein the one or more parameters are labeled such that the user can recognize what is required for each of the one or more parameters.
  • 15. The non-transitory computer-readable medium of claim 13, wherein the configuring the set of one or more CRs includes populating in an API, a Command Line Interface (CLI), and a User Interface (UI) as a new feature or extension for the service so end users can recognize and use the service.
CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Stage application that claims priority under 35 U.S.C. 371 to PCT Application No. PCT/US2013/025186, filed on Feb. 7, 2013, titled “CLOUD COMPUTING SERVICES FRAMEWORK”, which claims the benefit and priority under 35 U.S.C. 119(e) from U.S. Provisional Application No. 61/597,610, filed Feb. 10, 2012, titled “CLOUD COMPUTING SERVICES FRAMEWORK”.

PCT Information
Filing Document Filing Date Country Kind
PCT/US2013/025186 2/7/2013 WO 00
Publishing Document Publishing Date Country Kind
WO2013/119841 8/15/2013 WO A
US Referenced Citations (189)
Number Name Date Kind
4086628 Woodrum Apr 1978 A
5239648 Nukui Aug 1993 A
5463774 Jenness Oct 1995 A
5832505 Kasso et al. Nov 1998 A
6038399 Fisher et al. Mar 2000 A
6047129 Frye Apr 2000 A
6473800 Jerger et al. Oct 2002 B1
6772350 Belani et al. Aug 2004 B1
6944777 Belani et al. Sep 2005 B1
7200865 Roscoe et al. Apr 2007 B1
7225210 Guthrie, II May 2007 B2
7366793 Kenner et al. Apr 2008 B2
7475419 Basu et al. Jan 2009 B1
7602756 Gu et al. Oct 2009 B2
7631306 Puig et al. Dec 2009 B1
7734643 Waterhouse et al. Jun 2010 B1
7886038 Ferris Feb 2011 B2
7890626 Gadir Feb 2011 B1
7921452 Ridion et al. Apr 2011 B2
7953823 Rider et al. May 2011 B2
7958246 Barber Jun 2011 B2
8037108 Chang Oct 2011 B1
8086177 Trift et al. Dec 2011 B2
8108377 Jiang et al. Jan 2012 B2
8112505 Ben-Shaul et al. Feb 2012 B1
8201175 Zhu Jun 2012 B2
8209415 Wei Jun 2012 B2
8291079 Colton Oct 2012 B1
8341141 Krislov Dec 2012 B2
8347263 Offer Jan 2013 B1
8429162 Wang et al. Apr 2013 B1
8438654 von Eicken et al. May 2013 B1
8464250 Ansel Jun 2013 B1
8468455 Jorgensen et al. Jun 2013 B2
8472438 Kini et al. Jun 2013 B2
8495611 McCarthy et al. Jul 2013 B2
8504689 Ferris et al. Aug 2013 B2
8509231 Hoole et al. Aug 2013 B2
8577937 Offer Nov 2013 B1
8584215 Narasimha et al. Nov 2013 B2
8650152 Dettinger et al. Feb 2014 B2
8850528 Van Biljon et al. Sep 2014 B2
8938540 Van Biljon et al. Jan 2015 B2
8977679 Van Biljon et al. Mar 2015 B2
9021009 Van Biljon et al. Apr 2015 B2
9032069 Van Biljon et al. May 2015 B2
9076168 Van Biljon et al. Jul 2015 B2
9087352 Van Biljon et al. Jul 2015 B2
9171323 Van Biljon et al. Oct 2015 B2
9202239 Van Biljon et al. Dec 2015 B2
9218616 Van Biljon et al. Dec 2015 B2
9232000 Pittman Jan 2016 B1
20020052941 Patterson May 2002 A1
20020097747 Kirkby et al. Jul 2002 A1
20030018927 Gadir et al. Jan 2003 A1
20030037284 Srinivasan et al. Feb 2003 A1
20030105810 McCrory et al. Jun 2003 A1
20030229623 Chang et al. Dec 2003 A1
20040024892 Creswell et al. Feb 2004 A1
20040184070 Kiraly et al. Sep 2004 A1
20040250120 Ng Dec 2004 A1
20050038834 Souder et al. Feb 2005 A1
20050055306 Miller et al. Mar 2005 A1
20050065855 Geller Mar 2005 A1
20050187937 Kawabe et al. Aug 2005 A1
20050193218 Susser et al. Sep 2005 A1
20050278441 Bond et al. Dec 2005 A1
20060112176 Liu et al. May 2006 A1
20060209868 Callaghan Sep 2006 A1
20060212545 Nichols et al. Sep 2006 A1
20060259947 Aarnos et al. Nov 2006 A1
20070072591 McGary et al. Mar 2007 A1
20070162456 Agassi et al. Jul 2007 A1
20070234332 Bundridge et al. Oct 2007 A1
20070255798 Schneider Nov 2007 A1
20080052203 Beyer et al. Feb 2008 A1
20080195760 Nudler Aug 2008 A1
20080228734 Kang Sep 2008 A1
20080295094 Korupolu et al. Nov 2008 A1
20080316980 Ahlen et al. Dec 2008 A1
20090024522 Reunert et al. Jan 2009 A1
20090172662 Liu Jul 2009 A1
20090182622 Agarwal et al. Jul 2009 A1
20090210429 Agrawal et al. Aug 2009 A1
20090228950 Reed Sep 2009 A1
20090235342 Manion et al. Sep 2009 A1
20090240728 Shukla et al. Sep 2009 A1
20090276771 Nickolov et al. Nov 2009 A1
20090300210 Ferris Dec 2009 A1
20090300350 Gai et al. Dec 2009 A1
20090319529 Bartlett et al. Dec 2009 A1
20090327471 Astete et al. Dec 2009 A1
20100036736 McGee et al. Feb 2010 A1
20100042720 Stienhans et al. Feb 2010 A1
20100061391 Sindhu et al. Mar 2010 A1
20100070501 Walsh et al. Mar 2010 A1
20100071035 Budko et al. Mar 2010 A1
20100083004 Kirshenbaum Apr 2010 A1
20100114714 Vitek May 2010 A1
20100161717 Albrecht Jun 2010 A1
20100169477 Stienhans et al. Jul 2010 A1
20100180014 Kannan et al. Jul 2010 A1
20100185455 Miller Jul 2010 A1
20100194963 Terashima Aug 2010 A1
20100197267 Raleigh Aug 2010 A1
20100198972 Umbehocker Aug 2010 A1
20100217840 Dehaan et al. Aug 2010 A1
20100223385 Gulley et al. Sep 2010 A1
20100250748 Sivasubramanian et al. Sep 2010 A1
20100250956 Reed et al. Sep 2010 A1
20100251242 Sivasubramanian et al. Sep 2010 A1
20100269109 Cartales Oct 2010 A1
20100274769 Hazlewood et al. Oct 2010 A1
20100275059 Hazlewood et al. Oct 2010 A1
20100318645 Hoole et al. Dec 2010 A1
20100332629 Cotugno et al. Dec 2010 A1
20110016214 Jackson Jan 2011 A1
20110022652 Lai et al. Jan 2011 A1
20110022812 van der Linden et al. Jan 2011 A1
20110055034 Ferris et al. Mar 2011 A1
20110055378 Ferris et al. Mar 2011 A1
20110055399 Tung et al. Mar 2011 A1
20110055712 Tung et al. Mar 2011 A1
20110078679 Bozek et al. Mar 2011 A1
20110096174 King et al. Apr 2011 A1
20110099146 McAlister et al. Apr 2011 A1
20110106875 Koenig May 2011 A1
20110119381 Glover et al. May 2011 A1
20110126047 Anderson et al. May 2011 A1
20110126168 Ilyayev May 2011 A1
20110126197 Larsen et al. May 2011 A1
20110185063 Head et al. Jul 2011 A1
20110191610 Agarwal et al. Aug 2011 A1
20110209064 Jorgensen et al. Aug 2011 A1
20110213687 Ferris et al. Sep 2011 A1
20110214124 Ferris et al. Sep 2011 A1
20110225299 Nathuji et al. Sep 2011 A1
20110225467 Betzler et al. Sep 2011 A1
20110231525 Balani et al. Sep 2011 A1
20110231899 Pulier et al. Sep 2011 A1
20110243553 Russell Oct 2011 A1
20110246253 Yu et al. Oct 2011 A1
20110246984 Sharp et al. Oct 2011 A1
20110258692 Morrison et al. Oct 2011 A1
20110265081 Lucovsky et al. Oct 2011 A1
20110265147 Liu Oct 2011 A1
20110265164 Lucovsky Oct 2011 A1
20110270721 Kusterer Nov 2011 A1
20110270968 Salsburg et al. Nov 2011 A1
20110282832 Rishel et al. Nov 2011 A1
20110282940 Zhang Nov 2011 A1
20110295999 Ferris et al. Dec 2011 A1
20110296019 Ferris et al. Dec 2011 A1
20110307391 Mercuri et al. Dec 2011 A1
20110307899 Lee et al. Dec 2011 A1
20120017112 Broda et al. Jan 2012 A1
20120030069 Garg et al. Feb 2012 A1
20120030672 Zygmuntowicz et al. Feb 2012 A1
20120047239 Donahue Feb 2012 A1
20120072985 Davne et al. Mar 2012 A1
20120078849 Bryant et al. Mar 2012 A1
20120079276 Evans et al. Mar 2012 A1
20120096158 Astete et al. Apr 2012 A1
20120110055 Van Biljon et al. May 2012 A1
20120110056 Van Biljon et al. May 2012 A1
20120110180 Van Biljon et al. May 2012 A1
20120110188 Van Biljon et al. May 2012 A1
20120110636 Van Biljon et al. May 2012 A1
20120110650 Van Biljon et al. May 2012 A1
20120110651 Van Biljon et al. May 2012 A1
20120116937 Van Biljon et al. May 2012 A1
20120117229 Van Biljon et al. May 2012 A1
20120124211 Kampas et al. May 2012 A1
20120150791 Willson Jun 2012 A1
20120185913 Martinez Jul 2012 A1
20120221603 Kothule et al. Aug 2012 A1
20120246646 Bakman Sep 2012 A1
20120259815 Olson Oct 2012 A1
20120310880 Giampaolo et al. Dec 2012 A1
20120310991 Frantz et al. Dec 2012 A1
20130019015 Devarakonda Jan 2013 A1
20130030853 Agarwal et al. Jan 2013 A1
20130036213 Hasan et al. Feb 2013 A1
20130060839 Van Biljon et al. Mar 2013 A1
20130125217 Edwards et al. May 2013 A1
20130173798 Micucci et al. Jul 2013 A1
20150006482 Hardy et al. Jan 2015 A1
20150120936 Palan et al. Apr 2015 A1
20150264121 Van Biljon et al. Sep 2015 A1
Foreign Referenced Citations (23)
Number Date Country
101969475 Feb 2011 CN
102223398 Oct 2011 CN
103488697 Jan 2014 CN
104303175 Jan 2015 CN
104335179 Feb 2015 CN
105339940 Feb 2016 CN
104335179 Jan 2018 CN
104303175 Jun 2018 CN
2583211 Apr 2013 EP
2812809 Dec 2014 EP
2815346 Dec 2014 EP
3014485 May 2016 EP
2015507301 Mar 2015 JP
2015512091 Apr 2015 JP
2016524255 Aug 2016 JP
6231020 Nov 2017 JP
2009151729 Dec 2009 WO
WO2011159842 Dec 2011 WO
2011159842 Mar 2012 WO
2012167108 Dec 2012 WO
2013119841 Aug 2013 WO
2013122815 Aug 2013 WO
2014209848 Dec 2014 WO
Non-Patent Literature Citations (92)
Entry
U.S. Appl. No. 13/299,262, Non-Final Office Action dated Nov. 17, 2014, 14 pages.
U.S. Appl. No. 13/299,157 Non-Final Office Action dated Dec. 3, 2014, 10 pages.
U.S. Appl. No. 13/299,206 Notice of Allowance dated Dec. 19, 2014, 10 pages.
U.S. Appl. No. 13/299,319 Notice of Allowance dated Jan. 13, 2015, 9 pages.
U.S. Appl. No. 13/299,301, Final Office Action dated Jan. 29, 2015, 13 pages.
U.S. Appl. No. 13/299,262, Notice of Allowance dated Mar. 3, 2015, 10 pages.
U.S. Appl. No. 13/299,157, Notice of Allowance dated Feb. 27, 2015, 9 pages.
U.S. Appl. No. 13/299,066, Non-Final Office Action dated Mar. 19, 2015, 17 pages.
U.S. Appl. No. 13/299,339, Notice of Allowance dated Apr. 21, 2015, 17 pages.
U.S. Appl. No. 14/459,104 3rd Party Submission mailed on May 29, 2015, 14 pages.
International Preliminary Report on Patentability for application PCT/US2013/025211 (dated Aug. 28, 2014).
U.S. Appl. No. 13/299,066, Final Office Action dated Jul. 18, 2014, 19 pages.
U.S. Appl. No. 13/299,335, Notice of Allowance dated Jul. 18, 2014, 7 pages.
U.S. Appl. No. 13/299,339, Non-Final Office Action dated Sep. 3, 2014, 8 pages.
U.S. Appl. No. 13/299,287, Notice of Allowance dated Sep. 16, 2014, 16 pages.
U.S. Appl. No. 13/299,335, Notice of Allowance dated Sep. 15, 2014, 10 pages.
International Application No. PCT/US2011/040590, International Preliminary Report on Patentability dated Jan. 3, 2013, 11 pages.
International Application No. PCT/US2013/025186, International Preliminary Report on Patentability dated Aug. 21, 2014, 6 pages.
U.S. Appl. No. 13/299,287, Corrected Notice of Allowability dated Oct. 15, 2014, 2 pages.
U.S. Appl. No. 13/299,335, Corrected Notice of Allowability dated Oct. 6, 2014, 4 pages.
U.S. Appl. No. 13/299,339, Notice of Allowance dated Jun. 27, 2014, 17 pages.
“A White Paper from the Open Cloud Standards”, Interoperable Clouds, Version: 1.0.0, Nov. 11, 2009, pp. 1-21.
“Managing VMware VirtualCenter Roles and Permissions”, Retrieved from the Internet: URL:http://www.vmware.com/pdf/vi3_vc_roles.pdf, Apr. 4, 2007, 15 pages.
“VMware Virtual Networking Concepts”, Retrieved from the internet: https://www.vmware.com/files/pdf/virtual_networking_concepts.pdf, Jul. 18, 2007, 12 pages.
Danwei et al., “Access Control of Cloud Service Based on UCON”, Cloud Computing, Springer Berlin Heidelberg, Dec. 1, 2009, pp. 559-564.
EP11796398.3, “Extended European Search Report”, dated May 25, 2016, 14 pages.
EP11796398.3, “Partial Supplementary European Search Report”, dated Apr. 7, 2016, 6 pages.
EP13746545.6, “Extended European Search Report”, dated Apr. 26, 2016, 8 pages.
Fitzpatrick, “Tonido Keeps Cloud Computing Local”, Retrieved from the Internet: URL:http:f/lifehacker.com/5208833/tonido-keeps-cloud-computing-local, Apr. 13, 2009, 2 pages.
Thain et al., “Distributed Computing in Practice: The Condor Experience”, Concurrency and Computation: Practice and Experience, vol. 17, Jan. 1, 2005, 37 pages.
U.S. Appl. No. 14/226,557, “Non-Final Office Action”, dated May 19, 2016, 9 pages.
U.S. Appl. No. 14/459,104 , “Non-Final Office Action”, dated May 20, 2016, 28 pages.
U.S. Appl. No. 14/724,043, “Non-Final Office Action”, dated Jun. 16, 2016, 24 pages.
International Search Report for application PCT/US2013/025186 (dated Apr. 19, 2013).
International Search Report for application PCT/US2013/025211 (dated Apr. 16, 2013).
International Search Report for application PCT/US11/040590 (dated Jan. 5, 2012).
Karjoth, Gunter, “Access Control with IBM Trivoli Access”, May 2003, IBM Research, Zurich Research Laboratory, vol. 6, No. 2.
Canetti et al. “Practical delegation of computation using multiple servers,” CCS '11 Proceedings of the 18th ACM conference on Computer and communications security, pp. 445-454(Oct. 2011).
U.S. Appl. No. 13/299,004, Non-Final Office Action dated Apr. 15, 2013, 58 pages.
U.S. Appl. No. 13/299,004, Final Office Action dated Oct. 2, 2013, 42 pages.
U.S. Appl. No. 13/299,066 Non-Final Office Action dated Apr. 12, 2013, 26 pages.
U.S. Appl. No. 13/299,066, Final Office Action dated Oct. 22, 2013, 25 pages.
U.S. Appl. No. 13/299,157, Non-Final Office Action dated May 24, 2013, 14 pages.
U.S. Appl. No. 13/299,157 Final Office Action dated Dec. 5, 2013, 9 pages.
U.S. Appl. No. 13/299,262, Non-Final Office Action dated May 2, 2013, 12 pages.
U.S. Appl. No. 13/299,262, Final Office Action dated Oct. 30, 2013, 11 pages.
U.S. Appl. No. 13/299,287, Non-Final Office Action dated Oct. 28, 2013, 18 pages.
U.S. Appl. No. 13/299,335, Non-Final Office Action dated Nov. 22, 2013, 24 pages.
U.S. Appl. No. 13/299,339, Non-Final Office Action dated Nov. 16, 2012, 12 pages.
U.S. Appl. No. 13/299,339, Final Office Action dated Aug. 7, 2013, 16 pages.
U.S. Appl. No. 13/299,066, Non-Final Office Action dated Mar. 12, 2014, 16 pages.
U.S. Appl. No. 13/299,004, Notice of Allowance dated Mar. 24, 2014, 22 pages.
U.S. Appl. No. 13/299,335, Notice of Allowance dated May 1, 2014, 7 pages.
U.S. Appl. No. 13/299,206, Non-Final Office Action dated May 22, 2014, 10 pages.
U.S. Appl. No. 13/299,287, Final Office Action dated May 23, 2014, 12 pages.
U.S. Appl. No. 13/299,301, Non-Final Office Action dated Jun. 12, 2014, 18 pages.
U.S. Appl. No. 13/299,319, Non-Final Office Action dated Jun. 6, 2014, 13 pages.
European Search Report for Application No. EP 13748986 dated Sep. 8, 2015, 7 pages.
International Preliminary Report on Patentability for Application No. PCT/US2014/043599, dated Jan. 7, 2016, 8 pages.
International Search Report and Written Opinion for Application No. PCT/US2014/043599, dated Oct. 14, 2014, 11 pages.
U.S. Appl. No. 13/299,066, Notice of Allowance dated Jul. 31, 2015, 23 pages.
U.S. Appl. No. 13/299,301, Notice of Allowance dated Jun. 12, 2015, 5 pages.
U.S. Appl. No. 13/299,339, Advisory Action dated Oct. 18, 2013, 2 pages.
Baron Schwartz et al., “High Performance MySQL: Optimization, Backups, Replication, and More”, Jun. 2008, pp. 428-429.
Liu et al., “NIST Cloud Computing Reference Architecture”, NIST Special Publication 500-292, Sep. 8, 2011, 35 pages.
M-T Schmidt et al., “The Enterprise Service Bus: Making service-oriented acrchitecture real”, IBM Systems Journal, vol. 44, No. 4, Oct. 1, 2005, pp. 781-797.
U.S. Appl. No. 14/226,557, Notice of Allowance dated Nov. 25, 2016, 9 pages.
U.S. Appl. No. 14/724,043, Non-Final Office Action dated Dec. 30, 2016, 12 pages.
U.S. Appl. No. 14/459,104, Final Office Action dated Dec. 2, 2016, 43 pages.
Chinese Application No. 201380014187.4, Office Action dated May 27, 2016, 14 pages (6 pages of Original document and 8 pages of English Translation).
U.S. Appl. No. 14/724,043, Notice of Allowance dated May 19, 2017, 19 pages.
Chinese Application No. 201380014187.4, Office Action dated Feb. 4, 2017, 7 pages (3 pages for the original document and 4 pages for the English translation).
Chinese Application No. 201380014746.1, Office Action dated Mar. 27, 2017, 12 pages.
Japanese Application No. 2014-556691, Office Action dated Jan. 31, 2017, 8 pages (4 pages for the original document and 4 pages for the English translation).
U.S. Appl. No. 14/459,104, “Non-Final Office Action”, dated Oct. 18, 2017, 37 pages.
CN201380014187.4, “Office Action”, dated Aug. 17, 2017, 15 pages.
JP2014-556683, “Office Action”, dated Oct. 17, 2017, 5 pages.
European Application No. 11796398.3, Office Action dated Aug. 8, 2017, 4 pages.
European Application No. 13746545.6, Summons to Attend Oral Proceedings mailed on Apr. 25, 2018, 11 pages.
Japanese Application No. 2014-556683, Office Action dated Jun. 5, 2018, 6 pages (3 pages of the original document and 3 pages of the English translation).
U.S. Appl. No. 15/692,929, Non-Final Office Action dated Jun. 14, 2018, 10 pages.
Chinese Application No. 201480036405.9, Office Action dated Jun. 15, 2018, 15 pages (6 pages of the original document and 9 pages of the English translation).
European Application No. 14740064.2, Office Action dated Aug. 8, 2018, 7 pages.
Hewitt, Cassandra—The Definitive Guide, Distributed Data at web Scale, O'Reilly, ISBN: 978-1-449-39041-9, Nov. 2010, 330 pages.
Japanese Application No. 2016-523825, Office Action dated Jul. 31, 2018, 4 pages (3 pages of the original document and 1 pages of the English translation).
U.S. Appl. No. 14/459,104, Advisory Action dated Aug. 28, 2018, 7 pages.
U.S. Appl. No. 14/459,104, Non-Final Office Action dated Jan. 11, 2019, 27 pages.
U.S. Appl. No. 15/692,929, Notice of Allowance dated Jan. 9, 2019, 10 pages.
European Application No. 11796398.3, Office Action dated Dec. 3, 2018, 4 pages.
European Application No. 13748986.0, Office Action dated Apr. 24, 2018, 8 pages.
European Application No. 11796398.3, Office Action dated Mar. 26, 2018, 5 pages.
U.S. Appl. No. 14/459,104, Final Office Action dated May 1, 2018, 28 pages.
Related Publications (1)
Number Date Country
20160197843 A1 Jul 2016 US
Provisional Applications (1)
Number Date Country
61597610 Feb 2012 US