Patent Application
20160092297
The present invention relates to ecommerce, and more particularly, relates to a system and method for providing API-related services in a networked environment.
Application Programming Interfaces (APIs) are becoming increasingly popular for delivering services over the internet. APIs link businesses and developers with information and content around which they can grow their applications and their businesses. While it is becoming easier to expose an API to a potential audience, it is becoming more difficult to manage and monitor the operational aspects of the API, especially as the number of services accessed grows and the service systems become more complex. Services can employ anywhere from a few to hundreds of servers, and it is crucial for API performance that the API be able to locate a healthy server to obtain a response.
API gateways employ various load balancing schemes to determine where to send a request. Some, such as discovery services, provides some indication of which of a service′ servers are capable of receiving and processing a request. However, discovery services requires that the service register their servers with the gateway and maintain their registrations as their configuration changes. A method for discerning the health of a potential service and bringing failed systems back online without requiring maintenance on the part of the service is desired. The invention described herein offers a solution to that problem and offers other benefits over the prior art.
A self-contained API gateway is disclosed. In one embodiment, the gateway comprises all of the components required to provide its core features without relying out outside systems. All of the code and information that the system needs to process requests are available within the running system. It is not dependent on external features, such as databases, or other services to provide its core features, although integration with external systems is possible. The gateway may be comprised of a gateway developer portal, a code base stored in memory which when executed by the processer performs the features of the system by efficiently receiving and routing requests and responses from clients to services and back. A set of dynamic scripts provide all the details required for handling requests and responses, and an embedded server allows the system to be runnable with all features without first installing a web server.
In accordance with another aspect of the disclosure, efficient and accurate routing is provided using a wide variety of load balancing techniques, including a static list of available servers, a discovery service, a domain name, HTTP method-type routing, region-based routing, and performance-based routing. A modified circuit-breaking feature provides a failure policy that benefits an ailing server as it recovers from failure. Those of ordinary skill in the art will recognize that this feature offers significant improvements over the prior art and offers other benefits which will become apparent throughout the disclosure.
The presently disclosed embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments of the inventive concept are shown. The invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.
It will be appreciated by those of ordinary skill in the art that the described embodiments may be realized as a method, system, computer program product, or a combination of the foregoing. Accordingly, the disclosed embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including resident software, microcode, etc.), or an embodiment combining software and hardware aspects that may generally be referred to herein as a “system.” The embodiments may also take the form of a computer program product on a non-transitory computer-readable medium having computer-usable program code embodied in the medium.
Any suitable non-transitory computer-usable or computer-readable storage medium may be utilized. The computer-usable or computer-readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semi-conductor system, apparatus, or device, but does not include a signal per se. Other examples include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, or a magnetic storage device. In the context of this document, a non-transitory computer-usable or computer-readable storage medium may be any non-transitory medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations of the present invention may be written in any type of programming language capable of implementing its features. The program code may be written in sets of instructions, such as program modules that include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement the features described. The program code may execute on one or more processors that may be remote from each other, or as a stand-alone software package. One of skill in the art will recognize that computer program code contains instructions that may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the features of the invention described below. Computer-executable instructions may be stored in a non-transitory computer-readable memory, as described above, that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the features described herein. The computer program instructions may be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified herein described in text or as specified in a flowchart and/or block diagram block or blocks. Alternatively, computer program implemented steps or acts may be combined with operator or human implemented steps or acts in order to carry out an embodiment of the invention.
The present disclosure describes various embodiments of a system and methods for providing API-related services over a network. In one embodiment, instances of an API gateway is an entirely self-contained unit. All of the code and information that the system needs to process requests are available within the running system. It is not dependent on external features, such as databases, or other services to provide its core features, although integration with external systems is possible. Referring to
The dynamic scripts 104, 106 and 108 provide the system with the data it needs to process requests. A properties file 104 provides general configuration settings, such as the location of data. A key definition file 106 contains security and rate limit information including information on the user(s), public or confidential status (confidential keys may require a secret, which is optional with a public key), authorized services, custom attributes which may be sent to the service as headers any time a request is made with the key, rate limits and accessible environments. An API definition file 108 includes all the information related to the service and the API that the system needs. Included are the service name and purpose, expected traffic, capacity constraints, means of access, architecture details, whether card holder data is included, whether personal identifiable information is included, and security information. In addition, technical details are included such as service identification details, the origin URL, and environment. Data that would also be included in the API definition includes details on capacity, failure/failover protocol and other policies of the services system. Services may be internal or external. One of ordinary skill in the art will recognize that these files may contain other information, including any information that are API/service specific that would be required or helpful in providing API services to the origin service.
Referring again to
The API Gateway further comprises pre-routing, routing and post-routing logic 114, which will be discussed further, below. These modules provide the logic required to access the services registered with the gateway. Additional peripheral tools may be used by the API Gateway, including logging 116, which tracks all requests and responses and their errors, performance metrics, etc. for monitoring and analysis via an API Gateway dashboard 118.
In at least one embodiment, both the key (client) 202 and the origin service 210 have rate limits. Rate limits for the client 202 and origin service 210 may be configurable values in the key 106 and API definition files 108, respectively. In the rate limiting filter 306, the gateway system checks the key limit to ensure that the client has not exceeded their limit set for a particular period of time. The service may specify its own rate limit in its API definition file 108. If the request passes these checks and no errors are present, the system will add metadata, such as a request identifier, to the request header, and send the request to the service cache 308. If the cache can respond to the request, the response is provided from the cache 308. If not, routing logic 310 is applied. As will be described below, different embodiments may use one or more various routing, or load balancing, schemes to determine which particular service server to call. The request is sent to the origin service, and a response is returned.
The origin service 210 generates a response and returns it to the gateway. The gateway post-routing filter 312 manipulates the header to provide valuable information such as the request ID, and caches the result if a cache indicator has been received. The post-routing filters 312 may also block certain headers or strip out information that the client doesn't require. For example, some services may respond with cache indicators in their headers. The gateway 100 will use that information to update its cache, but will strip the information from the header when returning the response to the client.
Error handling and logging 116 are performed continually throughout the transaction. Messages (requests and responses), performance measures, errors, and more are continually sent to an event aggregator throughout the process. Once the response to the client is fully generated and sent, a record is written to the logging system for the entire transaction. This data is fed to the gateway dashboard 118 for monitoring and analysis.
The API gateway 100 may use one or more methods of load balancing in its routing logic 310 to determine where to send the request 204 to the target origin service 210. The methods described here are provided by way of example and not limitation. In some embodiments, the system may use a static list of available servers, a discovery service, a domain name, HTTP method-type routing, region-based routing, performance-based routing, and circuit breaking, alone or in some combination.
A static list of servers available for routing a request 204 to an origin service 210 may be provided. The static list has the advantage of being quickly and easily read, but must be updated as server status changes. The gateway may use a discovery service to register all available servers. A discovery service queries the registered services periodically to determine which servers are available, and updates the status. A static list of available servers may then be pulled periodically to replace the previous static list.
Routing 310 may also be performed by directing the message to a domain name. Additionally, messages might be directed to a particular server or set of servers depending on their HTTP method-type. For example, all GET methods might go to one server, but POST messages might go to another. Region-based (choosing the closest servers), performance-based (choosing the servers with the best historical response rates), or service-based routing may be used.
In addition to load balancing features, routing 310 filters can also include applying a failure policy, a modified “circuit-breaking” algorithm, to its service servers. In an API gateway 100, the failure policy may be applied via a configuration setting in the API definition file 108. If the system discovers that the origin service 210 is having problems handling a request, it prevents the client 202 from making calls to the origin service 210. Instead of repeatedly calling the service 210, it does a ‘fail-fast’ on subsequent calls, so the client 202 may decide to wait to make the call, or forgo it altogether. This scheme protects the origin service, allowing it to recover and restart, and aids the client by not making him wait for a response that will eventually fail.
The ramp up state 408 provides the system with a process that allows the origin service some time to recover, and for the API gateway to reestablish processing with the affected server. If the breaker is half open 406, and the API definition file 108 specified stages that the service 210 must go through once the system is in a normal state, it does this rather than going to directly to closed. Ramp up 408 limits the number of calls that can be made to the service for a certain period of time, for example, 10 calls per minute for 5 minutes. Anything above that is rejected. The process loops through the stages, recording the successes and failures in order to determine whether the origin service 210 is healthy again and can receive calls successfully. If yes, the circuit may be closed 402 and receive calls. An example of the circuit breaker definition, policy and ramp up stages configured in the API definition file are provided in Table 1.
The example failure policy specifies ramp up policy stages that provide for a rate limit of 5 calls per second for 10 seconds, and then 10 calls per second until the breaker is closed. One of ordinary skill in the art will recognize that many configurations of a ramp up policy are possible.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. For example, the present techniques can be implemented in any kind of system that includes a hard disk drive. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
This application claims the benefit of U.S. Provisional Patent Application No. 62/057,118, filed on 29 Sep. 2014 and titled “Dispatch Gateway for E-Commerce Platform System and Method,” which is incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62057118 | Sep 2014 | US |
