Patent Application
20100257009
The invention concerns service orientated computer system comprised of services, such as web services, connected by an event-driven middleware. In particular, the invention concerns adaptive and self-managing systems that can adapt their behaviour based on components that monitor the behaviour of the system and provide feedback. Aspects of the invention include a computer system, method, and software.
As Service Oriented Architecture (SOA) becomes more widely adopted in large software systems, the typical SOA environment has become more complex. Management of these increasingly complex environments is exacerbated by crosscutting components and services as well as overlapping SOA environments with service providers beyond the administrator's control. While some human inspection or administration tools can and should be provided, it is unrealistic to expect that all configurations and management can be effectively handled manually. Being fully dependent on manual controls would void the improvements in timeliness and adaptivity gained with an increased level of automation. Consequently, incorporating adaptive and self-managing capabilities into services [4,15] is attracting considerable attention as a means to respond to both the functional and environmental changes that occur after service deployment.
In principle, a system exhibiting adaptive and self-managing capabilities [7,11,12] consists of two parts: (1) a base system that implements the business logic and provides concrete functionalities; and (2) a set of controls that comprise control components for constantly monitoring the system, analyzing the situation and deciding on the actions to affect the system's behaviour. When the base system is composed of services in a SOA, the addition of these control components results in adaptive and self-managing service-oriented systems.
In practice, individual control components are dedicated to a specific quality attribute, such as load balancing for performance and scalability or failover for reliability. These are normally constructed independently.
As an example of a SOA, consider the loan brokering application in [6], where a customer submits requests for a loan quote to a loan broker. The loan broker checks the credit worthiness of a customer using a credit agency. The request is then routed to appropriate banks who each give a quote, and the lowest quote is returned to the customer. This application has been deployed (see left of
Suppose the responsiveness to requests of the credit agency is in question, and the administrator wants to allow a graceful fail over to an alternative credit agency should the primary agency fail. One solution is to insert an additional switching component between the loan broker and the credit agency that can reroute traffic to an alternative credit agency (see
In addition to ensuring the robustness of the credit agency, the administrator also wishes to prevent the loan broker from becoming saturated with requests. As shown in the right of
Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is solely for the purpose of providing a context for the present invention. It is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention as it existed before the priority date of each claim of this application.
In a first aspect the invention provides a service orientated computer system comprised of services connected by an event-driven message based middleware, the middleware further comprising:
Communication between all components is performed by sending and receiving messages. The controls define the logic for the sending of these messages that in turn change the behaviour of the system. The controls provide a layer of abstraction that allows loose coupling between the controls and components. This avoids the disadvantages associated with hard coding of logic within components.
The model of the quality control may be comprised of states and transitions having associated action code that when executed by the process engine changes the behaviour of the one or more associated components.
One or more components may have control parameters. The message may alter a control parameter of a component. This may be done by altering a configuration file associated with the component.
The set of controls may further comprise:
The coordinating controls may receive status messages from each of the associated quality controls and coordinates dependencies between the associated quality controls based on the received status messages and executable process model. The status messages may be received by an aggregation sensor component of the coordination control.
The coordination control may operate to change the behaviour of the system by sending a message to a component associated with a quality control that is in turn associated with the coordination control in order to change the behaviour of the one or more associated components.
A message sent from a coordination control to a component has precedence over a message sent from a quality control to the same component.
Coordination and quality controls can be updated, deployed or undeployed at runtime
A first quality control may be directed to overloading as the quality of service metric. A second quality control may be directed to failover as the quality of service metric.
The coordinating control may be associated with the first and second quality control. The coordinating control may receive the status message from the second quality control indicating that overload has occurred. The coordinating control coordinates the dependency by sending a message to a component associated with the overload control to cause the slowing down the acceptance rate of new requests to a service until the failover operations complete.
Changing the behaviour of the one or more associated components may be performed at runtime.
The change in the behaviour of one or more components may result in one or more of:
The components may be sensors, routers, throttlers and/or multicasters.
The models are in business process modelling (BPM) language.
Each coordinating control represents a unique cross cutting concern of the associated quality controls.
The services are spread across multiple organization boundaries. The components, controls and processing engine are deployed at service boundary rather than within services.
The event-driven middleware may include an Enterprise Service Bus (ESB). The services may be hosted on a distributed computer network or on Internet. The middleware may also be distributed to any host. Each host may be a web server or a platform that hosts services, such as Cloud computing infrastructure.
Services may be web based, and may be accessed by a unique identity (called endpoint).
In a further aspect the invention provides a method of automatically changing the behaviour a service orientated computer system comprised of services connected by an event-driven message based middleware to meet a quality of service, the method comprising executing at run time multiple quality controls, each quality control directed to one or more quality of service metrics and associated with one or more components that function to sense messages and effect delivery of messages within the middleware and between services, by for each quality control:
The method may further comprise the steps of the quality controls sending status messages and executing at run time coordinating controls that automatically coordinate dependencies between two or more associated quality controls, by for each coordinating control:
In yet a further aspect the invention provides software, that is computer executable instructions stored on a computer readable medium, that when installed causes a computer system to operate in accordance with the method described above.
The invention includes the realisation that control components need to be coordinated at runtime to resolve their dependencies that are incurred by cross-cutting concerns. For example, the operation to switch to a backup service may come at a cost of performance by degrading the throughput over a given period of time.
While component-based development helps to modularize and encapsulate adaptive and self-managing computation, there is still tight logical coupling and interdependencies between control components. Examples of such tight coupling include systems where the monitoring, analysis and configuration control components explicitly invoke one another without the intervening layer of abstraction. The aim of the invention is to abstract the controls and their dependencies so that their actual implementation is separated from the control logic.
An non-limiting example of the invention will now be described with reference to the accompanying drawings, in which:
a) is a simplified representation of the architecture of this example, and
b) shows the same architecture when customized for specific controls.
a) schematically shows the overload and failover control when coordinated, and
An example of the invention and its evaluation will now be described.
In this example, to support ease of service evolution, the composition of control components with existing services is transparent to business logic. That is, the control logic does not require modification of the original service operation.
However, in a component based implementation of the controls, flexibility is reduced as the control logic would be embedded in the components. For example, if the criteria to trigger the failover switch is changed, a rewrite of the basic switching and test message components would be required to coordinate their logic.
Moreover, introducing control components also creates dependencies between the business and management flows. For example, the loan broker needs to be aware of the switching component in order to send messages correctly to the switch and not to the credit agency. As more management controls are added, the introduced dependencies both obscure the original business flow as well as reduce the system's flexibility to changes in both flow types.
Accordingly, the following example meets the following architectural requirements:
Conceptually, the architecture of this example has five layers. The left of
The architecture framework provides a modelling based approach towards coordinating multiple controls in service-oriented systems. Adaptive and self-managing controls follow logic that transitions the system from one state into another in response to events dynamically generated at runtime. In addition, the logic represented by the models needs to be executed as well. Given this consideration, we use process models as the tool to present and execute controls. The use of process models is motivated by their rich semantics, alignment with business goals, integration with Web services and tool support for visual design. The process models can also be executed by process engines with middleware integration features, such as the Oracle BPM engine and JBoss jBPM. Here, the term control model is used to refer to a process model designed and executed in a similar way to the JBoss jBPM technology [8].
Referring to
The layer below the control models comprises handlers 22 that encapsulate the action code. Upon entering/leaving a state or before/after triggering a transition, the process engine checks and executes the handler for actions. Our architecture has default implementations for two handlers, AnalysisHandler and ConfigInitialisingHandler, which are responsible for managing dependencies between control models, and checking a data cache for individual control components respectively. Their usage is addressed further below. The combination of these two layers focus on architecture requirement (1).
The component layer 24 aims to address architecture requirement (2). The realization of controls depends on monitoring and actuating components, such as software sensors to collect status data to feed into the model, and effectors to execute actions. These components are placed into the component layer 24 to separate the control implementation from the business logic. The ApplicationManager is responsible for initializing the component instances. As control components participate in service oriented applications, each component has a unique endpoint as its identifier, so that messages can be received from and sent to individual components by service bus middleware.
The control layer 26 aims to fulfil the architecture requirement that controls can be composed, modified and deployed at runtime. Control components are deployed as the unit of the ControlDeploymentPackage. Each control has a default ControlDeploymentPackage generated by the framework. It contains methods to access all the components involved in a control. Each ControlDeploymentPackage. uses the ControlDeployer to deploy its components. The ControlDeployer is responsible for (un)deploying components, creating component descriptors and setting the implementation class for each component. This separated deployment of the component instance from its actual implementation further enhances the customization of the adaptive controls. This is because the modification of the implementation does not impact the control models nor the deployment structure, and the implementation can be updated at any time. Once the deployment is finished, an event is broadcasted to other controls about the availability of the new control components.
The bottom layer is the middleware platform 28. In this example it is a specific Java ESB—Mule [14]. Mule platform specific mechanisms are used to devise utilities such as concurrency configuration and event multicasting.
In summary, the architecture supports visual and declarative design of adaptive control logic. Controls are modelled as executable process models. These models are executed by a dedicated process engine, which is seamlessly integrated with the middleware. Hence these models can interact with service applications hosted by middleware, receiving and sending messages to realize the control logic. In addition, the architecture supports dynamic update and deployment of controls. As a result, the overall architecture is loosely coupled between business logic and adaptive controls. In the following sections, we further discuss the coordination of multiple controls.
A challenging issue to solve in this architecture is control dependencies occurring at runtime. Controls designed and deployed independently may involve cross-cutting concerns. For example,
Our architecture can address this issue by the techniques of modelling such concerns as coordination controls. The components coordinated are the sensors and effectors from individual controls. The dependencies are declared in a control model representing the cross-cutting concern; the specific resolution strategy, be it by heuristic hints or some form of machine learning, consists of implementation specific handlers attached to the process nodes. This leverages on the architecture framework presented, building on the basic idea of sensors, effectors and coordinating components. In the following subsections, we discuss the technical details of achieving such coordination among multiple controls.
In our approach, the dependencies of controls are declared by developers in a dedicated coordination control, as discussed at the top layer 22 of the architecture. The developer registers controls with dependencies using an AnalysisHandler that belongs to the handler layer. This coordination control is modelled and deployed the same way as ordinary QoS controls. When it is deployed, another handler, a ConfigInitialisingHandler checks if an instance of the registered controls exist. After the ConfigInitialisingHandler checks the controls and their dependencies, the AnalysisHandler can retrieve the configuration of individual components in one control. A configuration is part of the control layer. It is an abstraction of what the component does. A configuration contains information about interfaces and properties of a component. Through the configuration, the coordination control and the AnalysisHandler can access data that the component contains, invoke its interface on behalf of the coordination and change property values in order to change the control parameters. Sample code is shown in
Using this approach, individual controls are not aware of other controls nor their dependencies. They are transparently managed by coordinating controls. This approach also benefits from the architecture in that a coordination control can flexibly be composed by existing components, which allows quick composition and prototyping of alternative options for adaptive and self-management strategies. An example of composing coordination controls is given below. In addition, coordination controls can be updated, deployed or undeployed at runtime. This equips developers with the flexibility to trial-and-test different designs.
The deployment of controls takes two steps. First, the control design models in the format of an XML file are deployed to the process engine using an IDE shipped with the process engine. Any action code is attached to the states or transitions in this model. Second, the unit of deployment ControlDeploymentPackage in our architecture framework is generated, with a mapping to the component implementation record. Following this, the ControlDeploymentPackage invokes the deploy( ) method of ControlDeployer to deploy itself, creating instances of participating components using their descriptors.
Besides the above functionality of deployment, the architecture requires the ability to intercept incoming requests, and modify outgoing messages. This is also achieved through the control deployment. The control deployment automatically generates intercepting components as a proxy to the intercepted components. The intercepting component takes the identity—the unique endpoint of the intercepted component and forwards requests to and replies from the intercepted components. This feature enables the control composition by redirecting messages to/from any other component transparently to the intercepted components.
The computing overhead incurred by this architecture should be optimized. By nature of this service oriented architecture, the optimization problem falls into the category of minimizing messaging overhead. Research on messaging oriented middleware and Web services has demonstrated that the communication rate and payload of the messages have a significant impact on the overall performance and scalability of SOAs [10]. Hence our optimization focuses on reducing the number of messages and their payload with regards to sending collected data among control components including sensors, data analysers and effectors. Rather than wrapping data as a SOAP attachment, data collected by sensors are stored in a distributed cache. Whenever necessary, a distributed cache is attached with the control components such as software sensors. In this case, we select an open source distributed cache framework—Ehcache [13]. The performance and scalability of Ehcache has proven to satisfy large scale distributed systems [5]. In order to correlate data collected from different sensors, a sensor aggregation component is created at deployment time. Here, a default time-based correlation is implemented in the aggregator. The only limitation with using a distributed cache is that the data transition is separated from the web service messages and it is specific to the distributed cache framework.
A more detailed example will now be described using the loan brokering services discussed in the Background Art section. In addition to verifying the feasibility of our architecture in implementing a practical set of services, we also highlight the flexibility of our architecture for trial-and-test deployments by providing two options to coordinate the failover and overload controls, subsequently referred to as simple and auction-based coordination. Here, we discuss the specifics of the individual components making up our implementation, as well as two coordination heuristics employed.
The overload control implements the classic token-bucket algorithm for admission control. It consists of a Token Bucket Sensor 40, a Throttling Component 42, a Throughput Sensor 44 and a Coordination Component 46, as shown in
where α is a tuning parameter to adjust the component weight.
The failover control shown in
To coordinate these controls, our general approach is to let the overload control throttle the workload when failover takes place. In our implementation, two options are provided to realise this approach.
Our implementation of the architecture is deployed as shown in
The simple coordination control tunes the concurrency level of processing new, incoming requests in the middleware. The tuning is based on the number of messages yet to be resent by the failover control. This control is easier to implement, but has limitations when producing the optimal concurrency levels for a large set of services.
In the auction-based control, requests being resent by the failover control and new incoming requests at the overload control bid for tokens. Tokens are dynamically allocated to requests both from failover and overload controls. Only requests with a token can be processed, otherwise there is a wait for the next available token. In general, the auction-based control incurs more overhead in communication as a bid is multicast. However, the auction-based control is more practical and suitable when it is nontrivial to tune the concurrency level of the middleware.
Both options reuse the failover and overload controls, and it should be noted that the control model is identical for both options. The difference is in the way each of them process status data and the actions taken. This is reflected by the different options having different handlers attached to the appropriate control model nodes.
In this example, process modelling tools and middleware mechanisms are used to customize the general architecture to a specific implementation. As mechanisms from middleware (such as interceptors) and modelling features from the process engine (such as handlers) are commonly supported, other process modelling tools and service bus middleware can be applied to the framework. The only condition however, is that the process engine should be able to communicate with the middleware. For example, Mule provides a common interface for process engines to access its features [14]. We could have used an Oracle BPM implementation of the interface instead of JBoss jBPM, without any change to other implemented components. This illustrates the generic nature of our architectural solution.
In use the quality control 70 and 72 and the coordinating control 74 operate according to the method shown in
The quality controls 70 and 72 receive input messages from one or more associated components (i.e. sensor aggregator), step 100. Then the quality controls 70 and 72 execute their respective process models based on the received messages to send messages to the one or more associated components (i.e. throttling component and effecting router) to change the behaviour of the components, step 102.
Optionally, the quality controls 70 and 72 send status messages, step 104. The coordinating control 74 receives status messages from the one or more associated quality controls 70 and 72, step 106. Then the coordinating control 74 executes its process model based on the received status messages to send messages to the one or more associated components (i.e. effecting router associated with the coordinating control and then inturn the sensor aggregators associated with the quality controls 70 and 72), step 108.
Each option of the coordination control is measured and the results are compared to identify key performance factors.
We deploy the loan brokering services as shown in
We also develop a simple workload generator which injects a number of requests into the system under test with a bounded random time between request arrivals. For example, the interval [75,200] means the request arrival time bound is between 75 to 200 milliseconds. In order to observe performance, a simple console showing charts of metrics was developed (see
The testing environment includes two identical Windows XP machines with 2.4 GHz Dual Xeon Processors, one hosting loan broker services, credit agencies and adaptive controls and the other hosting five bank services which are identical to simplify implementation. The workload generated is a set of 500 requests with the interval [75,225]. If the overload control is enabled, the throttling component controls W, the number of concurrent requests that can be processed. W is set to 100 initially.
We test four scenarios: (1) only the overload control is enabled; (2) only the fail over control is enabled; (3) simple coordination; and (4) auction-based coordination. Obviously in (3) and (4) both failover and overload controls are enabled. The same environment configurations are used for each test.
Coordination can be enabled or disabled. The results of the simple coordination are shown in
It is worth noting that our focus directly above is not on studying and evaluating individual coordination controls but rather on demonstrating the practical usage of the architecture to compose them.
It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the invention as shown in the specific embodiments without departing from the scope of the invention as broadly described.
For example, the invention could be implemented in a cloud computing environment. The emerging trend is deploying services on Cloud so that software (i.e. Salesforce.com for customer-relationship-management) and platforms (i.e. Amazon Elastic Computing) can be encapsulated as services to support pervasive and on-demand usage.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.
