Patent Application
20150326499
The present invention relates generally to the field of “work manager threads” (see definition, below, in Definitions sub-section of the Detailed Description section), and more particularly to managing multiple active work manager threads.
In application server environments, there are typically hundreds, or thousands, of concurrent users (“clients”). Each of these clients typically sends requests to the application server system. In order to handle all of the clients, a server computer typically has: (i) many resources (for example, processors, data storage devices); (ii) for each resource, a resource manager to manage the use of each resource in responding to these client requests; and (iii) work manager threads for communicating: (a) incoming client requests to the appropriate resource manager, and (b) request responses from each resource manager back toward the client who made the request. More specifically, the work manager threads: (i) take requests from the clients; (ii) apply the requests to the various resource managers of the server; and (iii) control and manage responses to the requests (for example, send applications, or results from running applications, back to the clients).
According to an aspect of the present invention, there is a method, system and computer program product that performs the following steps (not necessarily in the following order): (i) dividing a set of vouchers into a plurality of voucher subsets so that each voucher sub-set respectively corresponds to a resource manager of a plurality of resource managers, where each resource manager of the plurality of resource managers respectively corresponds to a resource in an application server; (ii) receiving a plurality of incoming pending requests with each incoming pending request respectively corresponding to a resource; (iii) temporarily removing a voucher from its voucher subset corresponding to a given resource manager upon each instance that a given pending incoming request is assigned to that given resource manager for responsive processing such that the given pending incoming request thereby becomes an assigned request; and (iv) determining which request of the plurality of pending incoming requests will be next assigned to a corresponding resource manager based, at least in part, upon which resource manager has the greatest number of vouchers in its respective subset of vouchers.
This Detailed Description section is divided into the following sub-sections: (i) The Hardware and Software Environment; (ii) Example Embodiment; (iii) Further Comments and/or Embodiments; and (iv) Definitions.
The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
An embodiment of a possible hardware and software environment for software and/or methods according to the present invention will now be described in detail with reference to the Figures.
Sub-system 11 may be a laptop computer, tablet computer, netbook computer, personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any programmable electronic device capable of communicating with the client sub-systems via one or more network(s) (not separately shown). Program 75 is a collection of machine readable instructions and/or data that is used to create, manage and control certain software functions that will be discussed in detail, below, in the Example Embodiment sub-section of this Detailed Description section.
Sub-system 11 is capable of communicating with other computer sub-systems via a network (not separately shown). The network(s) can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of the two, and can include wired, wireless, or fiber optic connections. In general, the network(s) can be any combination of connections and protocols that will support communications between server and client sub-systems.
Sub-system 11 is shown as a block diagram with many double arrows. These double arrows (no separate reference numerals) represent a communications fabric, which provides communications between various components of sub-system 11. This communications fabric can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, the communications fabric can be implemented, at least in part, with one or more buses.
The memory and the persistent storage are computer-readable storage media. In general, memory 33 can include any suitable volatile or non-volatile computer-readable storage media.
Program 75 is stored in the persistent storage for access and/or execution by one or more of the processors of the processor(s) set, usually through one or more memories of memory device 33. Persistent storage 34: (i) is at least more persistent than a signal in transit; (ii) stores the program (including its soft logic and/or data), on a tangible medium (such as magnetic or optical domains); and (iii) is substantially less persistent than permanent storage. Alternatively, data storage may be more persistent and/or permanent than the type of storage provided by persistent storage 34.
Program 75 may include both machine readable and performable instructions and/or substantive data (that is, the type of data stored in a database). In this particular embodiment, persistent storage 34 includes a magnetic hard disk drive. To name some possible variations, persistent storage 34 may include a solid state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage media that is capable of storing program instructions or digital information.
The media used by persistent storage 34 may also be removable. For example, a removable hard drive may be used for persistent storage 34. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer-readable storage medium that is also part of persistent storage 34.
Communications unit 30, in these examples, provides for communications with other data processing systems or devices external to sub-system 11. In these examples, communications unit 30 includes one or more network interface cards. Communications unit 30 may provide communications through the use of either or both physical and wireless communications links. Any software modules discussed herein may be downloaded to a persistent storage device (such as persistent storage device 34) through a communications unit (such as communications unit 30).
I/O interface set 32 allows for input and output of data with other devices that may be connected locally in data communication with server computer 20. For example, I/O interface set 32 provides a connection to external device set 22.
External device set 22 will typically include devices such as a keyboard, keypad, a touch screen, and/or some other suitable input device. External device set 214 can also include portable computer-readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, for example, program 75, can be stored on such portable computer-readable storage media. In these embodiments the relevant software may (or may not) be loaded, in whole or in part, onto persistent storage device 34 via I/O interface set 32. I/O interface set 32 also connects in data communication with display device 21. In this embodiment, external device set 22 includes a first resource 22a and a second resource 22b. Each resource 22a, 22b includes a resource manager (not separately shown), but the resource manager may not be wholly, or even partially, physically located in the resource that it respectively manages. In this example, the primary purpose of first server sub-system 11 is to serve the services of these two resources. These two resources may themselves include one or more computers, processors, memories and/or storage devices, as well as software and/or firmware. The Further Comments And/Or Embodiments sub-section of this Detailed Description section will discuss at least some of the types of resources that a server may serve.
Display device 21 provides a mechanism to display data to a user and may be, for example, a computer monitor or a smart phone display screen.
The programs described herein are identified based upon the application for which they are implemented in a specific embodiment of the invention. However, it should be appreciated that any particular program nomenclature herein is used merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature.
Processing begins at step S52, where voucher division module (“mod”) 77 takes a set of vouchers and divides them into subsets respectively corresponding to the resources 22a, 22b (see
Processing proceeds to step S54 where normal operations start, meaning that receive requests mod 79 receives pending incoming requests, with each pending incoming request being directed, in this example, to the first resource or the second resource. These pending incoming requests come to first server sub-system from other computers in the larger networked computer system of which server sub-system 11 forms a part. These pending incoming requests are accepted in the currently conventional manner in this embodiment, making extended discussion of this method step unnecessary.
Processing proceeds to Step S56 where request assignment and response mod 81: (i) converts the pending incoming requests into assigned requests by assigning them to a work manager thread for queuing at the respectively requested resource; (ii) decrements the appropriate voucher subset as each pending incoming request is converted to an assigned request; (iii) upon receiving a response from a resource, converts the assigned request into a completed request by removing the request from its previously assigned work manager thread; (iv) delivers the response of the completed request back to the requesting device (this part is conventional); and (v) increments the appropriate voucher subset as each assigned request is converted to a completed request. In performing these actions: (i) the work manager threads come from work manager thread pool sub-mod 81a; (ii) the incrementing and decrementing of vouchers is performed by voucher tracking sub-mod 81b; and (iii) the requests are assigned work manager threads, and later “un-assigned” work manager threads, by request assignment sub-mod 81c.
During the normal operations of step S56, if either resource is performing fast then it will tend to have a short queue of assigned requests, and will therefore tend to have, at any given time, relatively few vouchers temporarily decremented from its corresponding voucher subset. For similar reasons, if either resource is performing slowly then it will tend to have a long queue of assigned requests, and will therefore tend to have, at any given time, relatively numerous vouchers temporarily decremented from its corresponding voucher subset. The foregoing reasoning is what allows the number of vouchers remaining in a resource's voucher subset to be used as a sort of proxy for how fast the resource is performing. This can be useful information as will be seen, below, in the discussion of step S58.
Processing proceeds to step S58, which focuses on the assignment of a single work thread to a single pending incoming request. Step S58 is simply a part of the actions that take place numerous times as step S56 is ongoing, but, for purposes of understanding operation of this embodiment of the present invention, it is helpful to focus on a single assignment of a single pending incoming request to a work manager thread for queuing at the corresponding resource manager/resource. More specifically, request assignment sub-mod has many pending incoming requests, variously directed to both the first and second resources, to handle and assign at any given time. However, the machine logic of sub-mod 81c must determine which pending incoming request to next assign by assigning a work manager thread.
In this simple example, the choice is limited to the following alternatives: (i) the next assigned request will be the earliest received request directed to the first resource; or (ii) the next assigned request will be the earliest received request directed to the second resource. As between these two alternatives, sub-mod 81c will choose the request whose corresponding resource has the most vouchers left in its subset, and this is the request that will be converted from a pending incoming request into an assigned request by assigning a work manager thread from work manager thread pool sub-mod 81a. If there are no threads left in the thread pool, then the determination of which pending incoming request to assign, and the consequent assignment, will await the availability of a work manager thread that is returned to the pool by virtue of completion of a previously-assigned request.
Some variations will now be mentioned. Some embodiments of the present invention will have more (perhaps many more) than three resources. Also, some embodiments may consider, in determining which request to assign next, factors that go beyond: (i) when the request was received; and (ii) voucher subset numbers. For example, there may be a priority scheme associated with the requests that additionally takes into account other factors such as the identity of the requester.
Some embodiments of the present invention recognize that when an application has to access multiple “resource managers” (which will generally have a different level of performance relative to each other), the resources in the application server may become disproportionately consumed by relatively slow-performing resource manager(s) at the cost of faster performance which could have been realized if faster resource managers had been used. For example, an application, and its attendant service back to the client, might need to access multiple resource managers such as: (i) message provider resource manager; (ii) a first database resource manager for providing data for running the application; and (iii) a second database resource manager for serving application requests. In this example, all three of the resource managers will have varying degrees of response time depending on factors such as: (i) network speed to individual resource managers; (ii) load and capability of the resource managers; and (iii) system configuration hosting resource managers. The requests to relatively slow-performing resource manager(s) will generally take a longer period of time to receive a response. This becomes a limiting factor with the speed at which the client is served.
Some embodiments of the present invention recognize that these long response periods of slow performing resource managers effectively hold the VM (virtual machine) resources, such as work manager threads, “hostage.” As a result, these resources will not be available to application requests interacting with better performing resources. For this reason, in a production environment, a work manager thread pool can get filled up quickly, and the pool will deny threads to requests, including threads that do not involve connecting to a slow resource manager (such as a slow database). Some embodiments of the present invention recognize that a single application module normally runs on a single VM and any performance problem with backend resource or network will render the thread pool as a sort of “hostage.” This could also happen when there is a complex query that might take a long time to complete on a single database. Many requests that will not need database access will not be immediately allocated for service threads in the filled-up thread pool and these requests will wait as well.
Some embodiments of the present invention further recognize one, or more, of the following potential problems, shortcomings and/or opportunities for improvement with respect to the state of the art: (i) changing the connection pool settings to lower value will still have impact because threads will still be forced to wait as a consequence of the connection pool having a lower size; (ii) increasing the thread pool, or setting a control that allows growth, could hang the VM and cause an application outage; (iii) running the same application on the multiple VMs or clusters will also not usually be helpful because all requests eventually connect to the same slow resource managers and all VMs will face similar performance issues.
Some embodiments of the present invention may include one, or more, of the following features, characteristics and/or advantages: (i) clustering of work manager threads based on the resource that the work manager threads need to utilize; (ii) prioritizing of requests, while allocating work manager threads, based on performance of the resource the work needs to utilize; and/or (iii) clusters of work manager threads connecting to slow-performing resource are given lower priority at runtime based on successful completion of work relatively compared to the faster performing resources.
However, in system 200: (i) work manager threads of work manager thread pool 204 that require connection to a common, single resource are identified and grouped (or “clustered”) in clustered requests set 200; (ii) each cluster is provided with a set of “I owe you” (IOU) tickets; (iii) every request to a resource 210, 220 or 230, from a cluster in cluster set 202, will utilize single IOU Ticket (not separately shown); (iv) once the execution is successfully completed by the resource, and control returned to work manager thread pool 204 (also referred to as the “container”), back from the resource, the IOU ticket is returned to the container; (v) the priority is determined based on the number of tickets consumed; (vi) in this embodiment, slowest resource 210, 220 or 230 would be the resource that is holding the most IOU tickets (because when requests take a longer time to return from a resource then that resource will tend to have a greater number of IOU tickets); (vii) during normal operations (also called “runtime”), the identity of the slowest resource(s) may change; (viii) when fastest-to-slowest ranking of the resources 220, 230, 210 changes (based on changes in how many IOU each resource holds) then there will generally be a change in priority order in assigning work manager threads of container 204.
Clustering work manager threads will now be discussed in more detail. Every request to access a different resource 220, 230, 210 will be identified with application components mapped to resources as part of a resource reference in web.xml. During bootstrap of the application, or first access of the application component each application component is provided with an identification code (or “ID”) based on the identity of the resource 210,220,230 required by that application component. Requests to the application are then assigned work manager threads based on the availability of IOU tickets for the associated ID. The clustering of the requests into cluster set 202 helps to prioritize the work manager threads of container 204.
The setting of dynamic priority for clustered work threads of container 204 (also called the “container”) will now be discussed in more detail. The container will assign a number of “I Owe You” (IOU) tickets (or, simply, “tickets”) to each clustered group of work manager threads. Each clustered group of work manager threads respectively corresponds to a resource manager. The number of tickets for each cluster is assigned according to the proportion of incoming request for a given resource. For instance, assume that there are: (i) 100 available work manager threads in work manager thread pool 204, and (ii) three resource managers/resources that are accessed through these 100 available work manager threads. Under these assumptions, if the ratio of incoming requests to these resource managers (RM) is approximately 1:1:2 (in this embodiment, this ratio is determined based on manually configured weights) for three resource managers, then: (i) IOU for 1st RM=100/4=25; (ii) IOU for 2nd RM=100/4=25; and (iii) IOU for 3rd RM=100/4*2=50. A default IOU value for each can be implemented as a configurable priority for a resource manager by application deployers.
In the embodiment under discussion, the container deducts one IOU ticket (or, simply “IOU,” also sometimes herein called “voucher”) from assigned IOU tickets of the clustered group for every request to a corresponding resource manager just before the control is passed to a driver or a “ConnectionFactory.” Once the update is done successfully and control is returned back to the application server container, the IOU is returned back to the clustered group.
If a given resource is slow (this can happen for various reasons), the IOUs keep getting decremented and a slow resource will have few, if any, IOUs. The fastest resource will: (i) have highest number of associated IOUs; and (ii) is given highest priority while allocating a work manager thread. If all the IOU count for a resource drops to zero, then the requests to that resource have to wait until a prior request returns and thereby replenishes an IOU so that a new work manager thread can be allocated. As the number of IOUs varies over time according to resource performance, the priority order of the system's resources will vary accordingly, which will effectively, dynamically control the manner in which work manager threads are allocated on an ongoing basis.
As shown in
As shown in
Some embodiments of the present invention may include one, or more, of the following features, characteristics and/or advantages: (i) optimal usage of work manager threads to serve faster performing resources and non resource access requests; (ii) work manager threads are not “hostage” to a slow Database or slow network performance; (iii) can also have traditional thread priority manually set on the work manager pool, that will override the clustering of requests according to resources and resource performance based priority setting; (iv) clustering work manager threads based on the resource the thread needs to access as part of work; (v) resource performance based priority to allocate work manager thread; (vi) method and approach of using IOU tickets to measure the performance at runtime, to set priority accordingly and to allocate work manager threads.
Present invention: should not be taken as an absolute indication that the subject matter described by the term “present invention” is covered by either the claims as they are filed, or by the claims that may eventually issue after patent prosecution; while the term “present invention” is used to help the reader to get a general feel for which disclosures herein that are believed as maybe being new, this understanding, as indicated by use of the term “present invention,” is tentative and provisional and subject to change over the course of patent prosecution as relevant information is developed and as the claims are potentially amended.
Embodiment: see definition of “present invention” above—similar cautions apply to the term “embodiment.”
and/or: inclusive or; for example, A, B “and/or” C means that at least one of A or B or C is true and applicable.
Module/Sub-Module: any set of hardware, firmware and/or software that operatively works to do some kind of function, without regard to whether the module is: (i) in a single local proximity; (ii) distributed over a wide area; (iii) in a single proximity within a larger piece of software code; (iv) located within a single piece of software code; (v) located in a single storage device, memory or medium; (vi) mechanically connected; (vii) electrically connected; and/or (viii) connected in data communication.
Software storage device: any device (or set of devices) capable of storing computer code in a manner less transient than a signal in transit.
Tangible medium software storage device: any software storage device (see Definition, above) that stores the computer code in and/or on a tangible medium.
Non-transitory software storage device: any software storage device (see Definition, above) that stores the computer code in a non-transitory manner.
Computer: any device with significant data processing and/or machine readable instruction reading capabilities including, but not limited to: desktop computers, mainframe computers, laptop computers, field-programmable gate array (fpga) based devices, smart phones, personal digital assistants (PDAs), body-mounted or inserted computers, embedded device style computers, application-specific integrated circuit (ASIC) based devices.
