Patent Application
20090300626
The embodiments herein relate to scheduling processes in computing systems with multiple levels of determinism.
Real-time computing systems are used in avionics, aeronautics, telecommunications, automobiles, medicine, finance, industrial control, and various military applications. Some real-time computing systems prioritize various aspects of the execution of more important processes over less important processes. In other types of real-time computing systems, specific processes (i.e., instances of an executable program) may need to be able to execute a certain number of times per quantum, and may need to be able to complete execution within a specific amount of time from when they were started. Failure to meet these constraints could result in significant inconvenience to users, financial loss, substantial damage to property, or even loss of life.
The former type of system can be referred to as a non-deterministic real-time system, wherein processes will share the computing system's resources, but exactly when a process will execute may or may not be known. The latter type of system can be referred to as a deterministic real-time system, wherein the system exhibits the ability to execute certain processes within a non-varying or minimally-varying time frame. In between these extremes are classes of systems in which certain processes may have to operate deterministically, but other processes require a lesser degree of determinism or are non-deterministic.
In order to guarantee the timely execution of processes within reasonable bounds, a schedule may be created to execute on a real time system. A software tool or a human may create the schedule, or a human may use a software tool to assist in creating the schedule. A typical schedule divides a timeline into repeating frames of a fixed length. Each frame can be further divided in a number of timeslots. A number of sequential frames may be referred to as a superframe. For example, a frame may be one second in length, and divided into fifty timeslots of twenty milliseconds each, while five consecutive frames may be referred to as a superframe and will be five seconds in length. Naturally, the lengths of frames, timeslots, and superframes are based on the specific needs of the computing system in question, and different lengths of time for each may be used.
When processes execute according to a schedule, any changes made to one of those processes may require re-validation of the entire schedule. For example, if a schedule includes three deterministic processes, process P1, process P2 and process P3, a change to any of these processes may require modification of the schedule or creation of a new schedule, because the changed process or processes may have different requirements of the computing system's resources. Any such modification of the schedule or creation of a new schedule will likely require re-testing the execution of all three processes for correctness, performance, and stability. Thus, each modification to a process can result in additional test and integration cycles for the entire computing system, and each of these cycles may further delay the completion of the computing system while incurring further cost. A goal of software engineering is to develop processes in a modular fashion, so that there are few interdependencies between processes. However, this goal can be difficult and time consuming to achieve using an integrated schedule. Therefore, it is desirable to be able to schedule real-time processes with multiple levels of determinism in such a way that their execution is substantially decoupled from one another.
A method and system of scheduling real-time processes with multiple levels of determinism is introduced. Processes are developed on separate sub-schedules and these sub-schedules are integrated into a common schedule. By doing so, each set of processes can be developed and tested separately, and integrated with minimal impact on one another. As a result, software development cycles may be accelerated while system performance requirements can still be met.
It should be understood, however, that this and other arrangements and processes described herein are set forth for purposes of example only. Other arrangements and elements (e.g., machines, interfaces, functions, orders of elements, etc.) can be added or used, and some elements may be omitted altogether. Further, as in most computer architectures, those skilled in the art will appreciate that many of the elements described herein are functional entities that may be implemented as discrete components or in conjunction with other components, in any suitable combination and location. For example, bus 120 may consist of multiple independent busses so that each CPU, 112, 114, 116, can access memory 122 without contending for a CPU bus with the other CPUs. Furthermore, computing system 100 might comprise multiple memories, and each unit of memory may comprise semiconductor memory, magnetic memory, optical memory, acoustic memory, biological memory, or any combination of these memory technologies, or any other memory technology used in conjunction with computational devices.
Modern computing systems are preferably designed around the concept of a process being the unit of execution. A computer program typically consists of a set of CPU-executable instructions. A process is an instance of execution of at least some of the instructions of a computer program. A computer program may comprise multiple processes. For example, when initially executed, a world-wide web server computer program may launch multiple processes. Each process may include the full world-wide web server functionality, and therefore would be capable of receiving world-wide web requests, processing these requests, and transmitting the appropriate world-wide web content back to the sender. By executing multiple world-wide web server processes in parallel, the computing system can serve multiple world-wide web requests in parallel. In some computing systems, the term “task” is used analogously to the term “process.” The term “thread” may also be used analogously with the term “process,” but some computing systems distinguish between processes and threads. For purposes of the methods and systems described herein, the term “process” shall be used interchangeably with the terms of art “task” and “thread.”
Processes within one process set may need to communicate with processes within the same process set or another process set. For example, the processes of process set 212 are communicatively coupled 224 with the processes of process set 214. Similarly, the processes of process set 232 are communicatively coupled 244 with the processes of process set 234. Communication between processes is typically called inter-process communication (IPC). Modern computing systems support a multitude of types of IPC, including shared memory, message queues, pipes, sockets, and remote procedure calls, to name just a few. IPC may take place between processes executing on the same processor (e.g., communication paths 224 and 244), or between processes executing on different processors (e.g., communication path 246). The data transmitted from one process to others in an IPC transaction may be temporarily stored in memory 122 or may be transmitted directly from one CPU to another CPU. IPC transactions may use one or more busses in a computing system or a similar means of communication. Furthermore, each discrete unit of IPC, such as a unit of data transmitted from one process to another, may need to be transmitted at a particular frequency and may have an associated delay constraint. For example, a process within partition A 216 may need to transmit a unit of data to a process within partition D 222 every 100 milliseconds (ms) such that the process within partition D 222 receives the unit of data in no more than 50 ms. Typically, these processes and their associated communication events are combined into a single schedule, where this schedule determines when each process will execute, for how long, and when a process will transmit a unit of data to one or more other processes.
In
Ultimately, in order for computing system 200 to be completed, process sets 212, 214, 232, 234 may be integrated with each other so that computing system 200 can be tested as a whole. As discussed above, some processes within each process set may be deterministic. Thus, these processes may need to execute at a particular frequency, and when they execute will utilize computing system resources for a particular length of time. This requires that the designers of computing system 200 create, either manually or via the use of software tools, an execution schedule that incorporates all processes of process sets 212, 214, 232, 234. The schedule must meet any execution constraints, such as frequency, duration and IPC requirements, associated with each process. Also discussed above, the processes within process sets 212, 214, 232, 234 may need to communicate with one another. Thus, the schedule must meet any communications requirements of the processes of process sets 212, 214, 232, 234, including associated communication delay and communication jitter constraints.
Communication delay is the time between when a unit of data is produced by a first process and later consumed by a second process. A goal in schedule creation is to minimize communication latency to the maximum extent possible. Communication jitter is the variation of communication delay for a given unit of data as the schedule is repeated. For example, as frames in the schedule repeat, a unit of data may be produced by a first process and later consumed by a second process many times. Due to various considerations, such as CPU, bus, and memory utilization to name a few, the communication delay associated with the processing of the unit of data may fluctuate from frame to frame. Communication jitter is a measure of these seemingly “random” variations in delay. Like with communication delay, a goal in schedule creation is to minimize communication jitter to the maximum extent possible.
Communication between any two or more processes can be presented in many forms; however, a flexible method of representing such communication is by using a simple matrix. IPC between N processes may be represented in an N×N matrix, wherein each cell of the matrix represents data that is transmitted from one process to another process. An exemplary communication matrix 300 is illustrated in
Matrix 300 in
Matrix 350 in
It should be understood that both matrix 300 and matrix 350 are simple examples of ways in which IPC requirements and process execution requirements can be represented, respectively. In a real time computing system, different means for representing IPC requirements and process execution requirements can be used, and additional factors or constraints can be included in these representations.
The combination of these IPC requirements and process execution requirements further complicates the task integrating process sets 212, 214, 232, 234. For example, assume that during integration of process sets 212, 214, 232, 234 a defect (or “bug”) is discovered in a process P1 within partition A 216. Not only will this defect have to be fixed, but the process P1 will have to be retested. Furthermore, since process P1 needs to be able to integrate with potentially many other processes, some of which are deterministic, the execution schedule for process sets 212, 214, 232, 234 may need to be re-designed. Additionally, if process P1 requires communication with any other process, the schedule for process sets 212, 214, 232, 234 may need to be re-designed to incorporate this IPC, as well as any associated delay constraint.
Thus, a series of relatively minor defects in a small number of processes can cause the schedule for process sets 212, 214, 232, 234 to have to be re-designed and retested several times. Each integration test cycle may take weeks of calendar time, and the integration test process may require a large team of engineers. Thus, these minor defects add many person-weeks of additional expense, perhaps even hundreds of thousands of dollars or more, to the cost of delivering a computing system and its associated software. Consequently, any means for reducing the cost of designing, developing or testing such a system could have enormous financial advantages.
In particular, it is advantageous for the design, development, and testing of the process sets to be as loosely coupled as possible. This way, changes to one process set will have minimal or no impact on changes to another process set. In such a loosely coupled system, if a defect is found in a process P1 during some phase of design, development or testing of the computing system's software, the software engineers can fix the defect without always having to redesign the entire schedule for all process sets. This means that less integration testing of other processes and process sets is required, and therefore both time and money are saved. In order to realize these savings, improvements to methods and systems of scheduling processes are introduced. In particular, each process set or partition may be scheduled independently from some or all of the other process sets and partitions. This way, modifications may be made to one or more processes in a given set or partition without substantially impacting the schedule of the processes in other sets and partitions.
Schedule 400 is organized in 80 ms frames. Two such frames, frame 416 and frame 418, are displayed in
There are various means with which processes can be assigned to timeslots. For example, priority can be given to processes with the highest level of determinism or the highest level of security. Additional factors may be used alone or in combination with a process's level of determinism or level of security to decide in which slots a process should execute. Furthermore, processes should be assigned to timeslots so that all processes can execute according to their execution frequency, execution duration, and IPC constraints.
Note that there is an amount of excess capacity in schedule 400; i.e., each 80 ms frame is only at 50% utilization. This is common in real-time systems, as a portion of a schedule's capacity is preferably reserved to ensure that a real-time system does not become overloaded. If the utilization of a schedule were to approach 100%, spare capacity could be added back into the schedule by adding more CPUs, memory and busses to the system, or replacing the existing CPUs, memory and busses with faster components.
As long as schedule 400 is not adjusted, the real-time system executing schedule 400 will operate in a reasonably predictable fashion. However if changes need to be made to any of the processes in schedule 400, these changes may require modification of schedule 400. For example, suppose that process P1 was altered so that its execution duration became 20 ms while its execution frequency and IPC requirements remained the same. In its current state, schedule 400 would not be able to support this change. In particular, the ordering of the execution of processes P1 and P2 would have to be modified so that processes P1 and P2 could both meet their execution frequency, execution duration and IPC requirements within any given frame. As discussed above, any change in schedule 400 may result in a new ordering of execution events that would have to undergo potentially extensive and costly integration testing.
Rather than risk having to redesign and retest the computing system's execution schedule each time a change is made to a process or to IPC requirements, schedule creation and management may be able to be substantially improved.
Schedule 500 substantially decouples the development and test of process P1 from that of process P2. The unshaded timeslots are assigned to process P1, while the shaded timeslots are assigned to process P2. This allows the software engineers who develop each process to modify their process so that it executes within the bounds of its assigned timeslots. This modification will not impact the execution schedule of the other process, unless such a modification prevents processes P1 and P2 from fulfilling their IPC requirements. For example, if process P1 is altered so that its execution duration became 20 ms while its execution frequency and IPC requirements remained the same, there would be little or no impact on process P2.
Thus, the software engineers may develop and unit test each process largely in isolation from any other process. Doing so will likely reduce the time and expense associated with software development and test cycles, because each process set or partition can be developed and tested independently from one another. In full generality, the technique illustrated by schedule 500 can be employed in larger, more complex systems that comprise tens, hundreds, or thousands of processes.
In an alternate embodiment, instead of assigning timeslots to processes, a set of timeslots may be assigned to a partition of processes or a plurality of processes. This would allow the software engineers in charge of a given partition or plurality of processes to schedule some of all of their processes within the constraints of their given timeslots, without needing to do so at the level of granularity of each process. Furthermore, the timeslots assigned to a process, partition of processes, or plurality of processes need not be contiguous and regular as in schedule 500, and may be assigned in a non-contiguous or irregular fashion.
Furthermore, the processes of either or both of the first and second plurality of processes may be divided into partitions based on their characteristics or execution requirements. For example, the first and pluralities of processes may each be divided into partitions based on process criticality, process security, or some other factor. These processes, or the partitions of processes may execute on the same CPU or across multiple CPUs.
Additionally, method 600 is not limited to just two sets of timeslots and two pluralities of processes. For example, the schedule may be further divided into three sets of timeslots, each set associated with one of three pluralities of processes. Preferably, any such additional plurality of processes is developed by yet another group of software engineers to execute within an additional set of timeslots.
In step 616, IPC events are assigned to the first and second set of timeslots in accordance with the associated delay constraints of each event. Ideally, these IPC events are assigned to timeslots such that the transmission of a unit of data occurs during a timeslot assigned to the transmitting process and the reception of a unit of data occurs during a timeslot assigned to the receiving process. In between being transmitted and received, a unit of data may be stored in the computer system's memory 122. The assignment of processes and IPC events to timeslots is preferably in accordance with the process's execution frequency, execution duration, IPC requirements, or some combination of these factors.
At step 618, the schedule is written to computer system memory. Once written, the schedule is preferably in a form such that some parts of the schedule can be modified without having a negative impact on the other parts of the schedule. Ideally, the software engineers associated with a plurality of processes can modify their processes without requiring that any other processes be modified.
Furthermore, the processes may be divided into partitions based on their characteristics or execution requirements. For example, the first and pluralities of processes may each be divided into partitions based on process criticality, process security or some other factor. These processes, or partitions of processes, may execute on the same CPU or across multiple CPUs.
At step 716, a schedule is defined to include a first set of timeslots and a second set of timeslots. At step 718, a first partition of processes is assigned to the first set of timeslots according to the first set of process execution frequencies, and at step 720, a second partition of processes is assigned to the second set of timeslots according to the second set of process execution frequencies. Additionally, at step 722, IPC events are assigned to the first set of timeslots and the second set of timeslots according to the matrix of inter-process communication event frequencies. At step 724, the schedule is written to memory 122.
The IPC events of steps 714 and 722 may include, for example, a process of the first partition of processes producing a unit of data during one of the first set of timeslots and a process of the second partition of processes consuming the unit of data during one of the second set of timeslots. Furthermore, these IPC events may include the unit of data being stored in memory 122 before it is consumed.
Similar to method 600, the each partition of processes may be developed by a group of software engineers to execute within the assigned set of timeslots. Once written to memory 122, the schedule is preferably in a form such that some parts of the schedule can be modified without having a negative impact on the other parts of the schedule. Ideally, the software engineers associated with a partition of processes can modify these processes without requiring that any other processes be modified.
Exemplary embodiments of the present invention have been described above. Those skilled in the art will understand, however, that changes and modifications may be made to these embodiments without departing from the true scope and spirit of the invention, which is defined by the claims.
