Patent Application
20130124174
Space Situational Awareness (“SSA”) programs rely on precise knowledge of the state and location of objects orbiting the earth for a variety of applications that may include predicting collisions (or near misses) between satellites and space junk, location and timing as a satellite falls to earth, a rocket's intercept of a satellite, and so on. The Space Surveillance Network (“SSN”) is a worldwide network of devices, such as radar and telescope sensors that may be ground-based or space-based, computers, databases, and communications links, involved in discovering, observing, and tracking such orbiting objects. The performance (e.g., accuracy and timeliness of information) and corresponding cost of the SSN depends in large part on the number, placement, and characteristics of the devices of the SSN, referred to as the design of the SSN. Some of the SSN sensors (e.g., Ground-Based Electro-Optical Deep Space Surveillance “GEODSS”) are dedicated to the SSA observations full time, but others (e.g., most radars) are only available for SSA observations part time. One useful metric for evaluating performance of a design of the SSN is the completeness of the coverage of its sensors for tracking a list of targets. The completeness of coverage depends on both sensor-specific and target-specific factors. The sensor-specific factors may include sky coverage of the sensors, slew rates, weather at the sensor sites, availability of the sensors, scheduling and targeting algorithms, re-targeting speed, and so on. The target-specific factors include number of targets, altitudes, inclinations, and so on.
One way to evaluate the performance of an SSN design is to perform a computer simulation of the operation of the devices of the SSN. In addition, any simulation of the SSN should also model the tens of thousands of orbiting objects including their reflective properties and their orbital motions. Computer simulation techniques may be classified as continuous models or discrete models with the discrete models being further classified as either time-stepped models or event-driven models. With a time-stepped model, the simulation times at which the state of the simulation changes are known in advance and typically increase by a constant amount at each step of the simulation. With an event-driven model, the simulation times at which the state of the simulation changes are dynamically computed as the simulation progresses with no advance knowledge of those times or assumption that time increases by a constant amount. A simulation that employs the event-driven model is referred to as a discrete event simulation (“DES”).
A DES model is composed of a flat set of “logical processes” that implement various algorithms of the simulation (e.g., simulating an image collected by a telescope). Each logical process has its state that is updated by the logical process based on event messages that are received. Each logical process may be implemented as an operating system (“OS”) process with its own address space and associated overhead for creating and destroying the OS process. The logical processes communicate and synchronize their processing using inter-process communication facilities of the OS. The state of the DES is the collection of the states of its logical processes. When a logical process receives an event message and subsequently changes its state—the event—the logical process may send event messages (i.e., scheduling an event) to other logical processes that in turn update their states and send event messages to other logical processes. Each logical process may have an “event method” represented by the signature m(t, a1, . . . , an) where t is event time and a1, . . . , an are parameters passed to the logical process. The invoking of an event method of logical process p may be represented as p.m (t, a1, . . . , an). When the event method of logical process p is invoked passing event time t, logical process p may invoke an event method q.m(t′, b1, . . . , br) of logical process q where event time t′ is greater than event time t. In such a case, the event p.m(t, a1, . . . , an) is said to have a direct causal relationship with the event q.m(t′, b1, . . . , br). A sequence of events e1, . . . , eq, each having a direct causal relationship with the next, forms a causal chain by transitivity. Each event in a causal chain is a causal antecedent of all of the following ones.
A DES model may be implemented as either a sequential algorithm or a parallel algorithm. A sequential DES is organized at runtime around a central shared priority queue containing all scheduled events, referred to as the event queue. An event p.m(t, a1, . . . , an) in the event queue is said to be scheduled for a simulation time corresponding to the event time t. A sequential simulation starts with one or more initial events in the event queue. The simulation proceeds by repeatedly removing the event with the earliest event time from the event queue, setting the simulation time to that event time, and then executing the event, which usually has the side effect of inserting one or more future events into the event queue. The simulation terminates normally when some threshold simulation time is reached, although it may also terminate abnormally when the event queue is empty.
A parallel DES (“PDES”) is organized so that the logical processes can be executed in parallel (e.g., on different cores, processors, or nodes of a multi-processor system) and generally without any shared memory. Where there is no shared memory, a PDES employs a distributed event queue rather than a central event queue that is shared by the logical processes. With a distributed event queue, each logical process maintains a local event queue of its scheduled events.
Each logical process in a PDES processes its event messages in event time order even though the event messages may not arrive in event time order. To take advantage of parallelism, each logical process will typically have different simulation times and not be constrained by having their simulation times synchronized (i.e., all the same). As a result, one logical process may have a simulation time that is well ahead of the simulation time of another logical process. Although the logical processes can have different simulation times, each logical process needs to process its event messages in event time order. A difficulty would occur if a logical process received an event message with an event time that was earlier than the simulation time of the logical process.
To avoid the difficulty of processing messages out of order, a PDES may employ a conservative or an optimistic approach. With a conservative approach, each logical process may rely on lookahead information from the logical processes that send event messages to it. The lookahead information sent from a source logical process to a target logical process represents a guarantee that the source logical process will not send any more event messages to the target logical process with an event time that is earlier than a lookahead time. Upon receiving the lookahead time from all of its source logical processes, a target logical process can process its event messages with event times less than the minimum of the lookahead times. With an optimistic approach, a logical process sets its simulation time to the lowest event time of its local event queue. If the logical process later receives an event message with an event time that is lower than its simulation time, then the logical process rolls back its state, processes that event message, and reprocesses event messages that were processed out of order. In addition, the state of other logical processes that received event messages generated while processing event messages out of order may need to also have their states rolled back.
In a PDES, a logical process needed to simulate a subsystem of the overall system being simulated can be very complex and consume vast amounts of computing resources such as memory and processor cycles. Such a logical process may be a bottleneck that results in an unacceptably slow simulation. For example, when one logical process needs significantly more processing cycles than the other logical processes, the other logical processes may need to wait for significant lengths of time for that one logical process to complete before continuing their processing. In some cases, the amount of memory accessible to a single node of a multi-processor system may not even be large enough to store the state of a logical process. As a result, the PDES may not even be able to run on such a multi-processor system or the logical process may need to be redesigned to swap state between memory and secondary storage. Such swapping of state, however, may mean that the other logical processes wait an unacceptably long time, greatly reducing the benefit of such a parallel simulation.
A method and system for performing a parallel discrete event simulation for a space surveillance network is provided. In some embodiments, an internally parallel PDES (“IP-PDES”) system performs logical processes in parallel and further performs the internal processing of at least one logical process in parallel. For example, a simulation for the SSN may include logical processes for optical sensors, radars, orbit determinations, object conjunctions, and so on. These logical processes simulate the occurrence of events (e.g., collecting an optical image by a telescope) within the SSN. Because the IP-PDES system is PDES-based, each logical process may have its processing performed in parallel with the other logical processes. For example, the SSN may have 6 telescopes (i.e., optical sensors) and 3 radars with 9 logical processes executing in parallel on 9 nodes (or processors) of a multi-processor computer system. The IP-PDES system allows for certain logical processes to be designated as internally parallel, meaning that that logical process, referred to as an IP logical process, has its internal processing also performed in parallel. The IP-DES system allocates multiple nodes for such an IP logical process so that executable code (e.g., logic) of the IP logical process executes in parallel at the allocated nodes to simulate the occurrence of an event. Internal parallelism within a PDES may help overcome resource limitations and help speed up the overall simulation. As an example of a resource limitation, an optical image generated by a telescope logical process may have so many picture elements (e.g., 32×1012) that the image cannot fit into the memory of a single node. In such a case, a telescope logical process may be developed so that each node is assigned a portion of the optical image (e.g., 1×1012). Each of the nodes (e.g., 32) can then generate its portion of the optical image in parallel with the other nodes. Moreover, even if the optical image could fit into the memory of a single node, it may be desirable to have the internal processing of the telescope logical process performed in parallel to speed up the overall simulation—especially when the time needed for different logical processes to generate their events varies greatly. In this way, the IP-PDES not only allows the processing of different logical processes to be performed in parallel but also allows internal processing of a logical process to be further performed in parallel.
In some embodiments, the IP-PDES system assigns each logical process of a simulation to be executed as a component of a cooperative parallel programming (“CPP”) system. A cooperative parallel programming system is described in Jefferson, D., Barton, N., Becker, R., Hornung, R., Knap, J., Kumfert, G., Leek, J., May, J., Miller, P., and Tannahill, J., “Overview of the Cooperative Parallel Programming Model,” UCRL-CONF-230029, which is hereby incorporated by reference. A CPP system allows components (e.g., executables) to run on different cores, processors, or generally nodes in parallel and allows components to have internal parallelism. The CPP system may create an operating system (“OS”) process at each node that is allocated to a logical process. A CPP system provides a remote method invocation (“RMI”) functionality that allows invocation of a method or function of a component via an interface exposed by that component. A CPP system may provide a CPP runtime that is linked into the executable of a logical process to provide support for an RMI. When a method of a component is invoked via an RMI, the CPP system identifies the primary node (or primary OS process) of that component and passes an indication of the invocation (e.g., actual arguments) to that node. An RMI can be blocking or non-blocking, also referred to as a synchronous or asynchronous invocation. A method that is invoked by an RMI can be either a sequential body or a parallel body method. A sequential body method is designed to have only one instance executed at a time—no internal parallelism. A parallel body method is designed to have multiple instances executed at the same time—internal parallelism. When a parallel body method of a component is invoked with RMI, the CPP system executing on the primary node of that component launches a thread on each of the nodes (including the primary node) allocated to the component to execute that method in parallel. The threads may communicate using a message passing interface (“MPI”) to synchronize their execution. For example, a thread may use a locking mechanism to coordinate access to a data structure. When a thread executing at a secondary node completes, it notifies the thread executing at the primary node. When all the threads complete, the execution of the parallel body method via an RMI completes. The CPP system creates and destroys threads within OS processes to execute the parallel body method.
In some embodiments, the IP-PDES system may use a sequential body method to send an event message between logical processes and use a parallel body method to process an event message in parallel by the receiving or target logical process. For example, a main thread executing at the primary node for a telescope logical process may receive event messages, queue the event messages, and select the event message to process next. When an event message is to be processed for the telescope logical process, the main thread of the primary node (or primary OS process) invokes a parallel body method of the telescope logical process via an RMI. The CPP system then creates an RMI thread on each of the nodes allocated to the telescope logical process to execute the method in parallel (i.e., generate the optical image for the telescope). When all the RMI threads complete, the telescope logical process can then process the next event message.
The computer system on which the IP-PDES system may be implemented may be a multi-processor computer system with nodes, may include a central processing unit and memory, and may include input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). Each node may include multiple processors and/or cores. Computer-readable media include computer-readable storage media and data transmission media. The computer-readable storage media include memory and other storage devices that may have recorded upon or may be encoded with computer-executable instructions or logic that implement the IP-PDES system. The data transmission media is media for transmitting data using signals or carrier waves (e.g., electromagnetism) via a wire or wireless connection
The IP-PDES system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.
From the foregoing, it will be appreciated that specific embodiments of the invention have been described herein for purposes of illustration but that various modifications may be made without deviating from the scope of the invention. Accordingly, the invention is not limited except as by the appended claims.
This application claims the benefit of U.S. Provisional Patent Application No. 61/493,323 filed Jun. 3, 2011, entitled TESTBED ENVIRONMENT FOR SPACE SITUATIONAL AWARENESS, which is incorporated herein by reference in its entirety.
The United States Government has rights in this invention pursuant to Contract No. DE-AC52-07NA27344 between the U.S. Department of Energy and Lawrence Livermore National Security, LLC, for the operation of Lawrence Livermore National Laboratory.
| Number | Date | Country | |
|---|---|---|---|
| 61493323 | Jun 2011 | US |
