Modular architecture for robotics and teleoperation

Abstract

Systems and methods for modularization and discretization of real-time robot, telerobot and teleoperation systems using passive, network based control laws. Modules consist of network one-ports and two-ports. Wave variables and position information are passed between modules. The behavior of each module is decomposed into uncoupled linear-time-invariant, and coupled, nonlinear memoryless elements and then are separately discretized.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention (Technical Field)

The present invention relates to methods and apparatuses for controlling and permitting communications between components of robotic systems.

2. Background Art

Technical advances in the field of robotics have occurred at an enormous rate during the past three decades. Unfortunately, the advances have occurred in a technological environment requiring generation of hardware and software substantially from scratch for each actual robotic system. This is due to the fact that no general modular architecture exists for development of robotic systems.

It has not heretofore been possible to design components of robotic/telerobotic control systems independently. Instead, the entire system had to be analyzed as a whole. Furthermore, because of the inherent nonlinearities of telerobotic systems, the difficulties involved in sampling, and the complexity of such systems, there was a practical limit to the number of features that could be included in such systems.

The present invention relates to several topics in the robotics literature, including open architecture control systems, telerobotic control systems, passivity based control systems, sensor integration, graphical control, and impedance control.

A number of researchers have introduced open architecture control systems for robots based on UNIX and C. Some have involved use of generic robot controllers. V. Hayward and R. Paul, "Robot Manipulator Control Under UNIX RCCL: A Robot Control `C` Library," International Journal of Robotics Research, vol. 5, pp. 94-111, 1986; L. Salkind, "The SAGE Operating System," IEEE International Conference on Robotics and Automation, pp. 860-865, 1989; E. M. Sollbach and A. A. Goldenberg, "Real-Time Control of Robots: Strategies for Hardware and Software Development," Robotics & Computer-Integrated Manufacturing, vol. 6, pp. 323-329, 1989; D. B. Stewart, D. E. Schmitz and P. K. Khosla, "Implementing Real-Time Robotic Systems Using CHIMERA II," IEEE International Conference on Robotics and Automation, pp. 598-603, 1990. Others have developed high-level interfaces to existing controllers. D. J. Miller and R. C. Lennox, "An Object-Oriented Environment for Robot System Architectures," IEEE International Conference on Robotics and Automation, pp. 352-361, 1990. These systems have been designed primarily for robotic control, and thus focus on issues such as task planning, programmability, and sensor interfacing. The present invention does not plan tasks nor optimize the control architecture for particular sensors. Instead, it provides an intuitive, reconfigurable environment for telerobotics, with ample capability for incorporation of a high-level task planner.

In the area of telerobotic control architectures, a number of open architectures have been proposed for controlling unilateral systems, i.e., those in which the operator is not dynamically coupled to the robot. P. T. Boissiere and R. W. Harrigan, "An Alternative Approach to Telerobotics: Using a World Model and Sensory Feedback," Third Topical Meeting on Robotics and Remote Systems, pp. 1-9, 1989; R. A. Brooks, "A Robust Layered Control System for a Mobile Robot," IEEE Journal of Robotics and Automation, vol. RA-2, pp. 14-23, 1986; S. Graves, L. Ciscon and J. D. Wise, "A Modular Software System for Distributed Telerobotics," IEEE International Conference on Robotics and Automation, pp. 2783-2785, 1992; R. Lumia, J. Fiala and A. Wavering, "The NASREM Robot Control System Standard," Robotics & Computer-Integrated Manufacturing, vol. 6, pp. 303-308, 1989. The present invention works at a lower level than unilateral control systems and provides a framework for developing sophisticated real-time dynamic behavior in bilateral telerobotic systems.

Work has been done in building high-performance bilateral teleoperation systems using special purpose hardware. P. G. Backes and K. S. Tso, "UMI: An Interactive Supervisory and Shared Control System for Telerobotics," IEEE International Conference on Robotics and Automation, pp. 1096-1101, 1990; A. Bejczy and Z. Szakaly, "Universal Computer Control System (UCCS) for Space Telerobots," IEEE International Conference on Robotics and Automation, pp. 318-324, 1987; S. Hayati, T. Lee, K. Tso and P. Backes, "A Testbed for a Unified Teleoperated-Autonomous Dual-Arm Robotic System," IEEE International Conference on Robotics and Automation, pp. 1090-1095, 1990. The present invention works at a higher level than these bilateral systems, allowing greater discretion in choosing robots and sensors, and also looser couplings between master and slave.

The underlying control equations of the present invention are based on passive two-port elements. Passive system behaviors have long been used as a model for controlling autonomous systems, N. Hogan, "Impedance Control: An Approach to Manipulation," ASME Journal of Dynamic Systems and Control, vol. 107, pp. 1-7, 1985, and have recently been applied to bilateral continuous teleoperation systems. R. J. Anderson and M. W. Spong, "Bilateral Control of Teleoperators with Time Delay," IEEE Transactions on Automatic Control, vol. 34, pp. 494-501, 1989; J. E. Colgate, "Power and Impedance Scaling in Bilateral Manipulation," IEEE International Conference on Robotics and Teleoperation, pp. 2292-2297, 1991; G. Raju, G. C. Verghese and T. B. Sheridan, "Design Issues in 2-port Network Models of Bilateral Remote Manipulation," IEEE International Conference on Robotics and Automation, pp. 1316-1321, 1989. A passive telerobotic controller can be proved stable in spite of the non-linear behavior of the operator, the robot, and the environment. R. J. Anderson and M. W. Spong, "Asymptotic Stability for Force Reflecting Teleoperators with Time Delay," The International Journal of Robotics Research, vol. 11, pp. 135-149, 1992.

The present invention is of a discrete real-time architecture providing for both stability and modularity of robotic systems. The invention solves the problems discussed above.

SUMMARY OF THE INVENTION (DISCLOSURE OF THE INVENTION)

The present invention is of a modular robotics system and a method for controlling the system, comprising: providing at least one manipulator to the system; providing a plurality of modules to the system; decomposing the modules into either linear time invariant uncoupled dynamic or nonlinear coupled memoryless elements, the elements being independently discretized; and passing wave variables between the modules. In the preferred embodiment, the invention further comprises passing between the modules position information corresponding to a current position of the manipulator. The behavior of each of the modules is passive and is represented by a scattering operator and each of the modules is decomposed into either linear time invariant uncoupled dynamic elements by a scattering operator upon which a bilinear map substitution has been performed or into nonlinear coupled memoryless elements by a scattering operator of Jacobians or damping terms.

The invention is also of a robotics system and a method for controlling the system, comprising: providing at least one manipulator to the system; providing a plurality of modules to the system; passing information between the modules; and controlling movements of the manipulator by bilateral teleoperation. In the preferred embodiment, wave variables and position information corresponding to the current position of the manipulator are passed between the modules. The behavior of each of the modules is passive and is represented by a scattering operator and each of the modules is decomposed into either linear time invariant uncoupled dynamic elements by a scattering operator upon which a bilinear map substitution has been performed or into nonlinear coupled memoryless elements by a scattering operator of Jacobians or damping terms.

A primary object of the present invention is to provide a modular architecture for discrete real-time dynamic control of bilateral telerobotic systems.

A primary advantage of the present invention is that it provides for rapid prototyping and synthesis of telerobotic systems.

Another advantage of the present invention is that it provides for guaranteed stability of the robotic system even with arbitrary time delays, sampling rates, and asynchronous communication.

A further advantage of the present invention is that it can be implemented on multiple relatively inexpensive computers rather than on a single high-performance computer.

An additional advantage of the present invention is that it can achieve greater complexity and permits more features than a non-modular system.

Other objects, advantages and novel features, and further scope of applicability of the present invention will be set forth in part in the detailed description to follow, taken in conjunction with the accompanying drawings, and in part will become apparent to those skilled in the art upon examination of the following, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and form a part of the specification, illustrate several embodiments of the present invention and, together with the description, serve to explain the principles of the invention. The drawings are only for the purpose of illustrating a preferred embodiment of the invention and are not to be construed as limiting the invention. In the drawings:

FIG. 1 is a diagram of a system of two continuous linear time invariant networks with impedances Z.sub.1 and Z.sub.2 ;

FIG. 2 is a diagram of the system of FIG. 1 as simulated by a computer as two separate modules;

FIG. 3 is a diagram of a system which is passive for any time delay and any passive impedance;

FIG. 4 is a diagram of a connection of two one-port modules;

FIG. 5 is a diagram of the system of FIG. 4 with a two-port connection;

FIG. 6 is a diagram of a Jacobian represented as a transformer;

FIG. 7 is diagram illustrating the transformation of a Jacobian scattering matrix to its inverse;

FIG. 8 is a diagram illustrating the passing of information between the modules of the present invention;

FIG. 9 is a diagram of a spring-damper system as a one-port module (KB1 Port);

FIG. 10 is a diagram of a forward kinematics port;

FIG. 11 is a diagram of an inverse kinematics port;

FIG. 12 is a diagram of a system having both a dynamic component and a nonlinear component;

FIG. 13 is a diagram of a system having a small amount of symmetrically placed damping;

FIG. 14 is a diagram of a circuit equivalent to that of FIG. 13;

FIG. 15 is a diagram of a first circuit for adding angular velocities;

FIG. 16 is a diagram of a second circuit for adding angular velocities;

FIG. 17 is a diagram of the circuit of FIG. 15 with damping;

FIG. 18 is a diagram of a circuit incorporating a proportional derivative control element;

FIG. 19 is a diagram of a module for a force reflecting master;

FIG. 20 is a diagram of an autonomous robot controller for Puma robot hardware;

FIG. 21 is a diagram of the system of FIG. 20 additionally incorporating a previewing/monitoring module;

FIG. 22 is a diagram of the system of FIG. 20 additionally incorporating force feedback modules;

FIG. 23 is a diagram of the system of FIG. 20 additionally incorporating real-time operator input modules;

FIG. 24 is a diagram of the system of FIG. 20 additionally incorporating force reflection modules; and

FIG. 25 is a diagram of a system incorporating the modules of FIGS. 20-24 and additionally incorporating world model modules.

DESCRIPTION OF THE PREFERRED EMBODIMENTS (BEST MODES FOR CARRYING OUT THE INVENTION)

The present invention is of a discrete time modular architecture for integrating-sensors, actuators, input devices, and world model information into a bilateral telerobotic system. By providing modularity and guaranteed stability for arbitrary connections, the present invention permits independent design of robot control systems and provides for the inherent nonlinearitites of telerobotic systems, difficulties involved in sampling, and the complexity of modern systems. The system designer need only follow the criterion for a module of the present invention, or follow the design methodology for the modules of the present invention, and know that when connected, the system will perform in a stable fashion. Furthermore, if reasonable sample times are maintained, then the end user can be assured that the discrete system will approach the behavior of a desired continuous system. The present invention also provides for computation in parallel and at asynchronous rates, permitting flexibility in computing power.

In teleoperation, the slave robot directly tracks motion commanded by a human operator. In robotics, a robot follows trajectories generated by a computer program. "Telerobotics" is a term for describing machine based manipulation using both computer control ("robotics") and direct human input ("teleoperation"). The present invention enables the entire spectrum of telerobotics ranging from teleoperation to robotics.

Bilateral telerobotics is a special class of telerobotics in which forces, either measured directly from force sensors, generated indirectly from proximity sensors, or artificially created using world models, are conveyed to the operator via a force-reflecting master input device. Bilateral teleoperation has shown great promise in laboratories, reducing task times and forces incurred during contact tasks, but has so far failed to achieve wide-spread commercial success.

A robot system will be considered "unstable" if the robot continues to move with increasing kinetic energy, even after a commanded velocity profile has halted. A robot system is "stable", and is said to exhibit "stability", if no initial conditions or initial velocity profiles exist which will cause the system to go unstable. For stricter definitions see the Lyapunov stability arguments in "Asymptotic Stability for Force-Reflecting Teleoperators with Time Delay," The International Journal of Robotics Research, vol. 11, pp. 135-149, 1992.

The ability to mix and match modules in a teleoperator system is an invaluable tool for the system designer. This allows independent development and testing of modules by multiple researchers, the rapid synthesis of telerobotic systems, and systems with much greater complexity than could be achieved by prior art, non-modular techniques. This capability benefits technological development in a wide variety of robotics applications, including waste storage tank remediation, buried waste retrieval, undersea operations, microsurgery, and hazardous activities performed by, for example, NASA, commercial reactor maintainers, and off-shore oil rig personnel. Furthermore, the present invention benefits development of improved flight and driving simulators.

The present invention employs a continuous time model of a two-port network for purposes of designing behaviors into a telerobotic system. Modules are represented by continuous time circuit elements and are broken into one-port and two-port elements. When joined together, the elements form a complete circuit. The advantages of two-port networks are that they give a compact representation, directly illustrate the passive nature of the underlying control laws, and are intuitively understood by a large audience.

In order to discretize the system and maintain a close approximation to a desired continuous response, the present invention incorporates the following elements: The system is decoupled into segments which have either linear time invariant (LTI) uncoupled dynamic behavior or nonlinear coupled memoryless behavior. "Dynamic" segments are associated with energy, storage, while "memoryless" segments are not. Each module is converted to scattering matrix form using wave variables (a.sub.p, b.sub.p, a.sub.n and b.sub.n) as inputs and outputs. The dynamic LTI subsystems are converted to discrete time operators by applying Tustin's method, which maintains the passivity of the scattering operator. Nonlinear memoryless coupled elements are represented as either Jacobians (J) or damping terms (B). Full matrix valued scattering operators for these operators are derived directly from the constituent equations. Because the output of a scattering operator does not lose its passivity when delayed, any amount of time delay in communicating data between modules can be tolerated. In addition to the wave variables, each port passes position information forward to the next module. The present invention insures, however, that the flow of position information does not create a feedback loop. By using wave variables for passing all feedback information, and by designing each module to have a discrete time mapping which maintains passivity for any time delay, the present invention provides for a modular discrete time architecture for robotics and telerobotics.

No existing system provides a modular architecture for discrete real-time dynamic control of bilateral teleoperator systems. Current semi-modular software architectures for teleoperation do not deal with either dynamic control (i.e., direct control of manipulator impedance values) or with bilateral teleoperation. Current bilateral teleoperator systems are not modular in that a master is coded to work with a particular slave, portions of the control system are not easily parallelized, and the control systems cannot work at asynchronous rates.

The present invention is based upon the following basic approach to creating a stable and modular telerobotic control system: combining scattering operators with Tustin's method to obtain modules which are passive for any delay. This approach is set forth below. A basis for understanding the approach is set forth by R. J. Anderson and M. W. Spong "Bilateral Control of Teleoperating with Time Delay," IEEE Transactions on Automatic Control, Vol. 34, 1989, and "Asymptotic Stability for Force Reflecting Teleoperators with Time Delay," International Journal of Robotics Research, Vol. 11, No. 2, 1992, which are herein incorporated by reference.

Suppose one has two continuous LTI networks with impedances Z.sub.1 and Z.sub.2 respectively as shown in FIG. 1, and one would like to have a computer simulate the system as two separate modules. Using f and v as variables one gets the feedback structure shown in FIG. 2 where e.sup.-sT represents a delay inherent in sampling. This system may become unstable since e.sup.-sT Z.sup.-1 (s) is no longer passive. For example let Z.sub.1 =B.sub.1, Z.sub.2 =B.sub.2 and e.sup.-sT =d, the unit delay operator. Then the characteristic equation for the system is 1+Z.sub.2 Z.sub.1.sup.-1 e.sup.-sT =1+(B.sub.2 /B.sub.1)d which will be unstable if B.sub.2 >B.sub.1. ##EQU1## By passing wave variables between modules, rather than force and velocity, one cannot be affected by sample delays and passivity still remains. For instance, the system shown in FIG. 3 will be stable for any time delay and any passive impedance. A stable system is one in which the energy out of the system will be less than or equal to the energy put in. The system of FIG. 3 will be stable because Z.sub.i is passive and then ##EQU2##

For the example case Z.sub.1 =B.sub.1 and Z.sub.2 =B.sub.2 one gets ##EQU3## and the characteristic equation is ##EQU4## which is always stable.

The network in FIG. 4 represents a connection of two one-ports. A generic two-port connection is shown in FIG. 5.

A two-port can be represented in myriad ways, as an impedance, an admittance and in two different hybrid forms. All of these can be mapped to a scattering form as shown in Table 1.

TABLE 1__________________________________________________________________________Type Equation Scattering Matrix Inverse__________________________________________________________________________Impedance ##STR1## S = (Z - Z.sub.0) (Z + Z.sub.0).sup.-1 Z = (I + S) (I - S).sup.-1 Z.sub.0Admittance ##STR2## S = (I - Z.sub.0 Y) (I + Z.sub.0 Y).sup.-1 Y = Z.sub.0.sup.-1 (I - S) (I + S).sup.-1Hybrid ##STR3## ##STR4## ##STR5## ##STR6## ##STR7##__________________________________________________________________________

Each sensor, robot, filter, input device, or other behavior, which is to be incorporated into the architecture of the present invention is first represented as a continuous domain one-port or two-port module containing passive network elements and independent sources. The continuous model represents the ideal behavior that the architecture should achieve. Such representation has been shown to provide a stable method to develop teleoperator systems. In the present invention the passive, stable properties of the continuous system can be preserved upon discretization in such a way that two-port and one-port modules can be individually specified and designed, and then combined in an asynchronous, distributed computing environment.

It is a basic principal of the architecture of the present invention to separate the ideal continuous time two-ports and one-ports into either uncoupled, linear, time-invariant, dynamic network elements (i.e., ideal springs and masses), or memoryless coupled, nonlinear network elements (i.e., Jacobians and nonlinear dampers). This is the PHIDE concept discussed in Anderson and Spong, "Asymptotic Stability for Force Reflecting Teleoperators with Time Delay." Once such a decomposition has been conducted, then the system can be discretized using the techniques laid out in this document.

LTI impedances, Z.sub.1, are given as continuous time operators, e.g., Z(s)=Ms+B.sub.1 and Z=B+K/s. One wants a map, T:S.sub.c .fwdarw.S.sub.d, which maintains passivity upon discretization, namely

.parallel.S(jw).parallel..ltoreq.1.A-inverted.w.sub..fwdarw..sup.T .parallel.S.sub.d (d).parallel.=.parallel.S.sub.d (e.sup.j.spsp.e).parallel..ltoreq.1.A-inverted..crclbar..epsilon.[0,2 .pi.] where d=e.sup.-sT is the delay operator, and S.sub.d is the discrete time version of the continuous scattering operator S.sub.c. By using a bilinear map substitution (i.e., Tustin's method) ##EQU5## passivity is maintained.

Let S.sub.d (d) be the scattering matrix obtained from applying Tustin's method to a passive continuous scattering matrix S.sub.c (s). To compute the norm of S.sub.d (d) one takes ##EQU6##

Making the variable substitution w=tan(.THETA.) we get

.parallel.S.sub.d (d).parallel.=.sub.w.sup.max .parallel.S.sub.c (jw).parallel.w.epsilon.[-.infin.,.infin.]=.parallel.S.sub.c (.crclbar.).parallel.. Thus, because .parallel.S.sub.c (.crclbar.).parallel..ltoreq.1 and .parallel.S.sub.d (d).parallel..ltoreq.1, the sampled system is passive. As an example, let ##EQU7## and then it follows that: ##EQU8##

This can be shown to satisfy the norm conditions. When the present invention is controlling systems having more than one degree-of-freedom (DOF) then the linear dynamic terms are assumed to be decoupled and the filters designed using the previous techniques will be applied along each DOF.

One of the important advantages of the present invention is that it is a full nonlinear system. One can show that a nonlinear system is passive using the same scattering matrix techniques. One of the most common nonlinearities is the Jacobian, J(x.sub.p) which is matrix valued (i.e., coupled Degrees of Freedom) and is functionally dependent on the position, x.sub.p. One can represent the Jacobian as a transformer (see FIG. 6) with a matrix valued hybrid representation: ##EQU9##

Here one cannot start with the equations in Table 1 directly. Instead one works from the constituent equations for the wave variables: ##EQU10##

Using the equation in (*) gives: ##EQU11##

From (**) one gets: ##EQU12## are full rank and f.sub.n and Z.sub.0 v.sub.p are both arbitrary, it follows that ##EQU13##

Thus, one gets the equations

J.sup.T =S.sub.11 J.sup.T +S.sub.12 -I=S.sub.11 -S.sub.12 J and therefore S.sub.11 =(J.sup.T J-I)(I+J.sup.T J).sup.-1 S.sub.12 =2J.sup.T (I+JJ.sup.T).sup.-1.

One also gets from (***) that

S.sub.22 +S.sub.21 J.sup.T =I.fwdarw.S.sub.22 =(I-JJ.sup.T).sup.=I -2J(I+J.sup.T J).sup.-1 J.sup.T S.sub.21 -S.sub.22 J=J.fwdarw.S.sub.21 =2J(I+J.sup.T J).sup.-1.

In conclusion: ##EQU14## where D=2(I+J.sup.T J).sup.-1 and E=2(I+JJ.sup.T).sup.-1

For computing S one chooses the representation with the simplest inverse D or E depending on the number of rows and columns in J. In any case the resulting S is univariate, i.e., S.sup.T S=I. This is proven below: ##EQU15## but D.sup.2 +DJ.sup.T JD=D(2D.sup.-1)D=2D and E.sup.2 +EJJ.sup.t E=2E and DJ.sup.T =J.sup.T E and therefore S.sup.T S=I and .parallel.S.parallel.=1 for any x.

This is a powerful result. One can use Jacobians for much more than kinematics for a device. They can be used for switching non-linearities as well.

One case of a Jacobian map is particularly simple and is used frequently, namely Jacobians of one of the forms: ##EQU16## where R is a changing rotation matrix. The second occurs if the whole system must be mapped into task space. The third occurs whenever two rotations must be summed. In this case the rotational velocity must be rotated by R before it can be summed with a second rotational velocity. This is discussed later.

Applying Eq. .dagger. to J=R one gets D=2(I+J.sup.T J).sup.-1 =I and thus ##EQU17##

The scattering matrix for J.sup.-1 is as simple as that for J. It can be found by reversing the order of v.sub.p and v.sub.n as shown in FIG. 7. Applying Eq. .dagger-dbl. one gets: ##EQU18## Thus, for whatever order, matrix size or nonlinearity in J, one has a closed form representation of S with no singularities and a robust method for computation. (Note: use Cholesky decomposition to compute the positive definite terms D or E.) This represents a substantial gain in the state of the art in kinematics computations for robots.

Velocity maps and feedback structures have been discussed above along with the concept of discrete modules having a scattering operator with norm less than or equal to one. For robot control, however, one must be concerned with position tracking. In theory, one can get position data from integrating the velocities. In practice, one would always be accumulating errors. Therefore, position data must be passed as well. The key to such a technique is to feedforward the position data and feedback the wave variables. Each module should require that .parallel.S(d).parallel..ltoreq.1 for any position input. This philosophy is shown in FIG. 8.

Referring to FIG. 8, the initial module has one input, a.sub.n1, and two outputs, b.sub.n1 and x.sub.n1, each two-port has three inputs, a.sub.p1, a.sub.n1 and x.sub.p1 and three outputs b.sub.p1, b.sub.n1 and x.sub.n, and the final part has two inputs, a.sub.p and x.sub.p, and one output b.sub.p. If the modules maintain the passivity of S(d,x.sub.p) for all x.sub.p then the feedback stability will be guaranteed. The module should then compute the feedforward term x.sub.n =f(x.sub.p,a.sub.p,a.sub.n) for each two-port, and x.sub.n =f(a.sub.n) for the initial one-port. By convention, one would like the input devices to be located towards the left of the circuit and the output device/devices towards the right side of the circuit; thus the flow is directed from left to right. To start the flow in the proper direction one generally initializes the network with either a capacitive port (e.g., a spring damper) or a position/velocity input device. To illustrate these techniques for computing a.sub.n and x.sub.n, the KB1 port of FIG. 9 will be developed next.

The KB1 port shown in FIG. 9 is an LTI, diagonal one-port which simulates a spring with stiffness K and damping B and defining equation ##EQU19## where B and K are diagonal matrices. Rotational elements are discussed later, but note that .DELTA. .crclbar..sub.n can be computed from ##EQU20## and the generated quaternion, q.sub.n from .DELTA. .crclbar..sub.n.)

Applying Tustin's method and the bilinear transformation of Z one gets ##EQU21## Computing x.sub.n (d) gives ##EQU22## For other dynamic linear circuits this technique is much the same.

For the kinematics ports one either has to compute the forward kinematics or the inverse kinematics. Thus even though the feedback structure is well-behaved, the feedforward structure has the standard kinematic problems.

Let .OMEGA.(x) represent the forward kinematics, and then ##EQU23##

For the forward kinematics ports (see FIG. 10) one gets ##EQU24## and for the inverse kinematics ports (see FIG. 11) one gets ##EQU25##

Many modules according to the present invention (e.g., limits ports and box ports) consist of both a dynamic component and a nonlinear component. To develop these types of modules, one divides the system into subnetworks in which one port is linear and uncoupled and the other is a Jacobian connection. Consider the system in FIG. 12.

To represent this system one takes three steps:

1. Convert the two-port signals (a.sub.p, a.sub.n, b.sub.p, b.sub.n) to one-port signals (a.sub.b, b.sub.b). 2. Apply Jacobian mapping routines as just shown. 3. Apply linear dynamics routines as just shown.

To convert the signals consider the following control law: ##EQU26##

This mapping maintains passivity as is shown below. Let b.sub.b =S.sub.b a.sub.b. Then solving (*) as a function of S.sub.b gives ##EQU27##

Taking the inverse gives ##EQU28##

Using the norm bound ##EQU29## and therefore the mapping is passive. The remainder of the computation of the module in FIG. 12 requires the Jacobian and LTI techniques already introduced.

Many input modules add position and velocity to the system using an independent source. To demonstrate how this is done in the present invention, see FIG. 13 which also includes a small amount of symmetrically placed damping.

The equations defining this module are given by: ##EQU30##

This can also be shown by applying Kirchoff's laws and superposition to the equivalent circuit in FIG. 14, as follows: ##EQU31## and by symmetry

b.sub.n =(1-.alpha.)a.sub.p +.alpha.a.sub.n +Z.sub.0 v.sub.d.

In Cartesian space one can add velocity elements freely and pass only three position parameters. With rotations one needs to first rotate one of the inputs so that adding angular velocities makes sense. One also needs at least a four parameter representation for orientations for a singularity free representation.

FIGS. 15 and 16 show the two valid ways for adding angular velocity. From FIG. 15 one has

v.sub.n =R(x.sub.d)v.sub.p +v.sub.d (1) R(x.sub.n)=R(x.sub.d)R(x.sub.p) (2) Note that here v=w, x=.crclbar. and ##EQU32## is a skew-symmetry operator (not a scattering operator). One can differentiate (2) to get (1), as shown below using R(x)=S(v)R(x), R(x.sub.n)=S(v.sub.n)R(x.sub.n)=R(x.sub.d)R(x.sub.p)+R(x.sub.d)R(x.sub.p)S(v.sub.n)R(x.sub.n)R.sup.T (x.sub.n)=[S(v.sub.d)+R(x.sub.d)S(v.sub.p)R(x.sub.p)]R.sup.T (x.sub.n)S(v.sub.n)=S(v.sub.d)+R(x.sub.d)S(v.sub.p)R.sup.T (x.sub.d)+S(R(x.sub.d)v.sub.p)

One can also derive the same result with quaternions

q.sub.n =q.sub.p .sym.q.sub.d.

Differentiating and using ##EQU33## one gets ##EQU34##

But Q(q.sub.n)=Q(q.sub.p)Q(q.sub.d) and therefore by multiplying by 2Q.sup.T (q.sub.n) one gets ##EQU35##

Applying commutivity of Q.sup.T (q.sub.p)Q*(q.sub.d) gives ##EQU36##

Redrawing FIG. 15 with damping results in FIG. 17, the standard port for adding angular velocities.

This system must be done in two sections, with the intermediate values a.sub.m =f.sub.m -Z.sub.0 v.sub.m and b.sub.n =f.sub.m +Z.sub.0 v.sub.m. One thus gets the following equations for this port: ##EQU37##

To attach a slave device to a system according to the present invention, one must maintain the scattering norm conditions for the discretized system. This can be done for a common DC motor device with a PD (Proportional Derivative) control law, as follows:

f.sub.m =K*(x.sub.p -x.sub.m)+B2*(v.sub.p -v.sub.m)-B1*v.sub.m wherein f.sub.m is the motor torque, x.sub.p is the motor position, v.sub.m is the motor velocity, x.sub.p is the desire position and v.sub.p is the desired velocity.

Consider the one-port network in FIG. 18. Assume that the PD loop is closed in continuous time. (Note: a dedicated servo card such as a Green Springs IP servo can achieve sample rates of approximately 4 kHz which is close enough to continuous time.) Let the input to the PD controller (with damping gains B.sub.1 and B.sub.2 and position gain K) be the input position x.sub.p. While running the motor, record the wave a.sub.m =f.sub.motor -Z.sub.motor. Then compute ##EQU38## where S is the scattering operator for the LTI PD filter obtained by applying the previous principles. Because the motor is passive, it follows that

.parallel.b.sub.m .parallel..ltoreq.(f.sub.m +Z.sub.0 v.sub.m).ltoreq..parallel.a.sub.m .parallel..

Furthermore, .parallel.S.parallel..ltoreq.1. Thus by taking the two-norm of (*) one gets

.parallel..sub.p .parallel..sup.2 +.parallel.b.sub.m .parallel..sup.2 .ltoreq..parallel.a.sub.p .parallel..sup.2 +.parallel.a.sub.m .parallel..sup.2 and thus .parallel.b.sub.p .parallel..sup.2 .ltoreq..parallel.a.sub.p .parallel..sup.2 for any trajectory x.sub.p as long as x.sub.p =v.sub.p.

If one must filter x.sub.m to get v.sub.m one can still get a passive system. Consider the filtered velocity with filtering constant .gamma.. ##EQU39##

Applying the same filter to f.sub.m gives ##EQU40## and applying this to a.sub.p gives ##EQU41##

Using the same filter S(K,B.sub.1,B.sub.2) gives ##EQU42## because

.parallel.a.sub.p .parallel..ltoreq..parallel.a.sub.p .parallel. and .parallel.a.sub.n .parallel..ltoreq..parallel.b.sub.n .parallel. and it follows that .parallel.b.sub.p .parallel..sup.2 .ltoreq..parallel.a.sub.p .parallel..sup.2.

To implement a module for a force reflecting master, one puts the master device at the initial one-port position, as in FIG. 19. One would then like to back drive the master with a force, f.sub.motor and record the position x.sub.motor and velocity v.sub.motor while maintaining a passive coupling.

Let the motor command be given by ##EQU43## and define the wave output by ##EQU44## and suppose the master module's impedance is given by

f.sub.motor =-(B.sub.motor +Z.sub.motor)v.sub.motor for some B.sub.motor >0 and any positive real Z.sub.motor. Then it follows that for a suitably small T or suitably large filtering .alpha. one gets: ##EQU45##

EXAMPLE CONTROL SYSTEMS AND INDUSTRIAL APPLICABILITY

The invention is further illustrated by the following non-limiting examples.

To illustrate the present invention, building up of a complex system a few modules at a time is presented below. To build a working system, one starts with an initial one-port element, connect it to any number of two-port elements, and then terminate the network with another one-port element. For a valid connection, the degrees of freedom (e.g., 1, 6, or n) and the operational space (i.e., world space or joint space) at the interface of both connected ports must match. Icons are used in the figures to represent the connections. Each icon shows both a network and a pictorial representation of the module.

Building an autonomous robot controller

FIG. 20 illustrates an autonomous robot controller 10 for Puma robot hardware constructed according to the present invention. The controller 10 comprises three primary units: a module for generating trajectories 12, a module for computing the kinematics of the device 14, and a module for driving the joint servos 16. A termination one-port module 18 completes the system.

The Trajectory Module 12 is a two-port module which generates trapezoidal velocity profiles either in joint space or in world space. In joint space, the motion is interpolated; i.e., each joint motion starts and stops at the same time. In world space, the motion is straight-line; i.e., follows a straight path in Cartesian space and tracking the shortest rotational arc.

The Puma Kin Module 14 is a two-port module which converts forces and velocities in world space to forces and velocities in joint space. This module requires the Jacobian and inverse kinematics routines for the manipulator, but does not require any information about hardware connections.

The Puma Joint Module 16 is a one-port module which maintains a direct connection to the Puma robot hardware. It drives the Puma to any position commanded. Because this module knows the true position of the slave robot, it can also be used to align or synchronize the system to an initial configuration.

The KB1 Module 18 is a one-port module which terminates the left side of the network. It simulates the dynamics of a spring-damper system with stiffness K and damping B. By providing a capacitive termination, flow is directed toward the right side of the network. By modifying the stiffness and damping parameters, impedances ranging from very compliant to very stiff can be realized.

Combining the four modules results in the autonomous robot controller system shown in FIG. 20. In addition, to realize a joint space controller for the Puma, the Puma Kin Module 14 can simply be removed.

Previewing/monitoring motion graphically

In the following sections, the capabilities of the present invention to permit rapid configuration of sophisticated telerobotic systems exceeding the capabilities of commercial systems are illustrated.

FIG. 21 illustrates the incorporation into the system of FIG. 20 of a module permitting previewing and monitoring of robot motion. Visual Module 20 first initiates communication with a remote graphics host, and then continually updates joint information in the robot display. Placing this module adjacent to an input module allows visualization of operator intent. Placing it adjacent to a slave output module allows visualization of the actual motion of the slave. FIG. 21 illustrates one possible instantiation of a motion previewing and monitoring system.

Adding sensor information

The open architecture of the present invention simplifies the attachment of sensor devices through analog, digital, and serial I/O channels. Two possible instantiations are given below.

In FIG. 22, the Puma JR3 Joints Module 22 incorporates force feedback into the system using measurements from a JR3 force sensor. The force feedback signal enables contact operations such as part insertion to be achieved using a commercial robot.

The Sonic Module 24 provides variable damping based on information obtained from a Sandia National Laboratories developed ultrasonic sensor array. It tells the operator if collision of the slave with a sensed object is imminent and will automatically slow the slave's velocity.

The KB2 Module 26 and the Task KB2 Module 28 both introduce additional compliance into the system by simulating a spring K and a dashpot B. The compliance determines how the system reacts to sensor feedback. The KB2 Module 26 works in either world space or in joint space, while the Task KB2 Module 28 works in task space by using the tool rotation matrix R.

FIG. 22 illustrates a system according to the present invention using both force sensor feedback and proximity based damping with the compliance aligned in task space. This ability to align the impedance terms in task space is commonly called impedance control.

Including operator inputs

Real-time operator input can also be incorporated into robotic systems under the present invention. Exemplary modules achieving such capability are introduced in this and the following section.

There are many types of input devices that an operator can use to control the motion of a robot. Joy sticks, teach pendants, and wands all allow some form of velocity and/or position input to the system. In this section, a module for one such device is introduced. The Space Ball Module 30 of FIG. 23 takes velocity inputs from a CIS Dimension 6 space ball and adds them to the other velocities in the system.

In order to record operator generated motions for future playback, the Replay Module 32 is provided. It first records velocities without modification. Then, upon request, it will play back the motion at a faster or slower speed. During playback, the operator can still perturb the motion using other attached modules. As with all other non-force-reflecting input modules, the Replay Module 32 utilizes a force threshold. Motions which would result in this threshold being exceeded are automatically prevented. This capability represents a crucial safety feature of the system.

FIG. 23 illustrates a system in which the operator's space ball generated motion is recorded for playback. During playback, motion from the space ball is added to the overall velocity profile to fine tune the trajectory of the slave.

Including force reflection

Although the present invention provides an excellent system integration mechanism for both autonomous and non-force-reflecting telerobotic systems, a great strength of the present invention is its ability to provide meaningful force information to the operator kinesthetically.

To implement bilateral force reflection, the Kraft Module 34 and the Kraft Kin Module 36 of FIG. 24 are employed. These modules take care of both the joint level hardware interface and the kinematics from a Kraft force-reflecting master device, respectively.

In addition, FIG. 24 includes Index Module 38, which allows the operator to index the motion of a slave device. Furthermore, this module incorporates both axis motion restriction, so that the slave robot can be constrained to move along a line, a plane, or a particular arc, and a force/position scale factor so that arbitrary power and impedance scaling is possible. FIG. 24 illustrates how a Kraft master can be used for generic bilateral force-reflection with a Puma manipulator.

Adding world model information

There are two problems with using sensor information for detecting objects in a world environment. First, because of sensor delays, noise, and non-collocation problems, it is difficult to get a high-fidelity response directly from sensors. Second, as in the case of force sensors, once contact is sensed, it may be too late to respond--the damage has been done. A partial solution to these problems is world modeling.

FIG. 25 includes two modules for incorporating world models into a system: Limits Module 40 and Box Module 42. The Limits Module 40 is a two-port module operating either in world space or joint space. It simulates a spring-damper whenever a position threshold is exceeded. It can be used to prevent the operator from exceeding the joint limits of the manipulator or from straying outside a Cartesian workcell.

The Box Module 42 is a two-port module which simulates a spring-damper whenever the slave device enters a predefined box region. In so doing, it simulates contact forces, preventing the operator from proceeding into the box. By using simulated boxes that are larger than physical objects in the environment, collisions with the environment can be avoided entirely.

FIG. 25 shows an enhanced telerobotic system with kinematics for master and slave, force and position scaling in world space, indexing with axis exclusion, space ball superposition, world modeling, record/playback, graphics display, joint limits, and force feedback. This system, constructed from discrete modules in accordance with the present invention, illustrates the great power of the present invention to provide rapid development of new and more powerful telerobotic systems in a modular manner.

The preceding examples can be repeated with similar success by substituting the generically or specifically described reactants and/or operating conditions of this invention for those used in the preceding examples.

Although the invention has been described in detail with particular reference to these preferred embodiments, other embodiments can achieve the same results. Variations and modifications of the present invention will be obvious to those skilled in the art and it is intended to cover in the appended claims all such modifications and equivalents. The entire disclosures of all references, applications, patents, and publications cited above are hereby incorporated by reference.

Claims

1. A modular robotics system comprising at least one manipulator, a plurality of interchangeable and discrete operational modules, and means for passing wave variables between each of said modules and by causing retention of passivity thereby preventing instability in said system caused by computational delays within and between the modules, wherein each of said modules is decomposed into subnetworks containing either linear time invariant uncoupled dynamic or nonlinear coupled memoryless lumped parameter simulation means selected from the group consisting of an inertia simulation means (M), a stiffness simulation means (K), a linear damping simulation means (B), an independent flow source simulation means (v.sub.d), an independent effort source simulation means (f.sub.d), a Jacobian simulation means (J), and a nonlinear damping simulation means (B), said subnetworks being independently discretized;

wherein each of said modules comprises:

an initialization means; and

a synchronous update means mapping module inputs to module outputs via a discrete control equation to implement an action selected from the group consisting of receiving motion commands from a robot input device, receiving and processing readings from a sensor means, controlling motion of a robot by sending commands to a robot control means, implementing a motion constraint; filtering command signals, mapping force and velocity signals from one reference frame to another reference frame, and simulating a dynamic action of a robot component; and

wherein each of said modules:

requires connections to other modules to complete a functional modular robotics system,

is replaceable by one or more modules to implement a different action,

is initializable and controllable separately from each other module in the modular robotics system, and

receives inputs from neighboring modules and transfers outputs to neighboring modules in an arbitrary sequence, independent of type of said neighboring modules.

2. The system of claim 1 additionally comprising means for passing between said modules position information corresponding to a current position of said manipulator.

3. The system of claim 1 wherein each of said modules is passive and at least one of said modules comprises a subnetwork containing a nonlinear coupled memoryless lumped parameter simulation means.

4. The system of claim 1 wherein each of said modules comprises a scattering operator.

5. The system of claim 4 wherein each of said modules decomposed into subnetworks containing linear time invariant uncoupled dynamic parameter simulation means comprises a scattering operator upon which a bilinear map substitution has been performed.

6. The system of claim 4 wherein each of said modules decomposed into subnetworks containing nonlinear coupled memoryless lumped parameter simulation means comprises a scattering operator comprising a member selected from the group consisting of Jacobians and damping terms.

7. A robotics system comprising at least one manipulator, a plurality of interchangeable and discrete operational modules, means for passing information between said modules comprising means for passing wave variables between each of said modules, means for maintaining stability of said system despite computational delays within and between said modules by causing retention of passivity, and means for bilateral teleoperation of said manipulator, wherein each of said modules comprises:

an initialization means; and

a synchronous update means mapping module inputs to module outputs via a discrete control equation to implement an action selected from the group consisting of receiving motion commands from a robot input device, receiving and processing readings from a sensor means, controlling motion of a robot by sending commands to a robot control means, implementing a motion constraint; filtering command signals, mapping force and velocity signals from one reference frame to another reference frame, and simulating a dynamic action of a robot component; and

wherein each of said module:

requires connections to other modules to complete a functional modular robotics system,

is replaceable by one or more modules to implement a different action,

is initializable and controllable separately from each other module in the modular robotics system, and

receives inputs from neighboring modules and transfers outputs to neighboring modules in an arbitrary sequence, independent of the type of said neighboring modules.

8. The system of claim 7 additionally comprising means for passing between said modules position information corresponding to a current position of said manipulator.

9. The system of claim 7 wherein each of said modules is passive.

10. The system of claim 7 wherein each of said modules comprises a scattering operator.

11. The system of claim 10 wherein one or more of said modules comprises a linear time invariant uncoupled dynamic simulation means and a scattering operator upon which a bilenear map substitution has been performed.

12. The system of claim 10 wherein one or more of said modules comprise a nonlinear coupled memoryless simulation means and a scattering operator comprising a member selected from the group consisting of Jacobian and damping terms.

13. A method for controlling a modular robotics system comprising at least one manipulator, the method comprising the steps of:

a) initializing a plurality of interchangeable and discrete operational modules, wherein each of the modules comprises:

an initialization procedure; and

a synchronous update procedure mapping module inputs to module outputs via a discrete control equation to implement an action selected from the group consisting of receiving motion commands from a robot input device, receiving and processing readings from a sensor means; controlling motion of a robot by sending commands to a robot control means, implementing a motion constraint, filtering command signals; mapping force and velocity signals from one reference frame to another reference frame, and simulating a dynamic action of a robot component; and

wherein each of the modules:

requires connections to other modules to complete a functional modular robotics system

is replaceable by one or more modules to implement a different action,

is initializable and controllable separately from each other module in the modular robotics system, and

receives inputs from neighboring modules and transfers outputs to neighboring modules in an arbitrary sequence, independent of the type of the neighboring module;

b) decomposing the modules into subnetworks containing either linear time invariant uncoupled dynamic or nonlinear coupled memoryless lumped parameter simulation procedures selected from the group consisting of an inertia simulation procedure (M), a stiffness simulation procedure (K), a linear damping simulation procedure (B), an independent flow source simulation procedure (v.sub.d), an independent effort source simulation procedure (f.sub.d), a Jacobian simulation procedure (J), and a nonlinear damping simulation procedure (B), the subnetworks being independently discretized; and

c) passing wave variables between each of the modules and by causing retention of passivity thereby preventing instability in the system caused by computational delays within and between the modules.

14. The method of claim 13 additionally comprising the step of passing between the modules position information corresponding to a current position of the manipulator.

15. The method of claim 13 wherein the step of initializing a plurality of modules comprises initializing modules all of which are passive, at least one of which is decomposed into a subnetwork containing a nonlinear coupled memoryless lumped parameter simulation procedure.

16. The method of claim 13 additionally comprising the step of incorporating into each of the modules a scattering operator.

17. The method of claim 16 wherein the incorporating step comprises the step of incorporating into each of the modules decomposed into subnetworks containing linear time invariant uncoupled dynamic lumped parameter simulation procedures a scattering operator upon which a bilinear map substitution has been performed.

18. The method of claim 16 wherein the incorporating step comprises the step of incorporating into each of the modules decomposed into subnetworks containing nonlinear coupled memoryless lumped parameter simulation procedures a scattering operator comprising a member selected from the group consisting of Jacobians and damping terms.

19. A method of controlling a robotics system comprising at least one manipulator, the method comprising the steps of:

a) initializing a plurality of interchangeable and discrete operational modules to the system, wherein each of the modules comprises:

an initialization procedure; and

a synchronous update procedure mapping module inputs to module outputs via a discrete control equation to implement an action selected from the group consisting of receiving motion commands from a robot input device, receiving and processing readings from a sensor means; controlling motion of a robot by sending commands to a robot control means; implementing a motion constraint, filtering command signals; mapping force and velocity signals from one reference frame to another reference frame; and simulating a dynamic action of a robot component, and

wherein each of said modules:

requires connections to other modules to complete a functional modular robotics system,

is replaceable by one or more modules to implement a different action,

is initializable and controllable separately from each other module in the modular robotics system, and

receives inputs from neighboring modules and transfers outputs to neighboring modules in an arbitrary sequence, independent of the type of the neighboring modules;

b) passing information between the modules by passing wave variables between each of the modules and by retention of passivity thereby maintaining stability of the system despite computational delays within and between the modules; and

c) controlling movements of the manipulator by bilateral teleoperation.

20. The method of claim 19 wherein the step of passing information between the modules additionally comprises the step of passing between the modules position information corresponding to a current position of the manipulator.

21. The method of claim 19 wherein the step of initializing a plurality of modules comprises initializing modules all of which are passive.

22. The method of claim 19 additionally comprising the step of incorporating into each of the modules a scattering operator.

23. The method of claim 22 wherein the incorporating step comprises the step of incorporating into each of the modules comprising a linear time invariant uncoupled dynamic simulation procedure a scattering operator upon which a bilinear map substitution has been performed.

24. The method of claim 22 wherein the incorporating step comprises the step of incorporating into each of the modules comprising a nonlinear coupled memoryless simulation procedure a scattering operator comprising a member selected from the group consisting of Jacobians and damping terms.