Discrete-time tuning of neural network controllers for nonlinear dynamical systems

1. BACKGROUND OF THE INVENTION
The following references, to the extent that they provide exemplary procedural or other details supplementary to those set forth herein, are specifically incorporated herein by reference. These references may provide certain background regarding the subject matter discussed herein.
F. Chen, H. K. Khalil, "Adaptive control of nonlinear systems using neural networks," Int. J. Control, 55(6):1299-1317, 1992.
G. C. Goodwin, K. S. Sin, "Adaptive Filtering, Prediction and Control," Prentice-Hall Inc., Englewood Cliffs, N.J., 1984.
S. Jagannathan, F. L. Lewis, "Discrete-time neural net controller with guaranteed performance," IEEE Transaction on Neural Networks, January 1996.
I. D. Landau, "Evolution of Adaptive control," ADME Journal of Dynamic systems, Measurements, and Control, 115:381-391, June 1993.
I. D. Landau, "Adaptive Control: The Model Reference Approach," Marcel Dekker, Inc., 1979.
A. U. Levine, K. S. Narendra, "Control of nonlinear dynamical systems using neural networks: Controllability and stabilization," IEEE Trans, Neural Network, 4(2), March 1993.
F. L. Lewis, K. Liu, A. Yesildirek, "Multilayer Neural net robot controller with guaranteed performance," IEEE. Trans. on Neural Networks, March 1996.
K. S. Narendra, A. M. Annaswamy, "A new adaptive law for robust adaptation without persistent adaptation," IEEE Trans. Automatic Control, AC-32(2):134-145, Feb. 1987.
K. S. Narendra, K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, 1:4-27, March 1990.
R. Ortega, L. Praly, I. D. Landau, "Robustness of discrete-time direct adaptive controllers," IEEE Trans. Automatic Control, AC-30(12):1179-1187, Dec. 1985.
J. Park and I. W. Sandberg, "Universal approximation using radial-basis-function networks," Neural Computation, 3:246-257, 1991.
M. M. Polycarpou and P. A. Ioannu, "Identification and control using neural network models: design and stability analysis," Tech. Report 91-09-01, Dept. Elect. Eng. Sys., Univ. S. Cal., Sept. 1991.
D. E. Rumelhart, G. E. Hinton, R. J. Wiliams, "Learning internal representations by error propagation," readings in machine learning, Edited by J. W., Shavlik et al., Morgan Kaufman Publishers, Inc. 115-137, 1990.
N. Sadegh, "Nonlinear identification and control via neural networks," Control Systems with inexact Model, DSC-vol 33, ASME winter Annual Meeting, 1991.
R. M. Sanner, J. J. E. Slotine, "Stable adaptive control and recursive identification using radial gaussian networks," Proc. IEEE Conf. Decision and Control, Brighton, 1991.
H. J. Sira-Ramirez, S. H. Zak, "The adaptation of perceptrons with applications to inverse dynamics identification of unknown dynamic systems," IEEE Trans. Systems, Man, and Cybernetics, 21(3), May/June 1991.
J.-J. E. Slotine, W. Li, "Applied Nonlinear Control," Prentice-Hall Inc., Englewood Cliffs, N.J., 1991.
M. Tomizuka, "On the design of digital tracking controllers," ASME Journal o Dynamic Systems, Measurements, and Control, 115:412-418, June 1993.
M. Vidyasagar, "Nonlinear Systems Analysis," Prentice-Hall Inc., Englewood Cliffs, N.J., 1993. B. Widrow, J. Lehr, "30 Years Of Adaptive Neural Networks: Perceptrons, Madaline, And Backpropagation," Proceedings Of The IEEE, 78(9):1415-1442, Sept. 1990.
It has been a desire of scientists to develop control mechanisms so that machines could function independently without human intervention. Such controlled machines should be able to complete an unstructured task and learn from the feedback information about their performance. These machines, therefore, should be able to learn tasks not easily handled by existing machines, and more importantly, continue to adapt and perform these tasks with increasing efficacy under uncertainties.
In order to confront modern technological problems that require systems with intelligent functions such as simultaneous utilization of memory, learning, or high-level decision making in response to "fuzzy" or qualitative commands, intelligent controls is being investigated. Intelligent control should utilize cognitive theory effectively with various mathematical programming techniques. Learning is a first step toward intelligent control and would replace the human operator by making intelligent choices whenever the environment does not allow or justify the presence of a human operator. Learning has the capability of reducing the uncertainties affecting the performance of a dynamical system through on-line modeling (system identification), thereby improving the knowledge about the system so that it can be controlled more effectively.
Considerable research has been conducted in system identification (Narendra and Parthasarathy, 1990) or identification-based NN control (Levine and Narendra, 1993), and little about the use of direct closed-loop multilayer NN controllers that yield guaranteed performance (Chen and Khalil, 1992). On the other hand, some results presenting the relations between NN and direct adaptive control (Landau, 1993; Lewis et al., 1993), as well as some notions on NN for robot control, are given in (Sadegh, 1991; Sanner and Slotine, 1991). A direct continuous-time multilayer neural net robot controller was proposed in (Levine and Narendra, 1993) which guarantees closed-loop tracking performance. However, little about the application of discrete-time multilayer NN in direct closed-loop controllers that yield guaranteed performance is discussed in the literature.
The controller design with NN having multilayers for both continuous an d discrete-time is treated in (Chen and Khalil, 1992; Lewis et al., 1993; Sira-Ramirez and Zak, 1991). In (Chen and Khalil, 1992), the adaptive control of nonlinear systems using multilayer NN is presented very nicely. However, the performance of this controller is dependent upon the choice of the deadzone and the richness of the input signal. In addition, an explicit learning phase for the NN controller is needed initially. In (Sira-Ramirez and Zak, 1991), it is assumed that the input to the multilayer NN is considered to be fixed for successive iterations which is an unreasonable assumption for the controller design. Furthermore, in these papers (Chen and Khalil, 1992; Sira-Ramirez and Zak, 1991) passivity properties of the NN are not investigated. A three-layer NN controller design is presented in (Lewis et al., 1993) for the control of continuous-time systems. However, generalization of the stability analysis to NN having arbitrary number of hidden layers cannot be deduced due to the problem of defining and verifying the persistency of excitation condition for a multilayer NN. In addition, the weight tuning algorithms presented and the associated stability analysis discussed in (Lewis et al., 1993) is specific to robotic systems.
To confront all these issues head on in this invention, a family of novel learning schemes is investigated for a multilayer discrete-time NN whose weights are tuned on-line with no learning phase needed. The weight tuning mechanisms guarantee convergence of the NN weights when initialized at zero, even though there do not exist "ideal" weights such that the NN perfectly reconstructs a certain required function. The controller structure ensures good tracking performance, as shown through a Lyapunov's approach, so that the convergence to a stable solution is guaranteed. Finally, in contrast to adaptive control, it is not necessary to know a priori the structure of the plant; this structural information is instead inferred on-line by the NN.
The controller is composed of a neural net incorporated into a dynamical system, where the structure comes from filtered error notions standard in robot control literature. It is shown that the weight tuning algorithm using the standard backpropagation delta rule in each layer a passive neural net. This, if coupled with the dissipativity of the dynamical system, guarantees the boundedness of all the signals in the closed-loop system under a persistency of excitation (PE) condition disclosed below. However, PE is difficult to guarantee in a NN, which by design has redundant parameters for robust performance. Unfortunately, if PE does not hold, the delta rule generally does not guarantee tracking and bounded weights. Moreover, it is found here that the maximum permissible tuning rate for the developed tuning algorithm decreases as the NN size increases; this is a major drawback. A projection algorithm discussed herein is shown to easily correct the problem. Finally, new modified weight tuning algorithms introduced avoid the need for PE by making the NN robust, that is, state strict passive.
SUMMARY OF THE INVENTION
The present invention includes a method of on-line tuning of a multi-layer neural network used to control a plant, comprising the steps of defining a functional estimate from current values of a plurality of neural network weights and a plurality of layer activation functions; determining weight estimation errors to thereby tune the multi-layer neural network. In exemplary embodiments, the updating step may comprise the step of determining a modified functional estimation error and/or the step of determining a filtered tracking error.
The present invention also includes a method of on-line tuning of a multi-layer neural network used to control a plant, comprising the steps of defining a functional estimate from current values of a plurality of neural network weights and a plurality of layer activation functions; determining weight estimation errors for the plurality of neural network weights; updating the weight estimation errors to thereby tune the multi-layer neural network; and modifying the updated weight estimation errors, thereby obtaining a projection algorithm.
The present invention also includes a method for direct adaptive control of a nonlinear plant having an internal state x and a sensed state and an unknown function f(x), using a discrete-time neural network controller, comprising: calculating a functional estimate of the unknown function f(x) from current values of a plurality of neural network weights and a plurality of activation functions, the calculating being a function of the sensed state; comparing the sensed state with a desired trajectory in an outer feedback loop to produce a tracking error signal; calculating a control action as a function of the tracking error signal and the functional estimate; applying the control action to the plant to maintain the internal state at the desired trajectory; and adjusting the plurality of neural network weights such that the discrete-time neural network controller guarantees tracking stability and bounded weights. In an exemplary embodiment, the adjusting step may comprise: adjusting simultaneously on-line the plurality of neural network weights and the tracking error control; adjusting the plurality of neural network weights as a function of the tracking error signal by extending standard delta rule techniques, thereby providing guaranteed closed-loop stability without preliminary off-line training of the plurality of neural network weights. In another exemplary embodiment, the adjusting step may utilize a projection algorithm. The discrete-time neural network may have a passivity property, thereby guaranteeing robust performance of the discrete-time neural network controller.

BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a three-layer neural network.
FIG. 2 is a block diagram of a neural network controller according to the present invention.
FIG. 3 is a block diagram of a closed-loop error system.
FIGS. 4A and 4B show tracking response of a neural network controller of the present invention with delta rule.
FIGS. 6A and 6B illustrate tracking response of a neural network controller according to the present invention with delta rule.
FIGS. 7A and 7B illustrate tracking response of a neural network controller according to the present invention.
FIGS. 8A and 8B illustrate tracking response of a neural network controller according to the present invention.
FIG. 9 illustrates the tracking response of a PD controller without a neural network.

2. DETAILED DESCRIPTION
Let R denote the real numbers, R.sup.n denote the real n-vectors, R.sup.m.times.n the real m.times.n matrices. Let S be a compact simply-connected set of R.sup.n. With maps f:S.fwdarw.R.sup.k, define C.sup.k (S) as the space such that f is continuous. We denote by .parallel...parallel. any suitable vector norm. Given a matrix A=[a.sub.ij ], A.epsilon.R.sup.n.times.m the Frobenius norm is defined by ##EQU1## with tr () the tract operation. The Frobenius norm .parallel.A.parallel..sub.F, denoted by .parallel...parallel. throughout this application until unless specified explicitly, is nothing but the vector 2-norm over the space defined by stacking the matrix columns into a vector, so that it is compatible with the vector 2-norm, that is, .parallel.Ax.parallel..ltoreq..parallel.A.parallel...parallel.x.parallel..
2.1 Neural Networks
Given x.sub.k .epsilon.R, a three-layer neural network as shown in FIG. 1 has a net output ##EQU2## with .rho.(.), .sigma.(.) and .phi.(.) the activation functions, v.sub.jk the input-layer interconnection weights, w.sub.nj the hidden-layer interconnection weights and q.sub.in the output-layer interconnection weights. The .theta..sub.vm, .theta..sub.wm, .theta..sub.sm, m=1,2 . . . , are threshold offsets and the number of neurons in the input-layer is N.sub.1, with N.sub.2 the number of hidden-layer neurons and N.sub.3 the number of output-layer neurons. The weights of the neural net are adapted on-line in order to achieve a desired performance.
The neural net equation may be conveniently expressed in matrix format by defining x=[x.sub.0, x.sub.1 . . . , x.sub.N1 ].sup.T, y=[y.sub.0, y.sub.1 . . . , y.sub.N3 ].sup.T, and weight matrices Q.sup.T =[q.sub.in ], W.sup.T =[w.sub.nj ], V.sup.T =[v.sub.jk ]. Including x.sub.0 .tbd.1 in x allows one to include the threshold vector [.theta..sub.v1, .theta..sub.v2 . . . , .theta..sub.vN1 ].sup.T as the first column of V.sup.T, so that V.sup.T contains both the weights and thresholds of the input-layer connections. Then,
y=Q.sup.T .phi.(W.sup.T .sigma.(V.sup.T .rho.(x))), (2.2)
with the vector of hidden-layer activation functions .sigma.(z)=[.sigma.(z.sub.1), . . . , .sigma.(z.sub.a)].sup.T defined for a vector z=[z.sub.1, . . . , z.sub.a ].sup.T. Including a 1 as the first element in .sigma.(z) (i.e. placed above .sigma.(z.sub.1)), allows one to incorporate the thresholds .theta..sub.wj as the first column of W.sup.T. Any tuning of S, W and V then includes tuning of the thresholds was well.
A general function f(x).epsilon.C.sup.k (S) can be written, according to the Stone-Weirstrass Theorem [Park and Sandberg, 1991], as
f(x)=Q.sup.T W.sup.T .sigma.(V.sup.T x)+.epsilon.(k), (2.3)
with .epsilon.(k) a neural net functional reconstruction error vector, and the input and output layer activation functions, .rho.(.) and .phi.(.), respectively considered to be linear. If there exists N.sub.2 and constant ideal weights Q, W and V such that .epsilon.=O for all x.epsilon.S, then f(x) is said to be in the functional range of the neural net. In general, given a constant real number .epsilon..gtoreq.0, f(x) is within .epsilon..sub.N of the neural net range if there exists N.sub.2 and constant weights so that for all x.epsilon.R.sup.n, equation (2.3) holds with .parallel..epsilon..parallel..ltoreq..epsilon..sub.N.
Different well-known results for various activation functions .sigma.(.), based on the Stone-Weirstrass Theorem, say that any sufficiently smooth function can be approximately by a suitably large net (Part and Sandberg, 1991). Typical activation functions for .sigma.(.) are bounded, measurable, non-decreasing functions from the real numbers onto [0,1], which include for instance the sigmoid and so on. Note that the selection of .sigma.(.) and the choice of the N.sub.2 for a specified S.OR right.R.sup.n, and neural net reconstruction error bound .epsilon..sub.N, are current topics of research.
Lemma 2.1
Let .sigma.(.) be any bounded measurable non-decreasing function for a multilayered neural network. Then, the functional range of NN given by equation (2.3) is dense in C.sup.k (S).
Proof: See (Ortega et al., 1985).
In this result, the metric defining denseness is the supremum norm (Lewis et al., 1993), e.g., max.sub.x.epsilon.S .parallel.f(x).parallel., f:S.fwdarw.R.sup.k.
In general, equation (2.3) can be rewritten for an n-layer NN as
f(x)=W.sub.n.sup.T .phi..sub.n [W.sub.n-1.sup.T .phi..sub.n-1 (. . . .phi..sub.1 (x(k))]+.epsilon.(k), (2.4)
wherein W.sub.n, W.sub.n-1, . . . , W.sub.2, W.sub.1 are the constant weights and .phi..sub.n-1 (.), .phi..sub.n-2 (.), . . . , .phi..sub.2 (.), .phi..sub.1 (.) denote vectors of activation functions for the n-1 hidden layers, with .phi..sub.n (.) the vector of activation functions for the output layer. For notational convenience, the vector of activation functions of the input layer at the instant k is denoted as .phi..sub.1 (k)=.phi.(x(k)). Then, the vectors of hidden and output layer activation functions are denoted by
.phi..sub.m+1 (k)=.phi.(W.sub.m.sup.T .phi..sub.m (k)); .A-inverted.m=1, . . . , n-1. (2.5)
2.2 Stability and Passive Systems
In order to formulate the discrete-time controller, the following stability notions are needed (Vidyasagar, 1993). Consider the nonlinear system given by
x(k+1)=f(x(k), u(k)), y(k)=h(x(k)), (2.6)
where x(k) is a state vector, u(k) is the input vector and y(k) is the output vector. The solution is said to be globally uniformly ultimately bounded (GUUB) if for all x(k.sub.0)=x.sub.0, there exists and .epsilon.>0 and a number N(.epsilon., x.sub.0) such that .parallel.x(k).parallel.<.epsilon. for all k.gtoreq.k.sub.0 +N.
Consider now the linear discrete time-varying system given by
x(k+1)=A(k)x(k)+B(k)u(k), y(k)=C(k)x(k), (2.7)
with A(k), B(k), C(k) are appropriately dimensional matrices.
Lemma 2.2: Define .psi.(k.sub.1, k.sub.0) as the state-transition matrix corresponding to A(k) for the system (2.7), i.e., ##EQU3## the system (2.7) is exponentially stable.
Proof: See (Sadegh, 1991).
Some aspects of passivity (Goodwin and Sin, 1984) will subsequently be important. The set of time instants which are of interest is Z.sub.+ ={0,1,2 . . . }. Consider the Hilbert space 1.sub.2.sup.n (Z.sub.+) of sequences y:Z.sub.+ .fwdarw.R.sup.n with inner product <..> defined by ##EQU4## Let P.sub.T denote the operator that truncates the signal u at time T. We have ##EQU5## The basic signal space 1.sub.2e.sup.n (Z.sub.+) is given by an extension of 1.sub.2.sup.n (Z.sub.+) according to
1.sub.2e.sup.n (Z.sub.+)={u:Z.sub.+ .fwdarw.R.sup.n 1.A-inverted.T.epsilon.Z.sub.+, P.sub.T u .epsilon.1.sub.2.sup.n (Z.sub.+)}.
It is convenient to use the notation u.sub.T =P.sub.T u and <y, u>.sub.T =<y.sub.T, u.sub.T >.
Define the energy supply function E:1.sub.2e.sup.m (Z.sub.+)x1.sub.2e.sup.p (Z.sub.+)xZ.sub.+ .fwdarw.R. A useful energy function E is the quadratic form
E(u,y,T)=2<y, Su>.sub.T <u, Ru>.sub.T, (2.10)
with S and R appropriately defined matrices. In general, a system with input u(k) and output y(k) is said to be passive if it verifies an equality defined herein as the power form
.DELTA.J=y.sup.T (k)Su(k)+u.sup.T (k)Ru(k)-g(k), (2.11)
with J(k) lower bounded and g(k).gtoreq.0. In other words, ##EQU6## where .gamma..sub.0 .gtoreq.0.
A system is defined to be dissipative if it is passive and in addition ##EQU7## A special sort of dissipativity called state strict passivity (SSP) occurs if g(k) is a monic quadratic function of .parallel.x(k).parallel. with bounded coefficients, where x(k) represents the internal state of the system. Then the 1.sub.2 norm of the state is overbounded in terms of the 1.sub.2 inner product of the output and the input and the input sequence energy (i.e. power). This is used to advantage in concluding some internal boundedness properties of the system without the usual assumptions of observability (e.g., persistency of excitation), stability, and so on.
2.3 Dynamics of the mnth-Order MIMO System
Consider an mnth-order multi-input and multi-output discrete-time nonlinear system, to be controlled, given in multivariable canonical form ##EQU8## with state x(k)=[x.sub.1.sup.T (k), . . . , x.sub.n.sup.T (k)].sup.T with x.sub.i (k).epsilon.R.sup.m ; i=1, . . . n, control u(k).epsilon.R.sup.m, and d(k).epsilon.R.sup.m a disturbance vector acting on the system at the instant k with .parallel.d(k).parallel..ltoreq.d.sub.M a known constant. Given a desired trajectory x.sub.nd (k) and its delayed values, define the tracking error as
e.sub.n (k)=x.sub.n (k)-x.sub.nd (k), (2.15)
and the filtered tracking error, r(k).epsilon.R.sup.m,
r(k)=e.sub.n (k)+.lambda..sub.1 e.sub.n-1 (k)+ . . . +.lambda..sub.n-1 e.sub.1 (k), (2.16)
where e.sub.n-1 (k), . . . , e.sub.1 (k) are the delayed values of the error e.sub.n (k), and .lambda..sub.1, . . . , .lambda..sub.n-1 are constant matrices selected so that .vertline.z.sup.n-1 +.lambda..sub.1 z.sup.n-2 + . . . +.lambda..sub.n-1 .vertline. is stable. Equation (2.16) can be further expressed as
r(k+1)=e.sub.n (k+1)+.lambda..sub.1 e.sub.n-1 (k+1)+ . . . +.lambda..sub.n-1 e.sub.1 (k+1), (2.17)
Using (2.14) in (2.17), the dynamics of the MIMO system (2.14) can be written in terms of the filtered tracking error as
r(k+1)=f(x(k))-x.sub.nd (k+1)+.lambda..sub.1 e.sub.n (k)+ . . . +.lambda..sub.n-1 e.sub.2 (k)+u(k)+d(k). (2.18)
Define the control input u(k) as
u(k)=x.sub.nd (k+1)-f(x(k))+k.sub.v r(k))+k.sub.v r(k)-.lambda..sub.1 e.sub.n (k)- . . . -.lambda..sub.n-1 e.sub.2 (k), (2.19)
with the diagonal gain matrix k.sub.v, and f(x(k)) an estimate of f(x(k)). Then the closed-loop error system becomes
r(k+1)=(k.sub.v r(k)f(x(k))+d(k), (2.20)
where the functional estimation error is given by
f(x(k))=f(x(k))-f(x(k)). (2.21)
This is an error system wherein the filtered tracking error is driven by the functional estimation error.
In the remainder of the this application, equation (2.20) is used to focus on selecting NN tuning algorithms that guarantee the stability of the filtered tracking error r(k). Then, since (2.17), with the input considered as r(k) and the output as e(k) describes a stable system, standard techniques (Slotine and Li, 1991) guarantee that e(k) exhibits stable behavior.
In general, the closed-loop tracking error system (2.20) can also be expressed as
r(k+1)=k.sub.v r(k)+.zeta..sub.0 (k), (2.22)
where
.zeta..sub.0 (k)=f(x(k))+d(k). (2.23)
The next dissipativity result holds for this system.
Theorem 2.2
The closed-loop tracking error system (2.22) is state strict passive from .zeta..sub.0 (k) to k.sub.v r(k) provided that
k.sub.v.sup.T k.sub.v <I. (2.24)
Proof: Select a Lyapunov function candidate
J=r.sup.T (k)r(k). (2.25)
The first difference is given by
.DELTA.J=r.sup.T (k+1)r(k+1)-r.sup.T (k)r(k). (2.26)
Substituting (2.23) in (2.27) yields
.DELTA.J=-r.sup.T (k)[I-k.sub.v.sup.T k.sub.v ]r(k)+2r.sup.T (k)k.sub.v .zeta..sub.0 +.zeta..sub.0.sup.T (k).zeta..sub.0 (k). (2.27)
Note (2.27) is in power form defined in (2.11) with the first term taken as g(k), a monic quadratic function of the state r(k). Hence, (2.22) is a state strict passive system.
3. NN Controller Design
In the remainder of this application, a three-layer NN is considered initially and stability analysis is carried out for the closed-loop system (2.20). Thereafter, all the stability analysis presented for a three-layer NN is shown to be easily extended for a multilayer network having an arbitrary number of hidden layers. In this section, stability analysis by Lyapunov's direct method is performed for a family of multilayer weight tuning algorithms using a delta rule in each layer. These weight tuning paradigms yield a passive NN, yet persistency of excitation (PE) is generally needed for suitable performance. Specifically, this holds as well for standard backpropagation in the continuous-time case (Lewis et al., 1993) and the two-layer discrete-time case (Jagannathan and Lewis, 1993). Unfortunately, PE cannot generally be tested for or guaranteed in a NN, so that these naive multilayer weight tuning algorithms are generally doomed to failure. In addition, for guaranteed stability, the weight tuning using the delta rule at each layer must slow down as the NN becomes larger. By employing a projection algorithm, it is shown that the tuning rate can be made independent of the NN size. Modified tuning paradigms are finally proposed to make the NN robust so that the PE is not needed.
Assume that there exist some constant ideal weights W.sub.3, W.sub.2 and W.sub.1 for a three-layer NN so that the nonlinear function in (2.14) can be written as
f(x)=W.sub.3.sup.T .phi..sub.3 [W.sub.2.sup.T .phi..sub.2 [W.sub.1.sup.T .phi..sub.1 (x(k))]]+.epsilon.(k), (3.1)
where .parallel..epsilon.(k).parallel.<.epsilon..sub.N, with the bounding constant .epsilon..sub.N known. Unless the net is "minimal", suitable "ideal" weights may not be unique. The best weights may then be defined as those which minimize the supremum norm over S of .epsilon.(k). This issue is not a major concern here, as it is needed to known only the existence of such ideal weights; their actual values are not required.
For notational convenience define the matrix of all the ideal weights as ##EQU9## with padding by zeros as required for dimensional consistency. Then, some bounding assumptions can be stated.
Assumption 1
the ideal weights are bounded by known positive values so that .parallel.W.sub.1 .parallel..ltoreq.W.sub.1max, .parallel.W.sub.2 .parallel..ltoreq.W.sub.2max, and .parallel.W.sub.3 .parallel..ltoreq.W.sub.3max, or .parallel.Z.parallel..ltoreq.Z.sub.max.
3.1 Structure of the NN Controller and Error System Dynamics
Define the NN functional estimate by
f(x(k))=W.sub.3.sup.T (k).phi..sub.3 (W.sub.2.sup.T (k).phi..sub.2 (W.sub.1.sup.T (k).phi..sub.1 (x(k)))), (3.2)
W.sub.3 (k), W.sub.2 (k) and W.sub.1 (k) being the current value of the weights. The vector of input layer activation functions is given by .phi..sub.1 (k)=.phi..sub.1 (k)-.phi.(x(k)). Then the vector of activation functions of the hidden and output layer with the actual weights at the instant k is denoted by
.phi..sub.m +1(k)=.phi.(W.sub.m.sup.T .phi..sub.m (k)); .A-inverted.m=1, . . . , n-1. (3.3)
Fact 2
The activation functions are bounded by known positive values so that .parallel..phi..sub.1 (k).parallel..ltoreq..phi..sub.1max, .parallel..phi..sub.2 (k).parallel..ltoreq..phi..sub.2max, and .parallel..phi..sub.3 (k).parallel..ltoreq..phi..sub.3max.
The error in the weights or weight estimation errors are given by
W.sub.3 (k)=W.sub.3 -W.sub.3 (k), W.sub.2 (k)=W.sub.2 -W.sub.2 (k), W.sub.1 (k)=W.sub.1 -W.sub.1 (k), Z(k)=Z-Z(k), (3.4)
where ##EQU10## and the hidden-layer output errors are defined as
.phi..sub.2 (k)=.phi..sub.2 -.phi..sub.2 (k), .phi..sub.3 (k)=.phi..sub.3 -.phi..sub.3 (k). (3.5)
Select the control input u(k) to be
u(k)=x.sub.nd (k+1)=W.sub.3.sup.T (k).phi..sub.3 (k)-.lambda..sub.1 e.sub.n (k)- . . . -.lambda..sub.n-1 e.sub.2 (k)+k.sub.v r(k), (3.6)
where the functional estimate (3.2) is provided by a three-layer NN and denoted in (3.6) by W.sub.3.sup.T (k).phi..sub.3 (k). Then, the closed-loop filtered error dynamics become
r(k+1)=k.sub.v r(k)+e.sub.i (k)+W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k), (3.7)
where the identification error is defined by
e.sub.i (k)=W.sub.3.sup.T (k).phi..sub.3 (k). (3.8)
The proposed NN controller structure is shown in FIG. 2. The output of the plant is processed through a series of delays in order to obtain the past values of the output, and fed as inputs to the NN so that the nonlinear function in (2.14) can be suitably approximated. Thus, the NN controller derived in a straightforward manner using filtered error notion naturally provides a dynamical NN structure. Note that neither the input u(k) or its past values are needed by the NN. The next step is to determine the weight updates so that the tracking performance of the closed-loop filtered error dynamics is guaranteed.
3.2 Weight Updates for Guaranteed Tracking Performance
A family of NN weight tuning paradigms that guarantee the stability of the closed-loop system (3.7) is presented in this section. It is required to demonstrate that the tracking error r(k) is suitably small and that the NN weights W.sub.3 (k), W.sub.2 (k) and W.sub.1 (k) remain bounded, for then the control u(k) is bounded. In order to proceed further, the following definitions are needed.
Lemma 3.1: If A(k)=I-.alpha..phi.(x(k)).phi..sup.T (x(k)) in (2.7), where 0<.alpha.<2 and .phi.(x(k)) is a vector of basis functions, then .parallel..phi.(k.sub.1, k.sub.0).parallel.<1 is guaranteed if there is an L>0 such that ##EQU11## for all k. Then, Lemma 2.2 guarantees the exponential stability of the system (2.7).
Proof: See (Sadegh, 1991).
Definition 3.2: An input sequence x(k) is said to be persistently exciting (Sadegh, 1991) if there are .lambda.>0 and an integer k.sub.1 .gtoreq.1 such that ##EQU12## where .lambda..sub.min (P) represents the smallest eigenvalue of P. Note that PE is exactly the stability condition needed in Lemma 3.1.
In the following, it is considered that the neural net reconstruction error bound .epsilon..sub.N and the disturbance bound d.sub.M are nonzero but known constants. Theorem 3.3 gives two alternative weight tuning algorithms, one based on a modified functional estimation error and the other based on the filtered tracking error, guaranteeing that both the tracking error and the error in the weight estimates are bounded if a PE condition holds.
Theorem 3.3
Let the desired trajectory x.sub.nd (k) be bounded and the NN functional reconstruction error and the disturbance bounds, .epsilon..sub.N,d.sub.M, respectively, be known constants. Take the control input for (2.14) as equation (3.6) and the weight tuning provided for the input and hidden layers as
W.sub.1 (k+1)=W.sub.1 (k)-.alpha..sub.1 .phi..sub.1 (k)y.sub.1.sup.T (k), (3.10)
W.sub.2 (k+1)=W.sub.2 (k)-.alpha..sub.2 .phi..sub.2 (k)y.sub.2.sup.T (k), (3.11)
where
y.sub.1 (k)=W.sub.1.sup.T (k).phi..sub.1 (k), y.sub.2 (k)=W.sub.2.sup.T (k).phi..sub.1 (k), y.sub.2 (k)=W.sub.2.sup.T (k).phi..sub.2 (k). (3.12)
and the weight tuning update for the output layer is given by either:
a) W.sub.3 (k+1)=W.sub.3 (k)+.alpha..sub.3 .phi..sub.3 (k)f.sup.T (k), (3.13)
where f(k) is defined as the functional augmented error given by
f(k)=x.sub.n (k+1)-u(k)-f(x(k)),
or
b) W.sub.3 (k+1)=W.sub.3 (k)+.alpha..sub.3 .phi..sub.3 (k)r.sup.T (k+1), (3.15)
with .alpha..sub.1 >0, .A-inverted.i=1,2,3 denoting constant learning rate parameters or adaptation gains.
Let the output vectors of the input, hidden, and output layers, .phi..sub.1 (k), .phi..sub.2 (k), and .phi..sub.3 (k), be persistently exciting, then the filtered tracking error r(k) and the error in weight estimates, W.sub.1 (k), W.sub.2 (k) and W.sub.3 (k), are GUUB provided the following conditions hold: ##EQU13## where .eta. is given for algorithm (a) as ##EQU14## and for algorithm (b) as ##EQU15##
Remark: PE has not been defined in the literature, to our knowledge, for a multilayer NN and therefor, PE is defined and verified for a multilayer NN during the proof.
Proof:
Algorithm (a): Define the Lyapunov function candidate ##EQU16## The first difference is ##EQU17## Substituting equations (3.7) and (3.10) through (3.13) in (3.21), collecting terms together, and completing the square yields ##EQU18## where .eta. is given in (3.18) with k.sub.vmax the maximum singular value of k.sub.v and ##EQU19## Since (.epsilon..sub.N +d.sub.M) is constant, .DELTA.J.ltoreq.0 as long as ##EQU20## This demonstrates that the tracking error r(k) is bounded for all k.gtoreq.0 and it remains to show that the weight estimates W.sub.1 (k), W.sub.2 (k), and W.sub.3 (k), or equivalently W.sub.1 (k), W.sub.2 (k), and W.sub.3 (k), are bounded.
The dynamics relative to error in weight estimates using (3.10), (3.11) and (3.13) are given by
W.sub.1 (k+1)=[I-.alpha..sub.1 .phi..sub.1 (k).phi..sub.1.sup.T (k)]W.sub.1 (k)+.alpha..sub.1 .phi..sub.1 (k).phi..sub.1.sup.T (k)W.sub.1, (3.26)
W.sub.2 (k+1)=[I-.alpha..sub.2 .phi..sub.2 (k).phi..sub.2.sup.T (k)]W.sub.2 (k)+.alpha..sub.2 .phi..sub.2 (k).phi..sub.2.sup.T (k)W.sub.2, (3.27)
W.sub.3 (k+1)=[I-.alpha..sub.2 .phi..sub.3 (k).phi..sub.3.sup.T (k)]W.sub.3 (k)+.alpha..sub.3 .phi..sub.3 (k).phi..sub.3.sup.T (k)W.sub.3, (3.28)
where the functional reconstruction error .epsilon.(k) and the disturbance d(k) are considered to be bounded. Applying the PE condition (3.9) and Lemma 3.1 by using .phi.(k)=.phi..sub.1 (k)=.phi..sub.1 (k); .A-inverted.i=1; . . . , 3, the boundedness of W.sub.1 (k), W.sub.2 (k), and W.sub.3 (k), in 3.26), (3.27), and (3.28), respectively, and hence of W.sub.1 (k), W.sub.2 (k), and W.sub.3 (k), are assured.
Algorithm (b): Define a Lyapunov function candidate as in (3.20). Substituting (3.7), (3.10) through (3.12) and (3.15) in (3.21), collecting terms together, and completing the square yields ##EQU21## with .eta. is given by (3.19) and where ##EQU22## .DELTA.J.ltoreq.0 as long as equations (3.16), (3.17) hold and ##EQU23## the remainder of the proof follows similarly to that for algorithm (a).
One of the drawbacks of the available methodologies that guarantee the tracking and bounded weights (Lewis et al., 1993) is the lack of generalization of stability analysis to NN having arbitrary numbers of hidden layers. The reason is partly due to the problem of defining and verifying the persistency of excitation for a multilayered NN. For instance, in the case of a three layer continuous-time NN (Lewis et al., 1993), the PE conditions are not easy to derive as one is faced with the observability properties of a certain bilinear system. However, according to the proof presented in Theorem 3.3, the PE for a multilayer NN is defined as the PE (in the sense of Definition 3.2) of all the hidden layer inputs .phi..sub.i (k); .A-inverted.i =1, . . . , n.
The following corollary presents the generalization of Theorem 3.3 to NN having an arbitrary number of hidden layers when the NN functional reconstruction error and the unmodeled disturbances are nonzero but bounded by known constants.
Corollary 3.4
Assume the hypotheses presented in Theorem 3.3 and take the weight tuning of an n-layer NN provided for the input and the hidden layers as ##EQU24## and the weight tuning update for the output layer is given by either:
a) W.sub.n (k+1)=W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)f.sup.T (k), (3.35)
or
b) W.sub.n (k+1)=W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)r.sup.T (k+1), (3.36)
with .alpha..sub.i >0, .A-inverted.i=1, 2, 3, . . . n denoting constant learning rate parameters or adaptation gains.
Let the output vectors of the input, hidden and output layers, .phi..sub.i (k); .A-inverted.i=1, 2, . . . , n, be persistently exciting, then the filtered tracking error r(k) and the error in weight estimates, W.sub.i (k); .A-inverted.i=1, 2, . . ., n, are GUUB provided the condition ##EQU25## hold where .eta. is given for algorithm (a) as ##EQU26## and for algorithm (b) as ##EQU27##
Proof: Consider the filtered tracking error dynamics (3.7) and rewrite (3.7) for an n-layer NN to obtain
r(k+1)=k.sub.v r(k)+e.sub.i (k)+W.sub.n.sup.T .phi..sub.n (k)+.epsilon.(k)+d(k), (3.41)
where the identification error is given by
e.sub.i (k)=W.sub.n.sup.T (k).phi..sub.n (k). (3.42)
Algorithm (a): Define the Lyapunov function candidate ##EQU28## The first difference is ##EQU29## Considering the input and hidden (3.33), output (3.35) weight updates as well the tracking error dynamics (3.41), and using these in (3.44), one may obtain ##EQU30## with .eta. given in (3.39) .DELTA.J.ltoreq.0 as long as conditions (3.37), (3.38) hold and the bound on the filtered tracking error is given in (3.25) where the constants .gamma. and .rho. are presented in (3.46) and (3.47) respectively. This demonstrates the GUUB of r(k) and it remains to show that the weight estimates, W.sub.i (k), .A-inverted.i=1, . . . , n, or equivalently W.sub.i (k), .A-inverted.i=1, . . . , n, are bounded.
The dynamics relative to error in weight estimates are given for the input and hidden layers as
W.sub.i (k+1)=[I-.alpha..sub.i .phi..sub.i.sup.T (k).phi..sub.i (k)]W.sub.i (k)+.alpha..sub.i .phi..sub.i (k).phi..sub.i.sup.T (k)W.sub.i, .A-inverted.i=1, . . . , n-1 (3.48)
and for the output layer as
W.sub.n (k+1)=[I-.alpha..sub.n .phi..sub.n.sup.T (k).phi..sub.n (k)]W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)[W.sub.n.sup.T .phi..sub.n (k)+.epsilon..sub.N +d(k)].sup.T, (3.49)
where the functional reconstruction error and disturbances are considered to be bounded. Applying the PE condition (3.9) and Lemma (3.1) with .phi.(k)=.phi..sub.i (k); .A-inverted.i=1, . . . , n, the boundedness of W.sub.i (k), .A-inverted.i=1, . . . , n, in (3.48) and (3.49) and hence of W.sub.i (k), .A-inverted.i=1, . . . , n, are assured.
Algorithm (b): Define the Lyapunov function candidate as in (3.43). Substituting (3.33), (3.36) and (3.41) in (3.44), collecting terms together, and completing the square yields ##EQU31## with .eta. given in (3.40). The remainder of the proof follows similarly to that of algorithm (a).
Note from (3.25) and (3.32), that the tracking error increases with the NN reconstruction error bound .epsilon..sub.N and the disturbance bound d.sub.M, yet small tracking errors, but not arbitrarily small, may be achieved by selecting small gains k.sub.V. In other words, placing the closed-loop error poles arbitrarily close to the origin inside the unit circle forces smaller tracking errors. In contrast, by selecting large gains for the case of continuous system, arbitrarily small tracking errors can be obtained (Ortega et al., 1985).
It is important to note that the problem of initializing the net weights (referred to as symmetric breaking (Slotine and Li, 1991)) occurring in other techniques in the literature does not arise, since when Z(0) is taken as zero (with nonzero thresholds) the PD term of k.sub.v r(k) stabilizes the plant, on an interim basis, for instance in certain restricted class of nonlinear system such as robotic systems. Thus, the NN controller requires no learning phase.
3.3 projection Algorithm
The adaptation gains for an n-layer NN, .alpha..sub.i >0, .A-inverted.i=1, 2, . . . n, are constant parameters in the update laws presented in (3.33) through (3.36). These update laws correspond to the delta rule (Slotine and Li, 1991; Tomizuka, 1993), or referred to as the Widrow-Hoff rule (24), used in standard backpropagation. This reveals that the update tuning mechanisms employing the delta rule have a major drawback. In fact using (3.37), the upper bound on the adaptation gain can be obtained as ##EQU32##
Since .phi..sub.i (k).epsilon.R.sup.N.sbsp.pi, with N.sub.p the number of hidden-layer neurons in the ith layer, it is evident that the upper bound on the adaptation gain at each layer depends upon the number of hidden-layer neurons present in that particular layer. Specifically, if there are N.sub.p hidden-layer neurons and the maximum value of each hidden-node output in the ith layer is taken as unity (as for the sigmoid), then the bounds on the adaptation gain in order to assure stability of the closed-loop system are given by ##EQU33##
In other words, the upper bound on the adaptation gain at each layer for the case of delta rule decreases with increase in the number of hidden-layer nodes in that particular layer, so that learning must slow down for guaranteed performance. The phenomenon of large NN requiring very slow learning rates has often been encountered in the practical NN literature (Chen and Khalil, 1992; Narenda and Parthasarathy, 1990), but never adequately explained.
This major drawback can be easily overcome by modifying the update rule at each layer in order to obtain a projection algorithm (Jagannathan and Lewis, 1996). To wit, replace the constant adaptation gain at each layer by ##EQU34## where
.zeta..sub.i >0, .A-inverted.i=1, . . . , n (3.56)
and
0<.xi..sub.i <2, .A-inverted.i=1, . . . , n-1
0<.xi..sub.i <1, .A-inverted.i=n (3.57)
are constants. Note that .xi..sub.i, .A-inverted.i=1, . . . n is now the new adaptation gain at each layer and it is always true that ##EQU35## hence guaranteeing (3.37) for every N.sub.p at each layer.
From the bounds indicated for the adaptation gains in (3.58), it is interesting to not that the upper bound on the adaptation gains for the input and hidden layers is 2 whereas for the output layer the upper bound is given by 1. It appears that the hidden layers act as pattern extractors (Rumelhart et al., 1990). In other words, the hidden layers of a multilayer NN are employed for the identification of the nonlinear plant and the output layer is used for controlling the plant. Nowhere else does there appear an upper bound less than 1 for the adaptation gains in the output layer.
Note that for guaranteed closed-loop stability, it is necessary that the hidden-layer outputs .phi..sub.i (k); .A-inverted.i=1, . . . , n, be PE. In other words, Theorem 3.3 and Corollary 3.4 reveal the necessity of the PE condition to guarantee boundedness of the weight estimates in the presence of unmodelled bounded disturbances and functional reconstruction errors. However, it is very difficult to verify or guarantee the PE of the hidden-layer output functions .phi..sub.i (k); .A-inverted.i=1, . . . , n, in the case of a multilayer NN. This possible unboundedness of the weight estimates (c.f. parameter estimates in adaptive control) when PE fails to hold is known as parameter drift (Ortega et al., 1985). In the next section, improved weight tuning paradigms are presented so that PE is not required.
3.4 Weight Tuning Modification for Relaxation of Persistency of Excitation Condition
Approaches such as .sigma.-modification (Sira-Ramirez and Zak, 1991), or e-modification (Park and Sandberg, 1991) are available for the robust adaptive control of continuous systems wherein the persistency of excitation condition is not needed. A three-layer NN with continuous weight update laws and e-modification was developed (Lewis et al., 1993), and the GUUB of both the tracking error and the error in weight estimates was demonstrated. However, modification to the standard weight tuning mechanisms in discrete-time without the necessity of PE is, yet to be investigated in both the NN and adapative control communities. In our previous paper (Jagannathan and Lewis, 1993), an approach similar to e-modification is derived for two-layer (i.e., linear) discrete-time NN. In this invention, the modified weight tuning algorithms discussed for a two-layer discrete-time NN in (Jagannathan and Lewis, 1993) are extended to a multilayer discrete-time NN so that Pe is not needed. In fact, the following theorem shows two tuning algorithms that overcome the need for PE in the case of a multilayer NN.
Theorem 3.5
Assume the hypotheses presented in Theorem 3.3, and consider the modified weight tuning algorithms provided for the input and hidden layers as
W.sub.1 (k+1)=W.sub.1 (k)-.alpha..sub.1 .phi..sub.1 (k)y.sub.1.sup.T (k)-.GAMMA..parallel.I-.alpha..sub.1 .phi..sub.1 (k).phi..sub.1.sup.T (k).parallel.W.sub.1 (k), (3.59)
W.sub.2 (k+1)=W.sub.2 (k)-.alpha..sub.2 .phi..sub.2 (k)y.sub.2.sup.T (k)-.GAMMA..parallel.I-.alpha..sub.2 .phi..sub.2 (k).phi..sub.2.sup.T (k).parallel.W.sub.2 (k), (3.60)
and the modified weight update for the output layer by either:
a) W.sub.3 (k+1)=W.sub.3 (k)-.alpha..sub.3 .phi..sub.3 (k)f.sup.T (k)-.GAMMA..parallel.I-.alpha..sub.3 .phi..sub.3 (k).phi..sub.3.sup.T (k).parallel.W.sub.3 (k), (3.61)
or
b) W.sub.3 (k+1)=W.sub.3 (k)-.alpha..sub.3 .phi..sub.3 (k)r.sup.T (k+1)-.GAMMA..parallel.I-.alpha..sub.3 .phi..sub.3 (k).phi..sub.3.sup.T (k).parallel.W.sub.3 (k), (3.62)
with .GAMMA.>0 a design parameter. Then, the filtered tracking error r(k) and the NN weight estimates W.sub.1 (k), W.sub.2 (k), and W.sub.3 (k) are GUUB provided the following conditions hold: ##EQU36## for algorithm (a) and ##EQU37## for algorithm (b) where .eta. is given in (3.18) for algorithm (a) and .sigma. for algorithm (b) is given by ##EQU38## where .eta. in (3.66) given in (3.19). Remarks: No PE condition is needed with the modified tuning algorithms.
Proof:
Algorithm (a): Select the Lyapunov function candidate (3.20). Use the tuning mechanism (3.59) through (3.61) in (3.21) to obtain ##EQU39##
Consider (3.67), rewrite the last term in terms of .parallel.Z(k).parallel., denote the positive constants c.sub.max and c.sub.min as the maximum and minimum singular value of the diagonal matrix given by ##EQU40## and complete the squares for .parallel.Z(k).parallel. to obtain ##EQU41## Then .DELTA.J.ltoreq.0 as long as (3.63) through (3.65) hold and the quadratic term for r(k) in (3.71) is positive, which is guaranteed when ##EQU42## Similarly, completing the squares for .parallel.r(k).parallel. using (3.67) yields ##EQU43## where .lambda. and .rho. are given in (3.68) and (3.69) respectively. Then .DELTA.J.ltoreq.0 as long as (3.63) through (3.65) hold and the quadratic term for .parallel.Z(k).parallel. in (3.74) is positive, which is guaranteed when ##EQU44## From (3.73) and (3.75), it can be concluded that the tracking error r(k) and the error in weight estimates Z(k) are GUUB.
Algorithm (b): Select the Lyapunov function candidate (3.20). Use the tuning mechanism (3.59), (3.60) and (3.62) to obtain ##EQU45##
Completing the squares for .parallel.Z(k).parallel. in (3.77) similar to algorithm (a) results in .DELTA.J.ltoreq.0 as long as the conditions in (3.63) through (3.65) are satisfied and with the upper bound on the tracking error given by ##EQU46##
On the other hand, completing the squares for .parallel.r(k).parallel. in (3.77) results in .DELTA.J.ltoreq.0 as long as the conditions (3.63) through (3.65) are satisfied and ##EQU47## In general .DELTA.J.ltoreq.0 as long as (3.63) through (3.65) are satisfied and either (3.80) or (3.81) holds. This demonstrates that the tracking error and the error in weight estimates are GUUB.
Note that the NN reconstruction error bound .epsilon..sub.N and the bounded disturbances d.sub.M increase the bounds on .parallel.r(k).parallel. and .parallel.Z(k).parallel. in a very interesting way. Note that small tracking error bounds, but not arbitrarily small, may be achieved by placing the closed-loop poles inside the unit circle and near the origin through the selection of the largest eigenvalue, k.sub.vmax. On the other hand, the NN weight error estimates are fundamentally bounded by Z.sub.max, the known bound on the ideal weights W. The parameter .GAMMA. offers a design tradeoff between the relative eventual magnitudes of .parallel.r(k).parallel. and .parallel.Z(k).parallel.; a smaller .GAMMA. yields a smaller .parallel.r(k).parallel. and a larger .parallel.Z(k).parallel., and vice versa.
The effect of adaptation gains .alpha..sub.1, .alpha..sub.2, and .alpha..sub.3 at each layer on the weight estimation error Z(k), and tracking error r(k), can be easily observed by using the bounds presented in (3.80) and (3.81) through c.sub.min and c.sub.max. Large values of .alpha..sub.1, and .alpha..sub.2 forces smaller weight estimation error whereas the tracking error is unaffected. In contrast, a large value of .alpha..sub.3 forces smaller tracking and weight estimation errors. The next corollary, by using the improved weight tuning updates, extends the stability analysis presented for a three-layer NN in Theorem 3.5 to an n-layer NN.
Corollary 3.6
Assume the same hypotheses as Theorem 3.5, and consider the modified weight tuning algorithms provided for the input and hidden layers as
W.sub.1 (k+1)=W.sub.1 (k)-.alpha..sub.1 .phi..sub.1 (k)y.sub.1.sup.T (k)-.GAMMA..parallel.I-.alpha..sub.1 .phi..sub.1 (k).phi..sub.1.sup.T (k).parallel.W.sub.1 (k), .A-inverted.i=1, . . . , n-1, (3.83)
and the modified weight update for the output layer is given by either
a) W.sub.n (k+1)=W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)f.sup.T (k)-.GAMMA..parallel.I-.alpha..sub.n .phi..sub.n (k).phi..sub.n.sup.T (k).parallel.W.sub.n (k), (3.84)
or
b) W.sub.n (k-1)=W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)r.sup.T (k+1)-.GAMMA..parallel.I-.alpha..sub.n .phi..sub.n (k).phi..sub.n.sup.T (k).parallel.W.sub.n (k), (3.85)
with .GAMMA.>0 a design parameter. Then, the filtered tracking error r(k) and the NN weight estimates W.sub.i (k); .A-inverted.i=1, . . . , n are GUUB provided the following conditions hold: ##EQU48## for algorithm (a) and ##EQU49## for algorithm (b) where .eta. is given in (3.39) for algorithm (a) and .sigma. for algorithm (b) is given by ##EQU50## with .eta. in (3.89) given in (3.40). Remarks: No PE condition is needed with the modified tuning algorithms.
Proof:
Algorithm (a): Select the Lyapunov function candidate (3.43). Considering the input and hidden (3.83), output (3.84) weight updates as well as the tracking error dynamics (3.41), using these in (3.44), and completing the squares for .parallel.Z(k).parallel., one may obtain ##EQU51## Then .DELTA.J.ltoreq.0 as long as (3.86) through (3.88) hold and the quadratic term for r(k) in (3.90) is positive, which is guaranteed when (3.73) is satisfied with .lambda. .rho. in (3.90) are given in (3.91) and (3.92) respectively.
Similarly completing the squares for .parallel.r(k).parallel. in (3.90), the first difference is ##EQU52## where .rho. is given in (3.92). Then .DELTA.J.ltoreq.0 as long as (3.86) through (3.88) hold and the quadratic term for .parallel.Z(k).parallel. in (3.93) is positive, which is guaranteed when (3.75) is satisfied. This demonstrates that the tracking error r(k) and the error in weight estimates Z(k) are GUUB.
Algorithm (b): Select the Lyapunov function candidate (3.43). The first difference is given by using (3.83), (3.85) and (3.41) as ##EQU53##
Proof follows similarly to that of algorithm (a).
4. Passivity Properties of the NN
The closed-loop error system (3.7) is shown in FIG. 3, for instance when (3.36) is employed (the structure for (3.35) is the same); note that the NN now is in the standard feedback configuration as opposed to the NN controller in FIG. 2. Passivity is essential in a closed-loop system as it guarantees the boundedness of signals, and hence suitable performance, even in the presence of additional unforeseen bounded disturbances (i.e., NN robustness). Therefore, in this section the passivity properties of the multilayer NN, and hence of the closed-loop system, are explored for various weight tuning algorithms. Note the input and the hidden layer weight update laws employed for algorithms (a) and (b) are same.
Theorem 4.1
(i) The weight tuning algorithms (3.10) and (3.11) make the map from W.sub.1.sup.T to W.sub.1.sup.T, and W.sub.2.sup.T to W.sub.2.sup.T, both passive maps.
(ii) The weight tuning algorithms (3.13), (3.14) make the map from, (W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)) for the case of algorithm (a), and (k.sub.v r(k)+W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)) for the case of algorithm (b), to -W.sub.3.sup.T (k).phi..sub.3 (k) a passive map.
Proof:
(i) Define a Lyapunov function candidate ##EQU54## where first difference is given by ##EQU55## Submitting the weight update law (3.10) in (4.2) to obtain
.DELTA.J=-(2-.alpha..sub.1 .phi..sub.1.sup.T (k).phi..sub.1 (k)(W.sub.1.sup.T (k).phi..sub.1 (k)).sup.T (W.sub.1.sup.T (k).phi..sub.1 (k))+2(1-.alpha..sub.1 .phi..sub.1.sup.T (k).phi..sub.1 (k)) (W.sub.1.sup.T (k).phi..sub.1 (k)).sup.T (W.sub.1.sup.T (k).phi..sub.1 (k))+.alpha..sub.1 (.phi..sub.1.sup.T (k).phi..sub.1 (k))(W.sub.1.sup.T (k).phi..sub.1 (k).phi..sub.1 (k)).sup.T (W.sub.1.sup.T (k).phi..sub.1 (k))(4.3)
Note (4.3) is in power form (2.11) as long as the condition (3.16) holds. This demonstrates the passivity of the update law (3.10).
Similarly, it can be shown by using the hidden layer update (3.11) that the weight tuning (3.11) is in fact a passive map.
(ii) Algorithm (a): Define a Lyapunov function candidate ##EQU56## whose first difference is given by ##EQU57## Substituting the weight update law (3.13) in (4.5) yields ##EQU58## Note (4.6) is in power form (2.11) as long as the condition (3.16) holds. This in turn guarantees the passivity of the weight tuning mechanism (3.13).
Algorithm (b): Select the Lyapunov function candidate (4.4). Use (3.14) in (4.5) to obtain ##EQU59## which is in power form (2.11) for discrete-time systems as long as the condition (3.16) holds.
The next result demonstrates the passivity properties for NN with an arbitrary number of hidden layers.
Corollary 4.2
(i) The weight tuning algorithms (3.33) for the input and hidden layers in a n-layer NN make the maps from W.sub.i.sup.T to W.sub.i.sup.T (k); .A-inverted.i=1, . . . , n-1, passive maps
(ii) The weight tuning algorithm (3.35), (3.36) make the map from, (W.sub.n.sup.T.phi..sub.n (k)+.epsilon.(k)+d(k)) for the case of algorithm (a), and (k.sub.v r(k)+W.sub.n.sup.T .phi..sub.n (k)+.epsilon.(k)+d(k)) for the case of algorithm (b), to -W.sub.n.sup.T (k).phi..sub.n (k) a passive map,
Proof:
(i) Define a Lyapunov function candidate ##EQU60## whose first difference is given by ##EQU61## Substituting the weight update law (3.33) in (4.9) to obtain
.DELTA.J.sub.i =-(2-.alpha..sub.i .phi..sub.i.sup.T (k).phi..sub.i (k)(W.sub.i.sup.T (k).phi..sub.i (k)).sup.T (W.sub.i.sup.T (k).phi..sub.i (k)).sup.T (W.sub.i.sup.T (k).phi..sub.i (k))+2(1-.alpha..sub.1 .phi..sub.i.sup.T (k).phi..sub.1 (k)) (W.sub.i.sup.T (k).phi..sub.i (k)).sup.T (W.sub.i.sup.T (k).phi..sub.i (k))+.alpha..sub.i (.phi..sub.i.sup.T (k).phi..sub.i (k)).sup.T (W.sub.i.sup.T (k).phi..sub.i (k)); .A-inverted.i=1, . . . , n-1. (4.10)
Note (4.10) is in power form (2.11) as long as the condition (3.37) holds. This demonstrates the passivity of the update law (3.33) used for the input and hidden layers.
(ii) Algorithm (a): Define a Lyapunov function candidate ##EQU62## whose first difference is given by ##EQU63## Note (4.13) is in power form (2.11) as long as the condition (3.37) holds. This in turn guarantees the passivity of the weight tuning mechanism (3.35) for the output layer.
Algorithm (b): Select the Lyapunov function candidate (4.11). Use (3.36) in (4.12) to obtain ##EQU64## which is in power form (2.12) for discrete-time systems as long as the condition (3.37) holds.
Thus, the weight error block is passive and the closed-loop filtered tracking error system (3.7) in FIG. 3 is dissipative (see Theorem 2.2); this guarantees the dissipativity of the closed-loop system [21]. By employing the passivity theorem [2.1], one can conclude that the input/output signals of each block are bounded as long as the external inputs are this does not yield boundedness of the internal states of the lower block (i.e., W.sub.1 (k); .A-inverted.i=1, . . . , n) unless PE holds.
The next Theorem shows why PE is not needed with the modified update algorithms of Theorem 3.5.
Theorem 4.3
(i) The modified weight tuning algorithms (3.59) and (3.60) make the map from W.sub.i.sup.T to W.sub.i.sup.T (k), and W.sub.2.sup.T (k), both state strict passive maps.
(ii) The weight tuning mechanisms (3.61), (3.62) for a three-layer NN make the map from, (W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)) for the case of algorithm (a), and (k.sub.v r(k)+W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)) for the case of algorithm (b), to -W.sub.3.sup.T (k).phi..sub.3 (k) a state strict passive map,
Proof:
(i) The revised dynamics relative to input W.sub.1 (k), and hidden layer, W.sub.2 (k), error in weight estimates are given by
W.sub.1 (k+1)=[I-.alpha..sub.1 .phi..sub.1.sup.T (k)]W.sub.1 (k)+.alpha..sub.1 .phi..sub.1 (k)(W.sub.1.sup.T .phi.(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.1 .phi..sub.1 (k).phi..sub.1.sup.T (k).parallel.W.sub.1 (k), (4.15)
W.sub.2 (k+1)=[I-.alpha..sub.2 .phi..sub.2.sup.T (k)]W.sub.2 (k)+.alpha..sub.2 .phi..sub.2 (k)(W.sub.2.sup.T .phi.(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.2 .phi..sub.2 (k).phi..sub.2.sup.T (k).parallel.W.sub.2 (k), (4.16)
Select the Lyapunov function candidate (4.1) and use (4.15) in (4.2) to obtain ##EQU65## Note (4.19) is in power form for discrete-time systems given by (2.11) with g(k) a monic quadratic function of the state.
Similarly for the hidden layer, it can be easily shown that the weight update law is in fact a state strict map.
(ii) Algorithm (a): The revised dynamics for the output layer relative to W.sub.3 (k) are given by
W.sub.1 (k+1)=[I-.alpha..sub.3 .phi..sub.3.sup.T (k)]W.sub.3 (k)+.alpha..sub.3 .phi..sub.1 (k)(W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.3 .phi..sub.3 (k).phi..sub.3.sup.T (k).parallel.W.sub.3 (k), (4.20)
Select the Lyapunov function candidate (4.4) and use (4.20) in (4.5) to obtain ##EQU66## Note (4.21) is in power form for discrete-time systems given by (2.11) with g(k) a monic quadratic function of the state.
Algorithm (b): The revised dynamics for the output layer relative to W.sub.3 (k) are given by
W.sub.1 (k+1)=[I-.alpha..sub.3 .phi..sub.3.sup.T (k)]W.sub.3 (k)+.alpha..sub.3 .phi..sub.1 (k)(W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.3 .phi..sub.3 (k).phi..sub.3.sup.T (k).parallel.W.sub.3 (k), (4.24)
Select the Lyapunov function candidate (4.4) and using (4.24) in (4.5) yields ##EQU67## where .chi. and .gamma..sub.0.sup.2 are given in (4.22) and (4.23) respectively. Note (4.25) is in power form for discrete-time systems given by (2.11).
The next result demonstrates the passivity properties for NN with an arbitrary number of hidden layers.
Corollary 4.4
(i) The modified weight tuning of algorithms (3.82) for the input and hidden layers in a n-layer NN make the maps from W.sub.i.sup.T to W.sub.i.sup.T (k); .A-inverted.i=1, . . . , n-1, state strict passive mass.
(ii) The weight tuning mechanisms (3.83) and (3.84) for a n-layer NN make the map from, (W.sub.n.sup.T .phi..sub.n (k)+.epsilon.(k)+d(k)) for the case of algorithm (a), and (k.sub.v r(k)+W.sub.n.sup.T .phi..sub.n (k)+.epsilon.(k)+d(k)) for the case of algorithm (b), to -W.sub.n.sup.T (k).phi..sub.n (k) a state strict passive map,
Proof:
(i) The revised dynamics relative to input and hidden layers, -W.sub.i (k); .A-inverted..sub.i =1, . . . n-1, error in weight estimates are given by
W.sub.i (k+1)=[I-.alpha..sub.3 .phi..sub.3.sup.T (k)]W.sub.3 (k)+.alpha..sub.i .phi..sub.i (k)(W.sub.3.sup.T .phi..sub.3 (k)+.epsilon.(k)+d(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.i .phi..sub.i (k).phi..sub.i.sup.T (k).parallel.W.sub.i (k), .A-inverted.i=1, . . . , n-1. (4.26)
Select the Lyapunov function candidate (4.8) and use (4.26) in (4.9) to obtain ##EQU68## Note (4.27) is in power form for discrete-time systems given by (2.11) with g(k) a monic quadratic function of the state.
Similarly for the hidden layer, it can be easily shown that the weight update law is in fact a state strict map.
(ii) Algorithm (a): The revised dynamics for the output layer relative to W.sub.n (k) are given by
Select the Lyapunov function candidate (4.11) and use (4.30) in (4.12) to obtain ##EQU69## Note (4.31) is in power form for discrete-time systems given by (2.12) with g(k) a monic quadratic function of the state.
Algorithm (b): The revised dynamics for the output layer relative to W.sub.n (k) are given by
W.sub.n (k+1)=[I-.alpha..sub.n .phi..sub.n.sup.T (k)]W.sub.n (k)+.alpha..sub.n .phi..sub.n (k)(W.sub.3.sup.T .phi..sub.n (k)+.epsilon.(k)+d(k)).sup.T +.GAMMA..parallel.I-.alpha..sub.i .phi..sub.i (k).phi..sub.i.sup.T (k).parallel.W.sub.n (k) (4.34)
Select the Lyapunov function candidate (4.11) and using (4.34) in (4.12) yields ##EQU70## where .chi..sub.n and .gamma..sub.n0.sub.2 are given in (4.32) and (4.33) respectively. Note (4.35) is in power form for discrete-time systems given by (2.11) with g(k) a monic quadratic function of the state.
It should be noted that SSP of both the system dynamics and the weight tuning block does guarantee SSP of the closed-loop system, so that the norms of the internal states are bounded in terms of the power delivered to each block. Then, boundedness of input/output signals assures state boundedness even without PE.
A NN in defined to be passive if, in the error formulation, it guarantees the passivity of the lower subsystem in FIG. 3. Then, an extra PE condition is needed to guarantee boundedness of the weights. A NN is defined to be robust if, in the error formulation, it guarantees the state strict passivity of the lower subsystem in FIG. 3. Then, no extra PE condition is needed for boundedness of the weights. Note that (1) dissipativity of the error system is needed in addition to tracking stability to guarantee bounded weights, and (2) the NN passivity properties are dependent on the weight tuning algorithm used.
5. SIMULATION RESULTS
In order to illustrate the performance of the NN controller, a continuous-time nonlinear system is considered and the objective is to control this MIMO system by using a digital NN controller. Note that it is extremely difficult to discretize a nonlinear system and therefore to offer stability proofs. The second objective is to demonstrate that the learning rate for the delta rule employed at each layer in fact decreases with an increase in the number of hidden-layer neurons in that layer. Finally, it is shown that the improved weight tuning mechanisms make the NN weights bounded without the need for PE, and can allow fast tuning even for large NN when a projection algorithm is employed in conjunction with the modified weight tuning updates.
Note that the NN controllers derived herein require no a priori knowledge of the dynamics of the nonlinear system, not even the structure of the system being controlled, unlike conventional adaptive control nor is any learning phase needed.
Consider the nonlinear system described by
X.sub.1 =X.sub.2
X.sub.2 =F(X.sub.1, X.sub.2)+U, (5.1)
where X.sub.1 =[x.sub.1, x.sub.2 ].sup.T, X.sub.2 =[x.sub.3, x.sub.4 ].sup.T, U=[u.sub.1, u.sub.2 ].sup.T and the nonlinear function is (5.1) is described by F(X.sub.1, X.sub.2)=[M(X.sub.1)].sup.-1 G(X.sub.1, X.sub.2), with ##EQU71## The parameters for the nonlinear system were selected as a.sub.1 =a.sub.2 =1, b.sub.1 =2, and b.sub.3 =1. Desired sinusoidal ##EQU72## and Cosine inputs, ##EQU73## were preselected for the axis 1 and 2 respectively. The gains of the PD controller were chosen as k.sub.v =diag (20, 20) with .LAMBDA.=diag {5, 5} and a sampling interval of 10 msec was considered. A three-layer NN was selected with 4 input, 6 hidden and 2 output nodes. Sigmoidal activation functions were employed in all the nodes in the hidden layer. The initial conditions for X.sub.1 were chosen to be [0.5, 0.1].sup.T, and the weights were initialized to zero. No learning is performed initially to train the networks. The upper bound on the allowed adaptation gains .alpha..sub.1, .alpha..sub.2, and .alpha..sub.3 using (3.16) for the case of delta rule at each layer is computed to be 0.5, 0.32, and 0.5 respectively.
NN Controller with Delta Rule Weight Tuning and Projection Algorithm
The adaptation gains for the multilayer NN weight tuning are selected as .alpha..sub.1 =0.2, .alpha..sub.2 =0.01, and .alpha..sub.3 =0.1 for the case of the delta rule (3.10) through (3.4) and .xi..sub.1 =1.5 .xi..sub.2 =1.5, and .xi..sub.3 =0.7 with .zeta..sub.1 =.zeta..sub.2 =.zeta..sub.3 =0.001 for the case of the projection algorithm (3.10) through (3.14) with (3.53). FIGS. 4A and 4B and FIGS. 5A and 5B present the tracking responses of the controllers with delta rule and projection algorithm respectively. It is clear that the controller using the delta rule at each layer performs equally well with the projection algorithm when the value of the adaptation gain is small so that (3.16) is satisfied. However, large values of the weights initially were needed not only for the delta rule with small .alpha..sub.3 =0.1 (shown in FIG. 4B), but also for the projection algorithm with large adaptation gains for the case of projection algorithm, overshoots and undershoots are observed in the initial stages even though the tracking performance is extremely impressive.
FIG. 6A illustrates the response of the NN controller when the delta rule is employed with the adaptation gain .alpha..sub.3 in the last layer changed from 0.1 to 0.51. FIG. 6A, it is evident that the weight tuning using the delta rule at each layer becomes unstable at t=1.08 sec. Note that the representative weight estimates, as illustrated in FIG. 6B, of the NN are unbounded in this case. This demonstrates that the adaptation gain in the case of delta rule at each layer must decrease with an increase in the hidden-layer neurons. In fact, the theoretical limit implied by (3.16) in this case is .alpha..sub.3 =0.51, so that this bound appears to be a tight bound in general.
The performance of the NN controller was investigated while varying the adaptation gains at the output layer for the case of projection algorithm. FIG. 7A and FIG. 7B show the tracking response and some NN representative weight estimates of the NN controller with .xi..sub.1 =1.0, .xi..sub.2 =1.0, and .xi..sub.3 =0.1 with .zeta..sub.1 =.zeta..sub.2 =.zeta..sub.3 =0.001. As expected, the overshoots and undershoots have been totally eliminated but there appears to be a slight degradation in the performance. In other words at very small adaptation gains, overshoots and undershoots are not seen but there appears a slight degradation in the tracking performance with a slow and smooth convergence. On the other hand, at large adaptation gains overshoots are observed with a good tracking performance. As the adaptation gains are further increased, the oscillatory behavior continue to increase and finally the system becomes unstable. In other words, from the bounds presented in (3.17), as the adaptation gains are increased the margin of stability continues to decrease and at large adaptation gains (i.e. close to 1) the system becomes unstable. Thus, the simulation results conducted corroborate with the bounds presented in the previous sections.
NN Controller with Improved Weight Tuning and Projection Algorithm
In the case of projection algorithm (3.10) through (3.14) with (3.53), the weights in FIGS. 8A and 8B appear to be bounded, though this in general cannot be guaranteed without the PE condition. Therefore, the response of the controller with the improved weight tuning (3.59) through (3.62) with (3.53) is shown in FIG. 8B. The design parameter .GAMMA. is selected to be 0.01. Note that with the improved weight tuning, not only the tracking performance is improved, for instance in axis 2, but also the weights remain bounded without the necessity of PE. Finally, in all cases no initial NN training or learning phase was needed. In addition, the dynamics of the nonlinear system was not required to implement the NN controller as opposed to conventional adaptive control.
To study the contribution of the NN, FIG. 9 shows the response of the PD controller with no neural net. From FIG. 9, it is clear that the addition of the NN makes a significant improvement in the tracking performance.
6. CONCLUSION
A family of multilayer neural net controllers is developed for the control of nonlinear dynamical systems. The NN has a structure derived from passivity/filtered error notions and offers guaranteed performance. Weight updates using the delta rule at each layer were shown to yield a passive NN. Thus, a persistence of excitation condition is needed on t he internal signals NN signals. It was found that the adaptation gain in the case of the delta rule at each layer must decrease with an increase in the number of hidden-layer neurons in that layer so that learning must slow down for large NN.
In order to overcome the above deficiencies, a family of improved weight tuning algorithms was derived. The improved weight tuning paradigms consist of the weight updates used in the delta rule at each layer plus a correction term similar to e-modification approach in the case of continuous-time conventional adaptive control. The improved tuning algorithms make the NN state strict passive, so that bounded weights are guaranteed in practical non-ideal situations where PE may not hold. Furthermore, the adaptation gain is modified to obtain a projection algorithm so that the learning rate is independent of the number of hidden-layer neurons. At low adaptation gains, as expected, a smooth and slow convergence was observed with a slight degradation in tracking performance. On the other hand, at large adaptation gains, oscillatory behavior was seen with a good tracking performance and faster convergence. Finally, no NN learning or training phase was needed; simply initializing the NN weights at zero made for fast convergence and bounded errors.

Number	Name	Date
5119468	Owens	Jun 1992
5121467	Skeirik	Jun 1992
5159660	Lu et al.	Oct 1992
5471381	Khan	Nov 1995
5513098	Spall et al.	Apr 1996
5566065	Hansen et al.	Oct 1996

Discrete-time tuning of neural network controllers for nonlinear dynamical systems

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

US Classifications

Field of Search

US

International Classifications

Abstract

Description

Claims

CLAIM OF PRIORITY

US Referenced Citations (6)

Non-Patent Literature Citations (18)