Recursive recurrent neural network: A novel model for manipulator control with different levels of physical constraints

Manipulators actuate joints to let end effectors to perform precise path tracking tasks. Recurrent neural network which is described by dynamic models with parallel processing capability, is a powerful tool for kinematic control of manipulators. Due to physical limitations and actuation saturation of manipulator joints, the involvement of joint constraints for kinematic control of manipulators is essential and critical. However, current existing manipulator control methods based on recurrent neural networks mainly handle with limited levels of joint angular constraints, and to the best of our knowledge, methods for kinematic control of manipulators with higher order joint constraints based on recurrent neural networks are not yet reported. In this study, for the first time, a novel recursive recurrent network model is proposed to solve the kinematic control issue for manipulators with different levels of physical constraints, and the proposed recursive recurrent neural network can be formulated as a new manifold system to ensure control solution within all of the joint constraints in different orders. The theoretical analysis shows the stability and the purposed recursive recurrent neural network and its convergence to solution. Simulation results further demonstrate the effectiveness of the proposed method in end ‐ effector path tracking control under different levels of joint constraints based on the Kuka manipulator system. Comparisons with other methods such as the pseudoinverse ‐ based method and conventional recurrent neural network method substantiate the superiority of the proposed method.


| INTRODUCTION
Manipulators, which can greatly reduce the heavy burden on labour forces of workers, nowadays have been widely applied in many industrial fields such as welding, painting, and assembling areas. By using extra degrees of freedom (DOFs), manipulators are able to make use of inherent redundancy to perform flexible operations and fulfil complicated tasks [1,2]. Kinematic control of manipulators by taking advantage of such inherent redundancy in various circumstances have drawn intensive interests and concerns in recent years [3].
Generally, kinematic control of manipulators is to seek a suitable control action in the joint space that produces a desired motion for the end effector in the workspace.
However, strong coupled non-linearity exists in the mapping between a joint space and a Cartesian workspace of a redundant manipulator when it executes manipulation tasks. It is difficult to tackle this problem for analytical solutions in the joint space level through directly solving the coupled non-linear equations that are used to describe the kinematic characteristics of manipulators. Therefore, the kinematic control problem in the joint space is converted into a problem depicted by the velocity kinematic equations. Some early research studies found the control solutions directly by solving the pseudoinverse of the Jacobian matrix of a manipulator [4,5], and such a way of processing may increase local instability and even need more computational costs which were not expected.
To overcome the shortcomings of pseudoinverse-based approaches, optimisation-based (e.g. quadratic programming) methods have been proposed to find the optimal control solutions [6,7]. Such optimisation-based methods can involve physical constraints into the optimisation formulation [8] and solutions to such constrained optimisation problems cannot be obtained in an analytical manner as well. Thus, numerical methods in a way of serial processing were applied, but they are still resulting in low computational efficiency. In order to remedy this weakness, recurrent neural networks with parallel processing ability have been proposed for manipulator control [9][10][11][12][13][14][15][16][17][18]. Many researchers have developed kinematics control paradigms with the involvement of one or two of the three levels of constraints: joint angle and joint velocity, and joint acceleration constraints. In ref. [19], joint limits and joint velocity limits are taken into account for the kinematic resolution based on a dual neural network. In ref. [20], automatic handling of kinematic constraints is proposed in a real-time operational space motion planner. In ref. [21], repetitive motion planning is performed with acceleration-level constraints. In ref. [22], kinematic control of manipulators with obstacle avoidance in the joint-acceleration level is proposed with a minimum acceleration norm as the optimisation objective. In ref. [23], kinematic control of the manipulators with joint velocity constraints is solved by a passivity-based approach from an energy perspective. In ref. [24], the velocity-level and acceleration-level redundancy resolution are combined to form a unified quadratic programming for joint torque optimisation. In ref. [25], visual serving control of manipulators with joint physical limits to ensure safety. In ref. [26], both velocity-level and acceleration-level constraints are integrated into the kinematic control of manipulators. However, for joint constraints in a higher level such as more than the fourth-order joint constraints, kinematic control methods based on recurrent neural networks have not yet been proposed and reported.
Motivated by the aforementioned points, as current kinematic control methods based on recurrent neural networks can only deal with joint angular velocity and joint acceleration constraints, in this study, we are making breakthroughs to propose a novel recursive recurrent neural network for kinematic control of manipulators with different levels of physical constraints, with high-order joint constraints considered. The contributions of this study are summarised as follows: (1) To the best of our knowledge, this article is the first work to propose the kinematics control method for manipulators with different levels of physical constraints simultaneously. (2) The proposed method is described by a novel recursive recurrent neural network model, and the theoretical results on the purposed recursive recurrent neural network show the stability in a manifold system perspective and its convergence to kinematic control solutions with aforementioned physical constraints. (3) Simulation results with comparisons to the pseudoinversebased method and conventional recurrent neural network method on the Kuka manipulator system demonstrate the efficiency and superiority of the proposed method in kinematic control with different levels of physical constraints.

| PROBLEM FORMULATION
Let us consider the following velocity kinematics equation for manipulator control where J ∈ R m�n denotes the Jacobian matrix of the manipulator, r ∈ R m denotes trajectory in the Cartesian space of the end effector, and θ ∈ R n denotes the joint space variable. The end effector has to follow the desired path r d to make the tracking error e = r − r d as small as possible with suitable control actions from the joint space of the manipulator. Due to the physical restrictions of joint actuation of the manipulator, generally, the joint variable usually has to follow the limits in the joint angle level, joint velocity level, joint acceleration level, and higher order levels as follows

| CASCADED RECURSIVE RECURRENT NEURAL NETWORK
In order to achieve kinematic control of the manipulator with all levels of joint constraints, based on the aforementioned discussions, we have the equilibrium points in a different order as follows: where P Ω 1 ð⋅Þ; P Ω 2 ð⋅Þ; P Ω 3 ð⋅Þ; …; P Ω i ð⋅Þ; …; P Ω n−1 ð⋅Þ; P Ω n ð⋅Þ denote the piecewise linear projection function arrays for the solution sets Ω 1 , Ω 2 , Ω 3 , …, Ω i , …, Ω n−1 , Ω n in different levels of time derivatives for joint variables, and k 1 , k 2 , k 3 , …, k i , …, k n−1 , k n denote the convergence scaling parameters for desired equilibrium points in different levels of time derivatives for joint variables, which are all positive scalars. In order to track the aforementioned equilibrium points in different levels, we first define the following state variables for constructing a state-space dynamic model which can describe the time derivatives of different orders for the state variables of the recurrent neural network solver The state-space dynamic model is constructed as follows: Therefore, the equilibrium points can be described by the state variables as follows: Next, we define the congregation state variable vector as the following form Thus, we can get the following state-space dynamic equations where f(⋅) denotes the mapping function array which is composed of the linear projection function arrays P Ω 1 ð⋅Þ; P Ω 2 ðÞ; …; P Ω n−1 ð⋅Þ. The first equation is a fast-manifold relative to the second equation when k n ≫ 1. Applying a singular perturbation analysis [28], we conclude the following slow manifold _ x n−2 with the faster manifold _ x n−1 as follows: where 0 < 1/k n ≤ 1. We expect the manifold system, which is also a recursive recurrent neural network solver, can converge to the equilibrium points when dealing with the kinematic control of the manipulator system with high-level joint constraints involved as shown in Figure 1. In the recurrent neural networks with different solution sets, the higher order desired target variables are gradually generated as the inputs until the highest desired one is satisfied, and then the resolved joint angular variables in descending orders are generated until the resolved joint velocity variable is obtained. In this manner, lower order desired end-effector variable is acting as the input to the recurrent neural network model with a corresponding solution set to generate the output as the higher order desired end-effector variable, and higher order desired joint angular variable is acting as the input to the recurrent neural network model with the corresponding solution set to generate the output as the lower order joint angular variable. The overall computational complexity depends on the calling times of the primal-dual neural networks, but the dimension of the network variables is not increased which does not increase the complexity of each single constrained optimization. In the ensuing sections, we present the stability analysis and show the convergence properties.

| THEORETICAL ANALYSIS
In this section, we present a theoretical analysis on the solution of the control actions by the aforementioned recursive neural network based on the manifold system with stability and convergence results provided.

| Constraint compliance analysis
For the manifold system equations It can be rewritten as where solution set cone Ω 0 n is scaled by the parameter k n from the original solution set cone Ω n .
Therefore, the equations above can be equivalently Recall the stability properties of singular perturbation system, So such a recursive procedure repeats until the final equilibrium state is achieved, that is, _ x 0 → 0, and the desired path r d can be tracked by the manipulator system.

| Stability and convergence analysis of intermediate variables for high-order joint constraints
Review the second equation of the manifold system Now we analyse its stability and convergence for the solution. Define a Lyapunov function According to the definition of the piecewise linear projection function P Ω n ð⋅Þ, we have F I G U R E 1 Flow chart of the proposed recursive recurrent neural network model for kinematic control with different level of physical constraints satisfied LI AND LI That is Thus, we have As the following inequality always holds For the residual term, we have that is, which is a negative definite. According to Lyapunov theory, as the Lyapunov function V is a positive definite, the second equation of the manifold system is asymptotically stable. According to LaSalle's invariant set principle [28] and letting Since the following inequality holds we can obtain Review the definition of the linear projection operator P Ω n , the equation above is equivalent to arg min Therefore, we have That is For the inequality above, there exists a value ρ > 0 such that which yields Thus, we can get the following inequality Recall that the value of ρ can be chosen to be as small as possible and parameter k n is large enough, and thus the inequality holds for any ρ > 0. This implies that that is, and then we obtain Recursively, we have the following equations for the manifold system in the similar form and the convergence relationship which indicates x n−2 → w ðn−2Þ d with a small enough value ρ and a large enough value k n−1 .
Repeat the similar derivation procedure as for _ x n−3 ; _ x n−4 ; …; _ x 2 , we can get Repeat the same procedure until concerning _ x 1 , we can finally get with the convergence relationship showing that Generally, based on the aforementioned derivations, we can summarise that for the manifold system equation it can produce and we can conclude that x n−iþ1 ¼ w

| Stability and convergence analysis for the end effector in tracking path
After showing the stability and convergence of the state-space dynamic systems with intermediate variables for high-order constraints, we still need to analyse the stability and convergence properties for the end effector of the manipulator system to track the desired path. According to the velocity kinematics equation and the proposed recursive recurrent neural network solver, we define the term e = r − r d as the tracking error of the end effector of the manipulator, and we have As _ e ¼ _ r − _ r d ¼ _ r provided that the _ r d is constant, thus the equation above further becomes Define a Lyapunov function V = ‖e‖ 2 /2, and its time derivative is According to the definition of the projection operation, we have As the following inequality holds Thus, we have LI AND LI which leads to Therefore, we have which indicates that the tracking system is stable. Next, we again use LaSalle's invariant set principle [28] and force _ V ¼ 0, and then it yields As we can get The equation above can be equivalent to arg min Therefore, we have the following inequality That is There exists a value ρ > 0 such that F I G U R E 2 Tracking performance of circle and square paths by the proposed method.

-
which yields Therefore, we have Recall that the value of ρ can be chosen to be as small as possible and k 1 > 1 can be enlarged. This implies that that is, If the Jacobian matrix is full rank, which means the manipulator will not fall into the singularity situation during motion control process, then we have which indicates that the tracking error of the end effector of the manipulator can converge to zero.

| RESULTS
In this section, the proposed recursive recurrent neural network (10) based on the manifold system is utilised to control the manipulator with fifth-order joint variable constraints, that is, The desired targets for the manipulator to track are circle and square paths. In order to perform such kinematic control with these joint constraints, the resultant recursive recurrent neural network controller is According to aforementioned analysis, controller (62) depicted by recurrent neural network recursively updates to produce _ to perform the path tracking tasks and ensure the constraints of different orders are satisfied during control. The solution sets Ω 1 , Ω 2 , Ω 3 , Ω 4 and  Ω 5 are also generated to correspond the joint variable constraints in different orders.

| Simulation setup and parameter configuration
In the simulation, we evaluate the performance for the Kuka manipulator by the proposed method. The forward kinematics modelling of the manipulator is established based on the D-H table in ref. [2]. For the circle path tracking task, the radius of the circle is 0.15 m, and the parameters for the proposed recursive recurrent neural network are configured as follows: k 1 = 100, k 2 = 100k 1 ,  configured. From Figure 2a and 2b, we could observe that, the manipulator can fulfil the path tracking tasks well with its end effector promisingly following the positions of the paths. As seen from Figure 2c and 2d, the position errors e = [e x , e y , e z ] between the end effector and the desired paths can be rather small, with the range being from less than 10 � 10 −3 m for the circle path and less than 8 � 10 −3 m for the square path. These results demonstrate the efficiency of the proposed recurrent neural network solver based on the manifold system for kinematic control of the manipulator in the aspect of tracking accuracy.

| Joint constraints
In additional to fulfiling path tracking control, the manipulator system has to simultaneously follow the joint constraints in different orders. Figures 3 and 6 show the joint variables of different orders for circle and square path tracking tasks by the proposed method, that is, joint angles, joint angular velocity, joint angular acceleration, joint variable of third order, joint variable of fourth order, and joint variable of fifth order.
For the circle path tracking task, Figure 3a shows the revolved joint angle, and Together with the tracking results of the end effector in Section V-B for circle path tracking, the proposed method is demonstrated as an efficient way to deal with joint constraints in high order when fulfiling kinematic control. For comparisons, Figures 4 and 5 show the joint variables of different orders based on the pseudoinverse method and the conventional recurrent neural network method in the circle path tracking tasks, and it can be observed that, when encountering the higher order (>= 2) joint constraints (shown in Figures 4c-f and 5c-f, these two methods cannot work well as the higher order joint variables obviously exceed the corresponding joint limits. For the square path tracking task, Figure 6a shows the revolved joint angle, and Figure 6b  joint variable of the fourth order θ (4) falls into [ − 0.4 0.4] rad/ s 4 , and the joint variable of the fifth order θ (5) falls into [ − 0.5 0.5] rad/s 5 . Together with the tracking results of the end effector in Section V-B for square path tracking, the proposed method is demonstrated as an efficient way to deal with the joint constraints in high order when fulfiling kinematic control. Furthermore, Figures 7 and 8 show the joint variables of different orders based on the pseudoinverse method and the conventional recurrent neural network method in the square path tracking tasks, respectively and it can be observed that, when encountering the higher order (>= 2) joint constraints (shown in Figure 7c-f and Figure 8c-f), these two methods cannot work well as the higher order joint variables evidently exceed the corresponding joint limits.
To summarise, from all these results including comparisons, we can conclude that the proposed recursive neural network method is efficient and superior for the manipulator control with high-order joint constraints.

| CONCLUSION
Due to physical limitations and actuation saturation of manipulator joints, the involvement of joint constraints for kinematic control of manipulators is essential and critical. In this work, for the first time, a novel recursive recurrent network model is proposed to solve the kinematic control issue for manipulators with different levels of physical constraints, and the proposed recursive recurrent neural network model can be formulated as a new manifold system to ensure a control solution within all of the joint constraints in different orders. Theoretical analysis shows the stability and convergence of the purposed recursive recurrent neural network and its convergence of solution. Simulation results further demonstrate the effectiveness and superiority of the proposed method in endeffector path tracking control under different levels of joint constraints based on the Kuka manipulator system.