Networked iterative learning control for discretetime systems with stochastic packet dropouts in input and output channels
 Jian Liu^{1} and
 Xiaoe Ruan^{1}Email author
DOI: 10.1186/s1366201711038
© The Author(s) 2017
Received: 6 July 2016
Accepted: 2 February 2017
Published: 17 February 2017
Abstract
The paper develops a derivativetype (Dtype) networked iterative learning control (NILC) scheme for repetitive discretetime systems with packet dropouts stochastically occurred in input and output communication channels. The scheme generates the sequential recursivemode control inputs by mending the dropped instantwise output with the synchronous desired output, while it drives the plant by refreshing the dropped instantwise control input with the used consensusinstant control input at the previous iteration. By adopting statistic technique, the convergences of the developed NILC scheme for linear and nonlinear systems are derived, respectively. The derivations present that under certain conditions the mathematical expectations of the stochastic tracking errors in the sense of 1norm converge to zero. Numerical simulations exhibit the effectiveness and validity.
Keywords
iterative learning control mathematical expectation networked control systems stochastic packet dropouts1 Introduction
In biology, psychology, sociology as well as in philosophy, the notion of ‘learning’ has been acknowledged as one of intelligent capabilities for an individual to earn food and fit the environment for surviving and evolving persistently. It is noted as a process for an intelligent agent to acquire knowledge or experience from its perception and cognition of the environment and then to act on the environment so as to improve its behavior performance at the next time. Benefited from the advancing computer technology, learning algorithm has been algorithmically embedded into the control programming of a robotic manipulator to track a desired trajectory. The pioneer contribution is the iterative learning control (ILC) invented in the 1980s whose scheme is to utilize the historical tracking discrepancy to modify its control command so that the upgraded control command may drive the repetitive system to track a predetermined desired trajectory [1]. Overviewing the existing ILC investigations, the ILC has been acknowledged as one of the most effective intelligent control strategies for a repetitive system operated over a fixed time interval owing to its less system information requirement and precise tracking insurance [2–8].
Along with the development of internet service, some of efficacious control schemes can be networked for higher efficiency and lower cost, which forms networked control systems (NCSs). However, confined by the physical features of the wire or wireless net communication devices such as the limit bandwidth or temporal oscillation of the net, the embedment of the communication net into the traditional control loop may possibly incur the communication delay and packet dropout which will deteriorate the control effects [9–13]. In terms of the communication delays, a usual manner is to replace the delayed data by the captured data at the last sampling instant in the case when the delay is within one sampling step length [9–11]. In treating the packet dropout, the method is to replace the dropped data with the latest captured one [12, 13]. It has been shown that the aforementioned handling methods work satisfactorily under the assumptions that the probabilities of the communication delay and the packet dropout are constrained appropriately.
Inspired by the handling methods for the NCSs, the investigations have been emerged to embed the network into the conventional ILC system addressing the communication delay and/or packet dropout. In detail, a Dtype NILC strategy has been considered for a class of linear timeinvariant (LTI) multipleinputmultipleoutput (MIMO) systems, where both the packet dropout and the communication delay of the system output are considered [14]. Reference [15] has addressed a proportionaltype (Ptype) NILC for a class of nonlinear systems with random packet losses happening in both the input and the output communication channels, where the term ‘packet losses’ is no other than communication delays. The handling methods for the delayed data in [14, 15] are to substitute the onestepdelayed data by the captured data at the last sampling instant, which is no other than the conventional NCSs [9–11]. As the abovementioned replacement mechanism of the communication delayed data is onestepahead mode, it to some extent does not match the ILC scheme which is an exact time pointtopoint mapping along iteration direction. As shown in [14, 15], the tracking error is asymptotically upperbounded but nonzero when communication delays occur. In addition, one [16] has developed a Ptype NILC scheme for a class of nonlinear systems with stochastic delays happened in both system output and control input communication channels, where the delayed data is replaced with the synchronous data of the previous iteration. It has been shown that the proposed NILC scheme can drive the NILC system to track the desired trajectory precisely as the iteration goes on.
Regarding the communication data dropouts, the paper [17] has proposed a Dtype NILC scheme for a class of LTI MIMO systems with packet dropout in the output channel and has deduced the convergence by Kalman filtering approach. Further work [18, 19] has considered a Dtype NILC algorithm for a general case that only part of the system output data stochastically drops but the remaining is successfully transmitted, which induces the learning gain by minimizing the trace of the input error covariance matrix or assigns it in the sense of mean square. Besides, the literature [20, 21] has presented Dtype NILC schemes for a class of discretetime systems with packet dropout occurring in the output channel, and in particular it [20] has analyzed the convergence on basis of exponential stability for asynchronous dynamical systems, while one [21] has derived the learning performance based on 2D model. Further relevant work [22] has adopted a Hinfinity measurement to assess the tracking performance of the NILC schemes for systems with packet dropout occurred in output channel. Recently, one [23] has developed a Dtype NILC algorithm for a class of singleinputsingleoutput (SISO) systems with system output packet dropout modeled as a 01 Bernoullitype Markov chain along the iteration axis. Under the assumption that for a fixed sampling instant the quantity of the successive packet loss is less than a constant, the learning gain has been designed as an iterationdecreasing sequence and the convergence has been deduced by stochastic approximation and optimization techniques. In further work [24] one has considered the NILC design for nonlinear systems with unknown control direction and system output packet dropouts. It is recalled that the handling strategy of dropped data proposed in [17–24] is equivalent to replacing the dropped data with the synchronous desired output signal. Meanwhile, [25] has developed an NILC scheme by replacing the dropped output with the successfully captured latest synchronous output.
It is observed that, however, the literature [17–25] only considers the packet dropout occurring in the output communication channel. As a matter of fact, the packet dropout occurs not only in the output communication channel, but also possibly in the input communication channel. Under this circumstance, the synchronous desired signal replacement in the existing literature [17–24] is hardly adoptable for the input dropout as the desired input is unavailable but pursued. Nevertheless, it is worth recalling that the learning capability of the ILC is principally benefited from the time pointtopoint compensation for the control input along the iteration direction rather than the time axis. Thus, the replacement for the dropped data by the captured latest synchronous ones would be a feasibility to deal with the dropped input. This motivates the paper.
This paper is to develop a Dtype NILC strategy for discretetime systems with both stochastic input and output packet dropouts. The strategy mends the dropped instantwise output with the synchronous desired output, while it refreshes the dropped instantwise input with the consensusinstant input used at the previous iteration. By means of the statistic technique, the convergence of the developed NILC scheme for respective linear and nonlinear systems is derived, which shows that under certain conditions the mathematical expectations of the stochastic tracking errors in the sense of 1norm converge to zero.
The rest of the paper is organized as follows. In Section 2, a Ptype NILC scheme is formulated and some notations are presented. Section 3 analyzes the convergence of the proposed NILC scheme to linear systems and Section 4 addresses the convergent characteristic of the proposed NILC scheme imposed on a kind of affine nonlinear systems. The effectiveness and the validity are numerically simulated in Section 5 and Section 6 concludes the paper.
2 NILC algorithm and notations
Let \((X,F,P)\) be a probability space and \(p \in [0,1]\) be a constant number, where \(X = \{ 0,1\}\) is a sample space, \(F = \{ \emptyset,\{ 0\},\{ 1\},\{ 0,1\} \}\) is a set of events and P is a probability measure on set F satisfying \(P(\emptyset ) = 0\), \(P(\{ 0\} ) = p\), \(P(\{ 1\} ) = 1  p\) and \(P(\{ 0,1\} ) = 1\), respectively. A stochastic variable ξ is said to be subject to 01 Bernoulli distribution refers that ξ is defined on \((\Omega,F,P)\) satisfying \(\xi (0) = 0\) and \(\xi (1) = 1\). Denote \(E\{ \xi \}\) as the mathematical expectation of the stochastic variable ξ. Then \(E\{ \xi \} = P(\xi = 1) = 1  p\). Let \(x = (x_{1}, \ldots,x_{n})^{ \top}\) and \(y = (y_{1}, \ldots,y_{n})^{ \top} \in R^{n}\) be two ndimensional real vectors. The partial order relation ≺ is defined as \(x \prec y\) if and only if \(x_{i} \le y_{i}\) for all \(i = 1,2, \ldots,n\). Let \(H = (h_{ij})_{m \times n} \in R^{m \times n}\) be a real matrix. Denote \( x  = ( x_{1} , x_{2} , \ldots, x_{n} )^{ \top}\), \( H  = ( h_{ij} )_{m \times n}\), \(\x \_{1} = \sum_{i = 1}^{n}  x_{i} \) and \(\ H \_{1} = \max_{1 \le j \le n}\sum_{i = 1}^{m}  h_{ij} \).
Remark 1
 (A1):

Assume that the stochastic variable \(\omega_{k,t}\) is independent on the variable \(\omega_{l,s}\) for all \(k \ne l\), \(s,t \in S^{ }\). Meanwhile, assume that the stochastic variable \(\alpha_{k,t}\) is independent upon the variable \(\alpha_{l,s}\) for all \(k \ne l\), \(s,t \in S^{ +}\). Besides, assume that \(\alpha_{k,t}\) is independent on \(\omega_{l,s}\) for all \(k = 1,2, \ldots\) , \(l = 2,3, \ldots\) , \(t \in S^{ +}\) and \(s \in S^{ }\).
 (A2):

Assume that the probabilities of packet dropout in the input and output channels are ω̄ and ᾱ, respectively, mathematically,$$\begin{aligned}& P\{ \omega_{k,t} = 0\} = \bar{\omega},\quad 0 \le \bar{\omega} < 1, \mbox{for } t \in S^{ }, k = 2,3, \ldots, \\& P\{ \alpha_{k,t} = 0\} = \bar{\alpha},\quad 0 \le \bar{\alpha} < 1, \mbox{for } t \in S^{ +}, k = 1,2, \ldots. \end{aligned}$$
The following lemmas are useful in this paper.
Lemma 1
Let \(\{ e_{k}\}_{k = 1}^{\infty}\), \(\{ \sigma_{k}\}_{k = 1}^{\infty}\) and \(\{ \varphi_{k}\}_{k = 1}^{\infty}\) be nonnegative sequences, which satisfy \(e_{k + 1} \le \sum_{i = 1}^{k} \sigma_{i}e_{k  i + 1} + \varphi_{k}\), \(\sigma = \sum_{i = 1}^{\infty} \sigma_{i} < 1\) and \(\lim_{k \to \infty} \varphi_{k} = 0\). Then \(\lim_{k \to \infty} e_{k} = 0\).
Proof
First, we prove that the nonnegative sequence \(\{ e_{k}\}_{k = 1}^{\infty}\) is bounded. Since the sequence \(\{ \varphi_{k}\}_{k = 1}^{\infty}\) is nonnegative satisfying \(\lim_{k \to \infty} \varphi_{k} = 0\) and \(\sigma = \sum_{i = 1}^{\infty} \sigma_{i} < 1\), there exists a positive integer \(K_{1}\) such that \(\varphi_{k} + \sigma < 1\) for all \(k \ge K_{1}\). Let \(C = \max \{ e_{1},e_{2}, \ldots,e_{K_{1}},1\}\). Thus, \(e_{k} \le C\) for all \(k \ge K_{1} + 1\), i.e., the nonnegative sequence \(\{ e_{k}\}_{k = 1}^{\infty}\) is bounded. The proof is accomplished by induction.
Lemma 2
Proof
From \(\lim_{k \to \infty} \lambda_{k} = 0\), it follows that the nonnegative sequence \(\{ \lambda_{k}\}_{k = 1}^{\infty}\) is bounded. Let \(C = \sup_{k = 1,2, \ldots} \{ \lambda_{k}\}\) and \(\phi = \sum_{k = 1}^{\infty} \phi_{k}\). Since the sequence \(\{ \phi_{k}\}_{k = 1}^{\infty}\) is nonnegative and we have the assumption (ii), it is true that for any \(\varepsilon > 0\) there exists a positive integer \(K_{1}\) such that \(\sum_{k = K_{1} + 1}^{\infty} \phi_{k} < \frac{\varepsilon}{3C}\). In addition, from the assumption (i) it is immediate that there exists a positive integer \(K_{2}\) (\(K_{2} > K_{1}\)) so that \(\lambda_{k  K_{1} + 1} < \frac{\varepsilon}{3\phi}\) for all \(k  K_{1} + 1 > K_{2}\). Further, the assumptions that \(\{ \Phi_{k} \}_{k = 1}^{\infty}\) is nonnegative and \(\lim_{k \to \infty} \Phi_{k} = 0\) imply that there exists a positive integer \(K_{3}\) such that \(\Phi_{k} < \frac{\varepsilon}{3}\) for all \(k > K_{3}\).
3 Convergence analysis for LTI SISO systems
Theorem 1
Assume that the proposed NILC scheme (4) with (2) and (3) is applied to the system (8) and the initial state is resettable, namely, \(x_{k}(0) = x_{d}(0)\) for all \(k = 1,2, \ldots \) . Then the expectation \(E\{\ \delta y_{k}\_{1}\}\) of the tracking error \(\ \delta y_{k}\_{1}\) is convergent to zero as the iteration goes on if the inequality \(\rho_{1} = \ E\{  I  \Gamma \Lambda_{k}H\Omega_{k} \} \_{1} + (1  \bar{\alpha} )\bar{\omega}  \Gamma \ H \_{1} < 1\) holds.
Remark 2
By the assumptions 1 and 2, \(\ E\{  I  \Gamma \Lambda_{k}H\Omega_{k} \} \_{1}\) is a constant and is independent of k, i.e. \(\rho_{1}\) is a constant number and is independent of k.
Remark 3
For a given k (\(k \ge 2\)), \(\ \delta y_{k} \_{1} = \sum_{t = 0}^{N}  y_{d}(t)  y_{k}(t) \) is a nonnegative stochastic variable and dependent upon \(\{ \alpha_{i,t}:t \in S,1 \le i \le k  1\}\) and \(\{ \omega_{i,t}:t \in S^{ },1 \le i \le k\}\). Thus, \(E\{ \ \delta y_{k} \_{1}\}\) can be understood as the expectation of the stochastic variable \(\ \delta y_{k} \_{1}\) or the firstorder deviation of the stochastic output \(y_{k}(t)\) (\(t \in S\)) with respect to the desired output \(y_{d}(t)\) (\(t \in S\)).
Proof
Corollary 1
Remark 4
Remark 5
Remark 6
As shown in (15), \(\delta \tilde{u}_{k + 1}\) involves all the past signals \(\{ \delta \tilde{u}_{i}:1 \le i \le k\}\) and its dynamics is quite complex.
4 Convergence characteristics of nonlinear systems
 (A3):

Assume that the nonlinear function \(f(z)\) is uniformly globally Lipschitz with respect to z, i.e., for all \(z_{1}, z_{2} \in R^{n}\), there exists a positive constant \(L_{f}\) such that$$\bigl\Vert f(z_{1})  f(z_{2}) \bigr\Vert _{1} \le L_{f} \Vert z_{1}  z_{2} \Vert _{1}. $$
Theorem 2
Assume that the proposed NILC scheme (4) with (2) and (3) is applied to the nonlinear system (24) and the initial state is resettable, i.e., \(x_{k}(0) = x_{d}(0)\) for \(k = 1,2, \ldots \) . Then, under the assumptions (A1), (A2), and (A3), the expectation \(E\{ \ \delta y_{k} \_{1}\}\) of the tracking error \(\ \delta y_{k} \_{1}\) converges to zero as the iteration tends to infinity if \(L_{f} < 1\) and \(\rho_{2} = \ E\{  I  \Gamma \Lambda_{k}\overline{C}\overline{B}\Omega_{k} \} \_{1} + (1  \bar{\alpha} ) \Gamma \ \overline{B} \_{1}\ \overline{C} \_{1} (\bar{\omega} + \frac{L_{f}}{1  L_{f}}) < 1\) are satisfied.
Proof
5 Numerical simulations
For the sake of exhibiting the effectiveness of the proposed learning scheme, the simulations are done for the systems being linear and nonlinear, respectively, where the tracking error is formulated as \(\ \delta y_{k}\_{1} = \sum_{t = 0}^{30}  y_{d}(t)  y_{k}(t) \). In accordance with the derivation that the tracking error is evaluated in a statistical sense by mathematical expectation, the numerical experiments are made for 500 runs. Here, the terminology ‘one run’ means that the NILCdriven system operates 60 iterations until a perfect tracking is achieved. Namely, the expectation of system output \(y_{k}(t)\) is computed as \(E\{ y_{k}(t)\} = \frac{1}{500}\sum_{m = 1}^{500} y_{k}^{(m)}(t)\) and the expectation of tracking error is formulated as \(E\{ \ \delta y_{k} \_{1}\} = \frac{1}{500}\sum_{m = 1}^{500} \sum_{t = 0}^{30}  y_{d}(t)  y_{k}^{(m)}(t) \), where the superscript (m) marks the run order.
Example 1
Example 2
From Figures 29, it is found that the proposed ILC scheme (4) with the compensations (2) and (3) may drive both the linear and the nonlinear systems to track the desired trajectory perfectly in statistical mode.
6 Conclusion
In this paper, a Dtype NILC scheme is developed for discretetime systems with appropriate mending manners for dropped input and output data. Under the assumption that the stochastic data dropouts are subject to 01 Bernoullitype distributions and by assessing the tracking performance in the form of mathematical expectation, the zeroerror convergences of the NILC for the SISO linear and affine nonlinear timeinvariant systems are derived, respectively. Both the theoretical derivations and the numerical simulations convey that the proposed NILC scheme enables the linear and affine nonlinear timeinvariant systems to track the desired trajectory well, though the stochastic dropout may disturb the tracking behavior. However, the investigations for the networked ILC systems with noise and parameter uncertainties are challenging in future work.
Declarations
Acknowledgements
The authors sincerely appreciate the support of the National Natural Science Foundation of China under granted No. F01011460974140 and 61273135.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Arimoto, S, Kawamura, S, Miyazaki, F: Bettering operation of robots by learning. J. Robot. Syst. 1(2), 123140 (1984) View ArticleGoogle Scholar
 Bristow, DA, Tharayil, M, Alleyne, AG: A survey of iterative learning control. IEEE Control Syst. Mag. 26(3), 96114 (2006) View ArticleGoogle Scholar
 Chen, Y, Moore, KL, Yu, J, Zhang, T: Iterative learning control and repetitive control in hard disk drive industry  a tutorial. Int. J. Adapt. Control Signal Process. 22(4), 325343 (2008) View ArticleMATHGoogle Scholar
 Mi, CT, Lin, H, Zhang, Y: Iterative learning control of antilock braking of electric and hybrid vehicles. IEEE Trans. Veh. Technol. 54(2), 486494 (2005) View ArticleGoogle Scholar
 Chen, Y, Wen, C, Gong, Z, Sun, M: An iterative learning controller with initial state learning. IEEE Trans. Autom. Control 44(2), 371376 (1999) MathSciNetView ArticleMATHGoogle Scholar
 Park, KH, Bien, Z: Intervalized iterative learning control for monotonic convergence in the sense of supnorm. Int. J. Control 78(15), 12181227 (2005) MathSciNetView ArticleMATHGoogle Scholar
 Ruan, XE, Wang, Q: Convergence properties of iterative learning control processes in the sense of the LebesgueP norm. Asian J. Control 14(4), 10951107 (2012) MathSciNetView ArticleMATHGoogle Scholar
 Bu, XH, Hou, ZS: Adaptive iterative learning control for linear systems with binaryvalued observations. IEEE Trans. Neural Netw. Learn. Syst. (2016). doi:10.1109/TNNLS.2016.2616885. Google Scholar
 Krtolica, R, Ozguner, U, Chan, H, Goktas, H, Winkelman, J, Liubakka, M: Stability of linear feedbacksystems with random communication delays. Int. J. Control 59(4), 925953 (1994) MathSciNetView ArticleMATHGoogle Scholar
 Yang, FW, Wang, ZD, Hung, YS, Gani, M: Hinfinity control for networked systems with random communication delays. IEEE Trans. Autom. Control 51(3), 511518 (2006) View ArticleGoogle Scholar
 Wen, DL, Yang, GH: Dynamic output feedback Hinfinity control for networked control systems with quantisation and random communication delays. Int. J. Syst. Sci. 42(10), 17231734 (2011) MathSciNetView ArticleMATHGoogle Scholar
 Wu, J, Chen, TW: Design of networked control systems with packet dropouts. IEEE Trans. Autom. Control 52(7), 13141319 (2007) MathSciNetView ArticleGoogle Scholar
 Wang, D, Wang, JL, Wang, W: Hinfinity controller design of networked control systems with Markov packet dropouts. IEEE Trans. Syst. Man Cybern. Syst. 43(3), 689697 (2013) View ArticleGoogle Scholar
 Liu, C, Xu, J, Wu, J: Iterative learning control for remote control systems with communication delay and data dropout. Math. Probl. Eng. 2012, Article ID 705474 (2012) MathSciNetMATHGoogle Scholar
 Bu, XH, Yu, FS, Hou, ZS, Wang, FZ: Iterative learning control for a class of nonlinear systems with random packet losses. Nonlinear Anal., Real World Appl. 14(1), 567580 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Liu, J, Ruan, XE: Networked iterative learning control approach for nonlinear systems with random communication delay. Int. J. Syst. Sci. 47(16), 39603969 (2016) MathSciNetView ArticleMATHGoogle Scholar
 Ahn, HS, Chen, Y, Moore, KL: Intermittent iterative learning control. In: Proceedings of the 2006 IEEE International Conference on Intelligent Control, Munich, Germany, pp. 144149 (2006) Google Scholar
 Ahn, HS, Moore, KL, Chen, YQ: Discretetime intermittent iterative learning control with independent data dropouts. In: Proceedings of 17th IFAC World Congress, Seoul, Korea, pp. 1244212447 (2008) Google Scholar
 Ahn, HS, Moore, KL, Chen, YQ: Stability of discretetime iterative learning control with random data dropouts and delayed controlled signals in networked control systems. In: Int. Conf. Control., Autom., Robot. Vis., ICARCV Hanoi, Vietnam, pp. 757762 (2008) Google Scholar
 Bu, XH, Hou, ZS: Stability of iterative learning control with data dropouts via asynchronous dynamical system. Int. J. Autom. Comput. 8, 2936 (2011) View ArticleGoogle Scholar
 Bu, XH, Hou, ZS, Jin, ST, Chi, RH: An iterative learning control design approach for networked control systems with data dropouts. Int. J. Robust Nonlinear Control 26(1), 91109 (2016) MathSciNetView ArticleMATHGoogle Scholar
 Bu, XH, Hou, ZS, Yu, FS, Wang, FZ: Hinf iterative learning controller design for a class of discretetime systems with data dropouts. Int. J. Syst. Sci. 45(9), 19021912 (2014) View ArticleMATHGoogle Scholar
 Shen, D, Wang, YQ: Iterative learning control for networked stochastic systems with random packet losses. Int. J. Control 88(5), 959968 (2015) MathSciNetMATHGoogle Scholar
 Shen, D, Wang, YQ: ILC for networked nonlinear systems with unknown control direction through random lossy channel. Syst. Control Lett. 77, 3039 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Liu, J, Ruan, XE: Networked iterative learning control for lineartimeinvariant systems with random packet losses. In: Proceeding of the 35th Chinese Control Conference, Chendu, China, pp. 3843 (2016) Google Scholar