- Review
- Open Access
- Published:

# Survey of duality between linear quadratic regulation and linear estimation problems for discrete-time systems

*Advances in Difference Equations*
**volume 2019**, Article number: 90 (2019)

## Abstract

The linear estimation (or optimal control) problem in the emerging area of time-delay systems, multiplicative noise systems, and Markov jump linear systems has been taken into consideration by the control engineers. The purpose of this survey is to provide a review of duality between linear estimation and optimal control in the above-mentioned area. The mentioned subjects are studied to indicate the essential connection between control and estimation problems and to investigate the major cause that the classical duality still holds (not holds) for the broader systems, such as time-delay systems, multiplicative noise systems, Markov jump linear systems, and the like.

## Introduction

According to the duality principle, the linear quadratic regulation (LQR) problem for deterministic systems and the optimal estimation problem for additive noise including systems are dual [1]. According to this principle, the optimal estimation problem with the linear minimum mean square error (LMMSE) criterion could be converted to an optimal control problem with a quadratic performance index in the inverse time, and vice versa. The duality principle between control and estimate problems has been studied in the literature [2,3,4,5,6,7,8,9]. The duality principle between the LMMSE estimation problem for continuous systems containing multiple time delays appearing in a single observation channel and the LQR problem for continuous systems containing multiple input delays appearing in a single input channel has been established in [2]. Instead of using the information filter, a more general model of linear quadratic Gaussian duality has been established by using the Kalman–Bucy filter in [3]. Also, [5] discusses the duality between the LQR control with limited controller-system communication and the scalar Gauss Markov systems’ optimal schedule. Moreover, control and estimation problems have also generated considerable interest in the presence of constraints [6, 8]. The authors in [6] consider the constrained reference tracking’s control problem and show that there exists asymmetry in the estimation problem with that of the reference tracking problem. A natural question is: Does the classic duality between control and filtering of a linear system still hold for the Markov jump system, the multiplicative noise system, and the time-delay system? In the following we will give a survey of duality between the LQR and the linear estimation problem for discrete-time systems.

### Duality for time-delay systems

Over the past few decades, the study of time delays has exerted great interest of plenty of scholars [10,11,12,13,14]. According to different kinds of time delay, many results have sprung up, such as communication delay [15, 16], control input delay [17,18,19,20], state-dependent delay [21,22,23,24], state-independent delay [25, 26], output delay [27, 28], distributed delay [29,30,31,32], time-varying delay [33, 34]. For example, communication links are usually imperfect and the limited bandwidth may result in delays. The system performance degradation or system instability could be arise from time delays. As for the estimation or control problems for the systems with time delay, there are several methods to settle such as the (partial difference) Riccati equation [35, 36], the game-theoretic approach [37], the linear matrix inequality (LMI) method [38, 39], and the convex optimization approach [40, 41].

The state augmentation method [42] and Kalman filter [43] have been employed to verify the LMMSE problem for discrete time-delay systems. Nevertheless, an augmentation approach may often result in high system dimension causing high computational demand. To save the computing cost in solving the estimation problem for measurement delay including systems, a reorganized innovation tool has been proposed [44]. According to the proposed tool, the optimal filter is obtained using two standard Riccati difference equations with dimension similar to the original state model. Moreover, we expand the results of [44] to the systems with multiplicative noise [45,46,47]. The signal coming from measurement [44, 45] can be divided two types: delayed and instantaneous measurements. Systems studied in [44, 45] could be considered as special cases of systems with single measurement channel and multiple measurement delays [48,49,50]. The LMMSE estimation problem for these systems is so complicated that could not be directly solved using the reorganized innovation tool. Recently, a partial difference Riccati equation approach has been employed to derive the optimal estimators containing predictor, filter, and smoother in the linear minimum variance sense for discrete-time packet dropping systems undergoing bounded random measurement delays [48]. The LMMSE estimation problem has been studied for systems containing constant state delay \(d_{0}\), bounded random measurement delays, and packet dropping in [50].

In [51], smoothing estimation and duality concept have been employed to solve the LQR problem for linear time-invariant (LTI) systems with multiple input delays. Consequently, the authors in [35] established the duality between the LMMSE estimation problem for discrete systems containing multiple time delays appearing in a single observation channel and the LQR problem for discrete systems containing multiple input delays appearing in a single input channel. Unlike the works presented in [35, 51], in this study [36], a step-by-step optimization strategy has been proposed to deal with the control problem for universal time-delay systems. An associated dual backward stochastic system is defined to attain this goal. The optimal controller gains are calculated using a backward partial Riccati difference equation. After comparing the LMMSE estimator and the LQR controller results, we find that the duality between the LMMSE estimate problem for general time-delay discrete systems with additive noise and the LQR problem for general time-delay deterministic discrete systems with a quadratic criterion in the inverse time still holds, which extends the classic principle of duality. Both dual elements are clearly pointed out and the time inversion between the problems is indicated. A detailed discussion is carried out in Sect. 3.

### Duality for systems with time-delay and multiplicative noise

Multiplicative noise is inevitable in fading or reflection of transmitted signals over an ionospheric channel [52, 53]. Such examples often occur in communication systems [54], image processing [55], fault detection [56, 57], and the like. State estimation problem in the presence of multiplicative noise is important when the state is not directly measurable [58,59,60]. It is obvious that the system state will be a process without Gaussian noise and the optimal estimator which produces minimum error variance will be in nonlinear form when the system involves multiplicative noise; thus, practically, it will be very difficult to achieve. From the practical point of viewpoint, the construction of high quality and easy-to-implement suboptimal filters becomes crucial. An important case of suboptimal filters is the linear one with minimum mean square error (MMSE), which has been widely used in the past decades [61,62,63].

By formulating the observation uncertainty via an independent and identically distributed (i.i.d) Bernoulli process, the estimator equations have been derived [64]. In [65], the convergence and the asymptotic stability of the LMMSE filter for a special case of linear systems with measurements including both multiplicative and additive noises have been investigated. In [66] a finite-horizon robust Kalman filter has been proposed for discrete-time systems including stochastic uncertainties. The author in [67] investigates the linear estimation problem of a remote signal generated from a discrete-time periodic system subject to multiplicative noise. In [68] the linear optimal estimator has been constructed by presenting a different model for describing random delays and packet dropping. The estimation problem for multiplicative noise systems with measurement delay is considered in [69], where the multichannel multiplicative noise is represented by a diagonal matrix, and a sufficient condition for the existence of the steady filter is presented.

On the other hand, the LQR controller design for systems undergoing multiplicative noise has been a hot topic over the past decades [70, 71] and is actively studied nowadays as well. There are several ways to solve the control problems for systems containing multiplicative noise, including the spectrum analysis [72], the generalized Riccati equation method [73, 74], and the linear matrix inequality method [75,76,77,78]. In [79], the state estimation and optimal control problems for stochastic systems containing multiplicative and addictive noises have been extended to the filtering case. The state estimator is based on availability of the feedback signal. The authors in [80] studied the stochastic optimal control for discrete-time linear systems undergoing Markovian jumps and multiplicative noises. The LQR problem has been investigated for discrete-time systems with multichannel multiplicative noise describing with a diagonal matrix [81]. Stabilization and observability concepts have been employed to solve the infinite horizon linear quadratic optimal control problem for discrete-time stochastic systems subject to state and control-dependent noise [82].

For the LMMSE estimation problem, the estimator gains can be calculated via forward Riccati and Lyapunov equations [65, 67, 69]. Meanwhile, for the LQR problem associated with multiplicative noise systems, the controller gains can be yielded via one backward generalized Riccati equation [73, 81]. Therefore, under unconstrained conditions, the classic duality principle is not necessarily true for multiplicative noise systems; that is to say, the LMMSE estimate is not dual to the LQR control. Some conditions should be satisfied to hold the duality concept for time-delay multiplicative noise systems. We have discussed the duality between the LMMSE estimation for time-delay systems containing multiplicative noise, and the LQR problem for deterministic time-delay systems subject to constraint condition has been studied [83].

### Duality for Markov jump systems

Practically, some systems may undergo sudden changes in their structure and parameters and because of the environmental disturbance. These systems could be modeled as Markovian jump systems. Plenty of scholars have shown a great interest in studying the estimation problems for linear systems subject to Markovian jump. In these works, the LMI tool [84] and Riccati equation tool [85] are usually used to settle the estimation problems. The authors in [86] study a stationary LMMSE estimation problem for the linear systems with Markov jump, in which the jump parameters’ prior knowledge is unspecified. Apparently, it could be easily achieved, and the gain matrices could be calculated off-line. According to the discrete-time recursions [87], an estimator which is recursive and robust can be established for linear systems in addition to Markov jump with uncertain jump parameters. The model and the output measurement have been utilized to solve the state estimation problem for discrete-time Markov jump linear systems [88]. In [89], the finite-horizon LMMSE estimation problem for discrete-time linear systems which has measurement-delay and Markov jump is studied.

At the same time, the control problem for Markov jump systems has also been verified in the literature [90,91,92]. The book [93] introduces the discrete-time linear systems subject to Markov jump, including stochastic analysis and control theory. Solvability of the \(\mathcal{H}_{2}\) control problem for discrete-time periodic systems including Markovian jumps and multiplicative noise is studied in [94]. A natural question is: Under what conditions the classical duality still holds for Markov jump systems? Fortunately, recently [95] demonstrated that if the jump parameters satisfy \(\theta (t)=\eta (l-t)\), then the LMMSE estimation problem and the LQR problem are dual. The result beautifully extends the classic duality principle between filter and controller to the Markov jump systems. A detailed discussion will be presented in Sect. 5.

*Notation:* Throughout this paper, the superscripts “−1” and “*T*” represent the inverse and transpose of a matrix. \(\mathcal{R} ^{n}\) denotes the n-dimensional Euclidean space. \(\mathcal{R}^{n \times m}\) is the set of all \(n\times m\) real matrices. \(\delta _{ij}=0\) for \(i\neq j\) and \(\delta _{ii}=1\). The linear subspace spanning by the measuring samples \(\{y(0), \ldots , y(k)\}\) is specified by \(\mathcal{L} \{\{y(s)\}^{k}_{s=0} \}\). Moreover, the mathematical expectation operator is expressed with *E*. Furthermore, the matrix dimensions are assumed to be compatible such that algebraic operations could be written.

## Classical duality between quadratic optimal control and linear estimation

Consider a discrete-time linear dynamics described by the following state space equations:

where \(x(k)\in {\mathcal{R}}^{n}\), \(y(k)\in {\mathcal{R}}^{m}\), \({n}(k)\in {\mathcal{R}}^{n}\), and \({v}(k)\in {\mathcal{R}}^{m}\). \({ n}(k)\) and \({ v}(k)\) are zero mean white noises with covariance matrices \(E \{{n}(k) { n}^{T} (j)\} ={Q(k)}\delta _{kj}\), \(E\{{ v}(k) { v}^{T} (j)\} ={ R(k)}\delta _{kj}\), respectively. The random processes \({n}(k)\), \({v}(k)\) for all *k* and the initial states \({{x}}(0)\) are considered mutually independent.

### Lemma 2.1

([43])

*For system* (2.1)*–*(2.2), *the LMMSE one*-*step predictor is given by*

*where*

*and*
\(P(k)\)
*is obtained by solving the coming discrete Riccati equation*

Consider the discrete LQR problem for the following time-delay system:

associated with the quadratic performance index

where \({x}(k)\in {\mathcal{R}}^{n}\) and \({u}(k)\in {\mathcal{R}}^{m}\) are the state and the control signals, respectively. The initial state \(x(0)\) is considered, \(\overline{P}_{N+1}=\overline{P}^{T}_{N+1} \geq 0\) shows the penalty applied on the final state, the weighting matrices \(R(k)\) and \(Q(k)\) are considered as positive definite and non-negative definite matrices, respectively.

*LQR problem*: For the given system (2.7), calculate input signal samples \(\{u(0),u(1),\ldots , u(N)\}\) so that the performance index \(J(N)\) of (2.8) is minimized.

### Lemma 2.2

([1])

*The optimal control signal for an LQR problem with performance index* (2.8) *could be computed as*

*where the gain*
\(K(k)\)
*is respectively given as follows*:

*and*
\(\overline{P}(k)\)
*is obtained from the following difference Riccati equation*:

The classical duality can be seen from Table 1, now we give a brief summary in the following theorem.

### Theorem 2.1

([1])

*The LMMSE estimation problem presented in Lemma*
2.1
*and the LQR problem presented in Lemma*
2.2
*are dual*.

### Proof

For the classical duality principle, we present a simple explanation. Consider that the following transformations are performed in the right-hand side of (2.10), (2.11), and (2.12), respectively:

Now, we obtain \(K_{p}(k)\), \(R_{\varepsilon (k)}\), \(P(k+1)\) in the formulae (2.4), (2.5), and (2.6), respectively. The converse of the theorem could be proved similarly. □

## Duality for time-delay linear systems

The classic duality principle states that the optimal estimation problem with LMMSE criterion is equal to an optimal control problem with a quadratic performance index in the inverse time and vice versa. In this section, we want to discuss the duality between LMMSE estimation and LQR for time-delay systems.

Considering the reorganized innovation method, the LMMSE problem for systems which have multiple observation channels but single observation delay is considered in [44], that is,

where \(l_{i}\) is the measurement delay. Without loss of generality, \(l_{i}\) is assumed to satisfy that \(0=l_{0}< l_{1}<\cdots < l_{d} \). \({ v}_{i}(k)\) are white noises whose mean is zero and covariance matrices are \(E\{{ v}_{i}(k){ v}_{j}^{T} (s)\} ={ R_{i}(k)}\delta _{i,j}\delta _{k,s}\). By the way, the LQR problem is considered in [51] for systems which have multiple input channels but a single input

associated with this performance index

where \(u(-i)\) (\(0< i \leq l_{d}\)) are the control inputs with recognized statistical properties.

### Theorem 3.1

([51])

*The LMMSE estimation problem for the given systems* (3.1)*–*(3.2) *is dual to the LQR problem for the given system* (3.3) *associated with performance* (3.4).

However, note that systems studied in [44, 45] are a special case of systems which have only a single observation channel but multiple observation delays. The reorganized innovation could not be directly employed as a mathematical approach for solving the LMMSE estimation problem for these systems. According to the partial difference Riccati equation method [48], the LMMSE estimator could be designed for systems which have a single observation channel but multiple observation delays. These systems could be represented as

### Theorem 3.2

([35])

*For the given systems*

*associated with the following performance*:

*the LQR problem and the LMMSE estimation problem related to* (3.5)*–*(3.6) *are dual*.

Note that both in (3.1)–(3.2) and in (3.5)-(3.6), the delays only appear in the measurement equation. In the following we will investigate the estimator design for systems with delay which appears both in the state equation and in the observation equation.

### Lemma 3.1

([49])

*For the given systems*,

*the LMMSE filter and smoother are represented with the following equations*:

*and*
\(\widetilde{P}(\cdot ,\cdot ,\cdot )\)
*is computed based on the following partial difference Riccati equation*:

In the following, the aim is to establish the duality between the LMMSE estimation problem in Lemma 3.1 and the LQR problem for discrete linear systems including multiple time delays appearing in state and control input variables. To attain this aim, the LQR problem will be considered firstly, and then both results will be compared.

Suppose the discrete LQR problem for systems having a single input channel but multiple inputs, i.e.,

associated with the quadratic performance index

where \(\overline{P}_{N+1}=P^{T}_{N+1}\geq 0\) is the penalty applied on the final state, the weighting matrix \(R(k)\) and \(Q(k)\) are considered positive definite and non-negative definite, respectively. Moreover, the initial state \(x(-i)\) (\(0\leq i \leq h_{d}\)) and the control signal \(u(-i)\) (\(0< i \leq l_{d}\)) are considered to be known.

*LQR problem*: Find the control signal samples \(\{u(0),u(1), \ldots ,u(N)\}\) such that the performance index \(J(N)\) of (3.19) is minimized.

### Remark 3.1

The LQR problem mentioned above can be regarded as an \(N+1\) step decision-making problem in which the \(N+1\) control signal samples \(\{{u}(0);\ldots ;{u}(N)\}\) should be calculated so that the quadratic performance index (3.19) is minimized. The control law will be obtained by first finding the control for the last stage. Then the dynamic programming technique is employed to obtain the control law for each former stage and accordingly obtain the required control law for all the stages.

For convenience, we introduce the backward systems:

where \({\mathbf{v}}(k)\), \({\mathbf{n}}(k)\), and \({\mathbf{x}}(N+1)\) are uncorrelated white noises whose mean is zero and covariance matrices are \(\langle {\mathbf{x}}(N+1), {\mathbf{x}}(N+1) \rangle = \overline{P}_{N+1}\), \(\langle {\mathbf{n}}(k), {\mathbf{n}}(s) \rangle = {Q}(k)\delta _{ks}\), \(\langle {\mathbf{v}}(k), {\mathbf{v}}(s) \rangle = R(k)\delta _{ks}\), respectively. We also set that \({\mathbf{x}}(N+1+i)=0\) for \(i\geq 1\). Consider that \(x(k)\) with normal face of (3.18) and \(\mathbf{x}(k)\) with bold face of (3.20) are different.

### Lemma 3.2

*The optimal control signals obtained from the LQR problem for system* (3.18) *and performance index* (3.19) *are*

*in* (3.24), *the gains*
\(\overline{K}_{ij}(k)\)
*and*
\(\widetilde{K} _{ij}(k)\)
*could be obtained as follows*:

*while*
\(\overline{K}_{j}(k)\), *the smoothing gain of state*
\(\mathbf{x}(k+j)\), *could be calculated as*

*and*
\(\overline{P}(\cdot ,\cdot ,\cdot )\)
*is the solution of the following partial difference Riccati equation*:

*The initial conditions for this equation are as follows*:

### Remark 3.2

The LQR problem of the linear systems has been solved in [35], in which the multiple delays are contained in a single input channel and the controller is designed by means of a partial difference Riccati equation. Recently, we have extended the results of [35] to [36] by applying the dynamic programming technique in which the delays exist in both input and state. From Lemma 3.2, we notice that the controller gain can be obtained from a partial difference Riccati equation, where the dimension of the plant and the Riccati equation are the same. Compared with the previous augmentation method, the method in Lemma 3.2 is computationally efficient.

It can be seen from Lemma 3.2 that the controller gains can be transformed into solving the filtering and smoothing problems for backward systems (3.20)–(3.23). That is to say, the duality principle still holds for normal time-delay systems. Similar to the duality principle for delay-free systems, the duality principle for general time-delay systems will be introduced in the following.

It can be seen from Table 2 that the duality between estimation and control still holds for time-delay systems, now we provide a brief summary in Theorem 3.3.

### Theorem 3.3

*The LMMSE estimation problem provided in Lemma*
3.1
*is dual to the LQR problem provided in Lemma*
3.2.

### Proof

Similar to the proof of the classical duality principle, we present a simple explanation. If we make the following transformations:

with

in the right-hand side of (3.27), (3.29), (3.30), (3.31), and (3.32), respectively, then we obtain \(K_{j}(k)\), \(R_{\varepsilon (k)}\), \(\widetilde{P}(k-i,k-j,k)\), \(\widetilde{P}(k+1,k-j,k)\), \(\widetilde{P}(k+1,k+1,k)\) in the formulae (3.13), (3.14), (3.15), (3.16), and (3.17), respectively, and vice versa. □

## Duality for linear time-delay systems subject to multiplicative noise

Consider the following time-delay multiplicative noise system:

No generality will be lost if the delays are regarded to be ascending: \(0=h_{0}< h_{1}<\cdots < h_{d}\), \(0=l_{0}< l_{1}<\cdots < l_{d}\). \(\eta (k)\) and \(\omega (k)\) are random zero mean process and \({ E}\{\eta (k)\eta (j)\} = \sigma \delta _{kj}\), \({ E}\{\omega (k) \omega (j)\} = \sigma \delta _{kj}\), in addition, the initial states \({{x}}(-i)\) (\(i=0,1,\ldots ,\tau \), \(\tau =\max (h_{d},l_{d})\)) ensure \(E\{{x}(-i)\}=\mu _{-i}\) and \(E\{{x}(-i){x}^{T}(-j)\}=P_{0}(-i,-j)\). \({ n}(k)\) and \({ v}(k)\) are zero mean white noises with covariance matrices \(E \{{n}(k) { n}^{T} (j)\} ={\overline{Q}(k)}\delta _{kj}\), \(E \{{ v}(k){ v}^{T} (j)\} ={ \overline{R}(k)}\delta _{kj}\), respectively. \(\omega (k)\), \({v}(k)\), and \({n}(k)\) are the random processes which are mutually independent for all *k*.

*LMMSE problem*: The goal is finding the LMMSE estimation \(\hat{x}(k-l|k)\) (\(l=-1, 0, \ldots ,h_{d}\)) of the system state \(x(k-l)\) when given observation samples \(\{y(k),\ldots ,y(0)\}\), i.e., minimize the Euclidean 2-norm

at each time moment *k*. Before obtaining the LMMSE estimators of system (4.1)–(4.2), we first present a useful lemma.

### Lemma 4.1

*Based on system* (2.1), *for the given integer numbers*
*i*
*and*
*j*
\((0 \leq i\leq h_{d},0 \leq j\leq h_{d})\), *let*
\(D_{i,j}(k)=E\{x(k-i)x ^{T}(k-j)\}\), *then we have the following correlation functions*:

*which could be calculated by*

Based on equations (4.1)–(4.2), the following innovation sequence \(\varepsilon (k)\) is introduced:

where \(\hat{y}(k|k-1)\) is the projection of \(y(k)\) onto the linear space \(\mathbf{\ell }\{y(s)|_{0\leq s \leq k-1}\}\). Moreover, the cross covariance matrices for the estimation error are defined as

The LMMSE smoother and filter will be derived by the innovation analysis approach in the following.

### Theorem 4.1

*The smoother and the LMMSE filter for systems* (2.1)*–*(2.2) *are obtained as follows*:

*where the initial state values are*
\(\hat{x}(-k|0)=\mu _{-k}\)
*and*
\(K_{j}(k)\)
*could be obtained by calculating*

*while*
\(D_{0,0}(k)\)
*can be calculated from* (4.5)*–*(4.6), \(\widetilde{P}(\cdot ,\cdot ,\cdot )\)
*is obtained by calculating the following partial difference Riccati equation*:

### Proof

The projection theory [43] leads to

where \(K_{j}(k)\) is given as

Observe that \(x(k-j)=\hat{x}(k-j|k-1)+\tilde{x}(k-j|k-1)\). Now, according to (4.2) and (4.7), we have

The covariance matrices given in (4.7) of the innovation \(\varepsilon (k)\) will be calculated in the following, where Lemma 4.1 is utilized:

this means that (4.12) is true. Considering (4.18), we have

Based on (4.20)–(4.21), covariance matrices for the estimation error are as follows:

which is (4.13). Incorporating (4.1) and (4.17) gives

Furthermore, one has

therefore, (4.14) and (4.15) are true. Finally, (4.16) could be easily obtained from (4.8). Thus, the proof is completed. □

### Remark 4.1

In accordance with the innovation analysis approach, the optimal filter and smoother have been derived based on an *n*-dimensional partial difference Riccati equation (4.13)–(4.15). The number of divisions and multiplications is considered as the operation count because the divisions and multiplications are more valuable compared with additions. Consider that these numbers for our proposed method and for augmentation method are denoted by \(C_{\mathrm{new}}\) and \(C_{\mathrm{aug}}\), respectively. According to [35], it could be verified that the order of \(h_{d}\) in \(C_{\mathrm{aug}}\) is 3, while the order of \(h_{d}\) in \(C_{\mathrm{new}}\) is 2. Thus, \(C_{\mathrm{aug}}\gg C_{\mathrm{new}}\) when \(h_{d}\) is large enough. Compared with the state augmentation method [42, 79], the partial difference Riccati equation approach can greatly save the computational cost for the large delay.

However, the estimation problem in Theorem 4.1 is not dual anymore to the LQR problem provided in Lemma 3.2 because there are not only additive noises but also multiplicative noises in the estimation problem in Theorem 4.1. According to the delay-free systems, the authors in [8] established the duality between the LMMSE estimation for multiplicative noise systems with additive noise and the LQR for deterministic systems with constraint condition [96, 97]. Motivated by this point, we obtain the following result about duality between control and estimation.

### Theorem 4.2

*If we impose the conditions on the weight matrices*
*Q*
*and*
*R*
*in Lemma* 3.2

*then the forward partial difference Riccati equation* (4.13)*–*(4.15) *in Theorem*
4.1
*is dual to the backward partial difference Riccati equation* (3.30)*–*(3.32) *in Lemma*
3.2. *Moreover*, *similar to Theorem*
3.3, *we can find that the LMMSE estimation problem presented in Theorem*
4.1
*is dual to the LQR problem provided in Lemma*
3.2
*with constraint conditions* (4.23).

### Remark 4.2

([83])

It is needed to emphasize that the duality given in Theorem 4.2 is significant. The main result is that we have successfully revealed the essential links between the estimation problem for time-delay systems subject to multiplicative noise and the LQR problem for the deterministic time-delay systems with constraint conditions. Particularly, Theorem 3.3 establishes the duality principle between the estimation problem for general time-delay discrete systems and the LQR problem for the deterministic general time-delay discrete systems under free conditions.

## Duality for Markov jump systems

Consider special systems whose parameters are found by a Markov chain, where the chain \(\vartheta (t), t=0,1,\ldots ,l\), could have values in the set \(\tau =\{1,2,\ldots ,N\}\). Assume the process \(\theta (t)=\vartheta (l-t)\) which derives the parameters of the system

with initial condition \(x(0)=x_{0}\). As usual, \((x,\theta )\) stands for the state variable, *u* is the control variable, matrices \(A_{i},B _{i}, i\in \tau \) are known and of appropriate dimensions.

In fact, the authors in [95] assumed a quadratic performance index for system (5.1) with linear state feedback, resulting in a time reversed Markov jump LQR problem. The linear quadratic problem tends to control the system output

The quadratic performance index is given

Consider the standard LMMSE problem for Markov jump linear systems (MJLS) defined by

where the variables \(\omega (t)\) and \(z_{0}\) are i.i.d. random variables ensuring \(E\{\omega (t)\}=0\), \(E\{\omega (t)\omega (t)^{T}\}=I\), \(E\{z_{0}=0\}\), and \(E\{z_{0}z^{T}_{0}\}=\varSigma \).

### Theorem 5.1

([95])

*Under the condition that*
\(\theta (t)=\eta (l-t)\), *the LMMSE estimation problem for the given systems* (5.4)*–*(5.5) *is dual to the LQR problem for the given system* (5.1) *associated with performance* (5.3).

The result beautifully generalizes the classic duality principle between filtering and control to the Markov jump systems case; for details, please see [95].

## Conclusion

Motivated by the classical duality between control and filtering, in this paper we have reviewed the duality between linear estimation and linear quadratic regulation problems for time-delay systems, multiplicative noise systems, and Markov jump systems. It has effectively demonstrated the fundamental relations between the estimation problem and the LQR problem once again. In future research, is it possible to generalize the classical duality principle between control and filtering for linear systems (their matrices can be transposed in the dual sense) to network control systems? It is worth our thinking.

## References

Hassibi, B., Sayed, A.H., Kailath, T.: Indefinite quadratic estimation and control: a Unified approach to \(H_{2}\) and \(H_{\infty }\) theories. SIAM Studies in Applied Mathematics series (1998)

Basin, M.V., Rodriguez-Gonzalez, J.G.: Optimal control for linear systems with multiple time delays in control input. IEEE Trans. Autom. Control

**51**, 91–97 (2006)Todorov, E.: General duality between optimal control and estimation. In: Proc. of the 47th IEEE Conference on Decision and Control, pp. 4286–4292 (2008)

Goodwin, G.C., Doná, J.A.D., Seron, M.M., Zhuo, X.W.: Lagrangian duality between constrained estimation and control. Automatica

**41**(6), 935–944 (2005)Shi, L., Yuan, Y., Chen, J.: Finite horizon LQR control with limited controller-system communication. IEEE Trans. Autom. Control

**58**(7), 1835–1841 (2013)Mare, J.B., Doná, J.A.D.: Symmetry between constrained reference tracking and constrained state estimation. Automatica

**45**(1), 207–211 (2009)Mëller, C., Zhuo, X.W., Doná, J.A.D.: Duality and symmetry in constrained estimation and control problems. Automatica

**42**(1), 2183–2188 (2006)Sankaran, V.: Duality of linear discrete time systems with constraints. IEEE Trans. Aerosp. Electron. Syst.

**AES-11**(4), 654–659 (1975)Zhuk, S.: Duality principle for a class of ill-posed minimax control problems with linear differential-algebraic constraints. Appl. Math. Optim.

**68**(2), 289–309 (2013)Altman, E., Basar, T., Srikant, R.: Congestion control as a stochastic control problem with action delays. In: Proc. 34th IEEE Conf. on Decision and Control, New Orleans, pp. 1389–1394 (1999)

Kojima, A., Ishijima, S.: Robust controller design for delay systems in the gap-metric. IEEE Trans. Autom. Control

**40**(2), 370–374 (1995)Tadmor, G.: The standard \(H_{\infty }\) problem in systems with a single input delay. IEEE Trans. Autom. Control

**45**(3), 382–396 (2000)Luo, C., Liu, H.: Controllability of Boolean control networks under asynchronous stochastic update with time delay. J. Vib. Control

**22**(1), 235–246 (2016)Stamova, I., Stamov, T., Li, X.: Global exponential stability of a class of impulsive cellular neural networks with supremums. Int. J. Adapt. Control Signal Process.

**28**(11), 1227–1239 (2014)Wang, Z., Zhang, H., Fu, M., Zhang, H.: Consensus for high-order multi-agent systems with communication delay. Sci. China Inf. Sci.

**60**(9), 092204 (2017)Wang, Z., Zhang, H., Song, X., Zhang, H.: Consensus problems for discrete-time agents with communication delay. Int. J. Control. Autom. Syst.

**15**(7), 1515–1523 (2017)Chyung, D.H.: Discrete systems with delays in control. IEEE Trans. Autom. Control

**14**, 196–197 (1969)Tadmor, G.: Robust control in the gap: a state space solution in the presence of a single input delay. IEEE Trans. Autom. Control

**42**(9), 1330–1335 (1997)Tan, X., Cao, J., Li, X.: Leader-following mean square consensus of stochastic multi-agent systems with input delay via event-triggered control. IET Control Theory Appl.

**12**(2), 299–309 (2018)Pindyck, R.S.: The discrete-time tracking problem with a time delay in the control. IEEE Trans. Autom. Control

**17**, 397–398 (1972)Hu, J., Sui, G., Du, S., Li, X.: Finite-time stability of uncertain nonlinear systems with time-varying delay. Mathematical Problems in Engineering

**2017**(2017)Li, X., Wu, J.: Sufficient stability conditions of nonlinear differential systems under impulsive control with state-dependent delay. IEEE Trans. Autom. Control

**63**(1), 306–311 (2018)Li, X., Song, S.: Stabilization of delay systems: delay-dependent impulsive control. IEEE Trans. Autom. Control

**62**(1), 406–411 (2017)Ding, J., Cao, J., Feng, G., Alsaedi, A.: Stability analysis of delayed impulsive systems and applications. Circuits Syst. Signal Process.

**37**(3), 1062–1080 (2018)Yan, X., Song, X.: Global practical tracking by output feedback for nonlinear systems with unknown growth rate and time delay. The Scientific World Journal (2014)

Yan, X., Song, X., Wang, X.: Global output-feedback stabilization for nonlinear time-delay systems with unknown control coefficients. Int. J. Control. Autom. Syst.

**16**(4), 1550–1557 (2018)Meinsma, G., Mirkin, L.: \(H_{\infty }\) control of systems with multiple I/O delays via decomposition to Adobe problems. IEEE Trans. Autom. Control

**50**(2), 199–211 (2005)Duan, Z., Song, X., Qin, M.: Limited memory optimal filter for discrete-time systems with measurement delay. Aerosp. Sci. Technol.

**68**, 422–430 (2017)Zhang, X., Li, X.: Input-to-state stability of nonlinear systems with distributed-delayed impulses. IET Control Theory Appl.

**11**(1), 81–89 (2017)Hou, W., Wu, Z., Fu, M., Zhang, H.: Constrained consensus of discrete-time multi-agent systems with time delay. Int. J. Syst. Sci.

**49**(5), 947–953 (2018)Luo, C.: Hybrid delayed synchronizations of complex chaotic systems in modulus-phase spaces and its application. J. Comput. Nonlinear Dyn.

**11**(4), 041010-8 (2015)Zhang, B., Lam, J., Xu, S.: Stability analysis of distributed delay neural networks based on relaxed Lyapunov–Krasovskii functionals. IEEE Trans. Neural Netw. Learn. Syst.

**26**(7), 1480–1492 (2015)Lee, T.H., Park, J.H.: A novel Lyapunov functional for stability of time-varying delay systems via matrix-refined-function. Automatica

**80**, 239–247 (2017)Lee, T.H., Park, J.H., Xu, S.: Relaxed conditions for stability of time-varying delay systems. Automatica

**75**, 11–15 (2017)Liu, S., Xie, L., Zhang, H.: Infinite horizon LQR for systems with multiple delays in a single input channel. J. Control Theory Appl.

**8**(3), 368–374 (2010)Zhang, H., Song, X.: “Stage-by-stage” optimization approach to optimal control for general time-delay systems. In: Proc. 8th IEEE International Conference on Control and Automation, pp. 154–159 (2010)

Guo, P., Zhang, H., Alsaadi, F.E., Hayat, T.: Semi-tensor product method to a class of event-triggered control for finite evolutionary networked games. IET Control Theory Appl.

**11**(13), 2140–2145 (2017)Li, X., Zhang, X., Song, S.: Effect of delayed impulses on input-to-state stability of nonlinear systems. Automatica

**76**, 378–382 (2017)Lee, T.H., Park, J.H.: Improved criteria for sampled-data synchronization of chaotic Lur’e systems using two new approaches. Nonlinear Anal. Hybrid Syst.

**24**, 132–145 (2017)Duan, P., Li, J., Wang, Y., Sang, H., Jia, B.: Solving chiller loading optimization problems using an improved teaching-learning-based optimization algorithm. Optim. Control Appl. Methods

**39**(1), 65–77 (2018)Zheng, Z., Li, J.: Optimal chiller loading by improved invasive weed optimization algorithm for reducing energy consumption. Energy Build.

**161**, 80–88 (2018)Xiao, L., Hassibi, A., How, J.P.: Control with random communication delays via a discrete-time jump system approach. In: Proceedings of the American Control Conference, pp. 2199–2204 (2000)

Anderson, B., Moore, J.: Optimal Filtering. Prentice Hall, New York (1979)

Zhang, H., Xie, L., Zhang, D., Soh, Y.C.: A reorganized innovation approach to linear estimation. IEEE Trans. Autom. Control

**49**(10), 1810–1814 (2004)Zhao, H., Zhang, H., Zhang, C.: Duality principle between linear estimation and linear quadratic regulation for discrete time-delay systems, Robust filtering and fixed-lag smoothing for linear uncertain systems with single delayed measurement. In: Proceeding of the 26th Chinese Control Conference, pp. 23–27 (2007)

Song, X., Duan, Z., Park, J.H.: Linear optimal estimation for discrete-time systems with measurement-delay and packet dropping. Appl. Math. Comput.

**284**, 115–124 (2016)Song, X., Park, J.H., Yan, X.: Linear estimation for measurement-delay systems with periodic coefficients and multiplicative noise. IEEE Trans. Autom. Control

**62**(8), 4124–4130 (2017)Sun, S.: Linear minimum variance estimators for systems with bounded random measurement delays and packet dropouts. Signal Process.

**89**(7), 1457–1466 (2009)Song, X., Yan, X.: Linear quadratic Gaussian control for linear time-delay systems. IET Control Theory Appl.

**8**(6), 375–383 (2014)Chen, B., Yu, L., Zhang, W.A.: Robust Kalman filtering for uncertain state delay systems with random observation delays and missing measurements. IET Control Theory Appl.

**17**(5), 1945–1954 (2010)Zhang, H., Duan, G., Xie, L.: Linear quadratic regulation for linear time-varying systems with multiple input delays. Automatica

**42**(9), 1465–1476 (2006)Elia, N.: Remote stabilization over fading channels. Syst. Control Lett.

**54**(3), 237–249 (2005)Xiao, N., Xie, L., Qiu, L.: Feedback stabilization of discrete-time networked systems over fading channels. IEEE Trans. Autom. Control

**57**(9), 2176–2189 (2012)Zhang, H., Song, X., Shi, L.: Convergence and mean square stability of optimal estimators for systems with measurement packet dropping. IEEE Trans. Autom. Control

**57**(5), 1248–1253 (2012)Bioucas-Dias, J.M., Figueiredo, M.A.T.: Multiplicative noise removal using variable splitting and constrained optimization. IEEE Trans. Image Process.

**19**(7), 1720–1730 (2010)Li, Y., Karimi, H.R., Zhang, Q., Zhao, D., Li, Y.: Fault detection for linear discrete time-varying systems subject to random sensor delay: a Riccati equation approach. IEEE Trans. Circuits Syst. I, Regul. Pap.

**65**(5), 1707–1716 (2018)Ren, Y., Wang, A., Wang, H.: Fault diagnosis and tolerant control for discrete stochastic distribution collaborative control systems. IEEE Trans. Syst. Man Cybern. Syst.

**45**(3), 462–471 (2015)Li, Y., Liu, S., Zhong, M., Ding, S.X.: State estimation for stochastic discrete-time systems with multiplicative noises and unknown inputs over fading channels. Appl. Math. Comput.

**320**, 116–130 (2018)Liu, W.: Optimal filtering for discrete-time linear systems with multiplicative and time-correlated additive measurement noises. IET Control Theory Appl.

**9**(6), 831–842 (2014)Song, X., Park, J.H.: Linear minimum mean square estimation for discrete-time measurement-delay systems with multiplicative noise and Markov jump. IET Control Theory Appl.

**10**(10), 1161–1169 (2016)Carravetta, F., Mavelli, G.: Suboptimal stochastic linear feedback control of linear systems with state-and control-dependent noise: The incomplete information case. Automatica

**43**(5), 751–757 (2007)Li, L., Song, X.: State estimation for systems with packet dropping and state equality constraints. IEEE Transactions on Circuits and Systems – II: Express Briefs. https://doi.org/10.1109/TCSII.2018.2889047

Song, X., Zheng, W.X.: Linear estimation for discrete-time periodic systems with unknown measurement input and missing measurements. ISA Trans. (2018). https://doi.org/10.1016/j.isatra.2018.11.013

Nahi, N.E.: Optimal recursive estimation with uncertain observation. IEEE Trans. Inf. Theory

**15**(4), 457–462 (1969)Tugnait, J.K.: Stability of optimal linear estimations of stochastic signals in white multiplicative noise. IEEE Trans. Autom. Control

**26**(3), 757–761 (1981)Yang, F., Wang, Z., Hung, Y.S.: Robust Kalman filtering for discrete time-varying uncertain systems with multiplicative noises. IEEE Trans. Autom. Control

**47**(7), 1179–1183 (2002)Dragan, V.: Optimal filtering for discrete-time linear systems with multiplicative white perturbations and periodic coefficients. IEEE Trans. Autom. Control

**58**(4), 1029–1034 (2013)Sun, S.: Optimal linear filters for discrete-time systems with randomly delayed and lost measurements with/without time stamps. IEEE Trans. Autom. Control

**58**(6), 1551–1556 (2013)Song, X., Park, J.H.: Linear optimal estimation for discrete-time measurement-delay systems with multi-channel multiplicative noise. IEEE Trans. Circuits Syst. II, Express Briefs

**64**(2), 156–160 (2017)Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control Optim.

**6**(4), 681–697 (1968)Meditch, J.S.: On optimal control of linear systems in the presence of multiplicative noise. IEEE Trans. Aerosp. Electron. Syst.

**AES-12**(1), 80–85 (1976)Zhang, W., Chen, B.S.:

*H*-representation and applications to generalized Lyapunov equations and linear stochastic systems. IEEE Trans. Autom. Control**57**(12), 3009–3022 (2012)Zhang, H., Li, L., Xu, J., Fu, M.: Linear quadratic regulation and stabilization of discrete-time systems with delay and multiplicative noise. IEEE Trans. Autom. Control

**60**(10), 2599–2613 (2015)Ait Rami, M., Chen, X., Zhou, X.Y.: Discrete-time indefinite LQ control with state and control dependent noises. J. Glob. Optim.

**23**(3–4), 245–265 (2002)Ghaoui, L.E.: State-feedback control of systems with multiplicative noise via linear matrix inequalities. Syst. Control Lett.

**24**(3), 223–228 (1995)Gershon, E., Shaked, U., Yaesh, I.: \(\mathcal{H}_{\infty }\) control and filtering of discrete-time stochastic systems with multplicative noise. Automatica

**37**(3), 409–417 (2001)Zhu, Q., Li, X.: Exponential and almost sure exponential stability of stochastic fuzzy delayed Cohen–Grossberg neural networks. Fuzzy Sets Syst.

**203**, 74–94 (2012)El Bouhtouri, A., Hinrichsen, D., Pritchard, A.J.: \(\mathcal{H}_{ \infty }\)-type control for discrete-time stochastic systems. Int. J. Robust Nonlinear Control

**9**(13), 923–948 (1999)Crevecoeur, F., Sepulchre, R.J., Thonnard, J.L., Lefèvre, P.: Improving the state estimation for optimal control of stochastic processes subject to multiplicative noise. Automatica

**47**(3), 591–596 (2011)Costa, O.L.V., Oliveira, A.: Optimal mean variance control for discrete-time linear systems with Markovian jumps and multiplicative noises. Automatica

**48**(2), 304–315 (2012)Song, X., Park, J.H.: Linear quadratic regulation problem for discrete-time systems with multi-channel multiplicative noise. Syst. Control Lett.

**89**, 74–82 (2016)Huang, Y., Zhang, W., Zhang, H.: Infinite horizon linear quadratic optimal control for discrete-time stochastic systems. Asian J. Control

**10**(5), 608–615 (2008)Song, X., Yan, X.: Duality of linear estimation for multiplicative noise systems with measurement delay. IET Signal Process.

**7**(4), 277–284 (2013)Zhang, B., Zheng, W.X., Xu, S.: Filtering of Markovian jump delay systems based on a new performance index. IEEE Trans. Circuits Syst. I, Regul. Pap.

**60**(5), 1250–1263 (2013)Yang, Y., Liang, Y., Pan, Q., Qin, Y., Yang, F.: Linear minimum-mean-square error estimation of Markovian jump linear systems with stochastic coefficient matrices. IET Control Theory Appl.

**8**(12), 1112–1126 (2013)Costa, O.L.V., Guerra, S.: Stationary filter for linear minimum mean square error estimator of discrete-time Markovian jump systems. IEEE Trans. Autom. Control

**47**(8), 1351–1356 (2002)Terra, M.H., Ishihara, J.Y., Jesus, G., Cerri, J.P.: Robust estimation for discrete-time Markovian jump linear systems. IEEE Trans. Autom. Control

**58**(8), 2065–2070 (2013)Matei, I., Baras, J.S.: Optimal state estimation for discrete-time Markovian jump linear systems, in the presence of delayed output observations. IEEE Trans. Autom. Control

**56**(9), 2235–2240 (2011)Han, C., Zhang, H.: Optimal filtering in discrete-time systems with time delays and Markovian jump parameters. ANZIAM J.

**51**(2), 218–233 (2009)Li, X., Rakkiyappan, R., Sakthivel, N.: Non-fragile synchronization control for Markovian jumping complex dynamical networks with probabilistic time-varying coupling delay. Asian J. Control

**17**(5), 1678–1695 (2015)Xia, J., Chen, G., Sun, W.: Extended dissipative analysis of generalized Markovian switching neural networks with two delay components. Neurocomputing

**260**, 275–283 (2017)Xia, J., Gao, H., Liu, M., Zhuang, G., Zhang, B.: Non-fragile finite-time extended dissipative control for a class of uncertain discrete time switched linear systems. J. Franklin Inst.

**355**, 3031–3049 (2018)Costa, O.L.V., Fragoso, M.D., Marques, R.P.: Discrete-Time Markov Jump Linear Systems. Springer, Berlin (2005)

Ma, H., Jia, Y.: \(\mathcal{H}_{2}\) control of discrete-time periodic systems with Markovian jumps and multiplicative noise. Int. J. Control

**86**(10), 1837–1849 (2013)Gutierrez-Pachas, D.A., Costa, E.F.: On the linear quadratic problem for systems with time reversed Markov jump parameters and the duality with filtering of Markov jump linear systems. IEEE Trans. Autom. Control

**63**(9), 3040–3045 (2018)Li, J., Duan, P., Sang, H., Wang, S., Liu, Z., Duan, P.: An efficient optimization algorithm for resource-constrained steelmaking scheduling problems. IEEE Access

**6**, 33883–33894 (2018)Li, J., Sang, H., Han, Y., Wang, C., Gao, K.: Efficient multi-objective optimization algorithm for hybrid flow shop scheduling problems with setup energy consumptions. J. Clean. Prod.

**181**, 584–598 (2018)

### Acknowledgements

The authors are very grateful to the respectable reviewers for their constructive suggestions and useful comments.

### Availability of data and materials

No data is used to support this study.

## Funding

This work was supported in part by the National Science Foundation of China (61873152), the National Science Foundation of Shandong Province (ZR2016FM17), and the Chinese Postdoctoral Science Foundation (2017M612336).

## Author information

### Authors and Affiliations

### Contributions

The contribution of the authors to the writing of this paper is identical. The final manuscript was read and approved by all of them.

### Corresponding author

## Ethics declarations

### Competing interests

The authors express that there is no conflict of interest concerning with the publication of this work. The authors express that they have no competing interests.

## Additional information

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

### Cite this article

Song, X., Yan, X. & Li, X. Survey of duality between linear quadratic regulation and linear estimation problems for discrete-time systems.
*Adv Differ Equ* **2019**, 90 (2019). https://doi.org/10.1186/s13662-019-2043-2

Received:

Accepted:

Published:

DOI: https://doi.org/10.1186/s13662-019-2043-2

### Keywords

- Linear estimation
- Time-delay
- Multiplicative noise
- Markov jump
- Difference Riccati equation
- Linear quadratic regulation
- Duality