- Research
- Open access
- Published:
A maximum principle for fully coupled controlled forward–backward stochastic difference systems of mean-field type
Advances in Difference Equations volume 2020, Article number: 188 (2020)
Abstract
In this paper, we consider the optimal control problem for fully coupled forward–backward stochastic difference equations of mean-field type under weak convexity assumption. By virtue of employing a suitable product rule and formulating a mean-field backward stochastic difference equation, we establish the stochastic maximum principle and also derive, under additional assumptions, that the stochastic maximum principle is also a sufficient condition. As an application, a Stackelberg game of mean-field backward stochastic difference equation is presented to demonstrate our results.
1 Introduction
Let \(T>0\) be fixed, \((\varOmega ,\mathfrak{F},\{\mathfrak{F}_{t}\}_{0\leq t\leq T}, \mathbb{P})\) be a filtered probability space, on which a martingale process \(W_{t}\) with independent increments is defined, and \(\mathfrak{F}_{t}=\sigma \{W_{l},l=0,1,\ldots ,t-1\}\vee \mathcal{N}_{ \mathbb{P}}\) (the set of all \(\mathbb{P}\)-null subsets). Consider the following discrete-time fully coupled stochastic system:
with the cost functional
Here, we reserve the notation Δ for the backward difference operator \(\Delta X_{t}=X_{t+1}-X_{t}\). W, M are the square integrable martingale processes and M is strongly orthogonal to W. \(\mathbb{E}\) means the expectation operator and f, b, σ, g, h, l are given functions (satisfying some proper conditions to be elaborated later). Then we could present the following stochastic optimal control problem.
Problem \(\mathcal{A}\). Find \(\bar{u}\in \mathscr{U}_{\mathrm{ad}}\) (which shall be defined later) such that
For stochastic optimal control problems (see [1]), one of the main topics is to establish the stochastic maximum principle (SMP). A wide range of concerns have been given to different versions of SMP (see [2–5]), especially, to forward–backward stochastic control systems (see [6–9]). It is widely recognized that forward–backward stochastic differential equations (FBSDEs) are extensively studied and there are productive results (see [6, 10, 11]). Nevertheless, discrete-time optimal control problems are more relevant to economic, engineering, biomedical, operation research problems, optimizing complex technological systems, etc. As is known to all, Pontryagin maximum principle in continuous-time framework cannot be extended to discrete-time counterpart, except for some very special cases, due to the nature of admissible control variations. Naturally, it motivated us to formulate discrete analog and even some improper results were deduced. Butkovskii [12] clearly demonstrated some errors in the existing works. The intrinsic reason for the errors is that the significance of convexity has been ignored. Generally speaking, the discrete-time maximum principle fails unless a certain convexity precondition is imposed on the control system. Pshenichnyi [13] elaborated why discrete-time systems require a certain convexity assumption for the effectiveness of the necessary condition while continuous-time systems enjoy it automatically because of the so-called hidden convexity. To the best of our knowledge, the study on the SMP of forward–backward stochastic difference equations (FBSΔEs) is quite rare in the literature. To fill the gap, in this work, we are devoted to considering the SMP of the forward–backward stochastic difference systems.
As for the discrete-time framework, recently, Mahmudov [14] derived the first-order and second-order necessary optimality conditions for discrete-time stochastic optimal control problems by virtue of new discrete-time backward stochastic equation and backward stochastic matrix equation. Lin and Zhang [15] investigated the SMP where the state equation was just on a forward SΔE with the convex control domain. Xu et al. [16] considered the solvability of fully coupled FBSΔEs, in which the BSΔE was given as the conditional expectation form and the coefficients in the backward equation were degenerate. Some representative works in this direction include [17–20]. Very recently, Ji and Liu [21] first discussed the SMP for FBSΔEs under the convex control domain, which made substantial progresses in discrete-time forward–backward systems.
In 2009, Buckdahn et al. [22] investigated a special case of backward stochastic differential equations (BSDEs), the so-called mean-field BSDEs, which were derived by a limit of high dimensional FBSDEs, parallel to a large stochastic particle system. From then on, many authors discussed the mean-field system in different frameworks (see [4, 23]).
Motivated by the above discussions, our purpose of this paper is to derive the more general and constructive SMP for mean-field system (1.1)–(1.2) under weaker convexity assumption. From the perspective of the techniques adopted for discrete-time case, the obstacles encountered are twofold. The first issue entails choosing a suitable expression of the product rule
In our setting, the Itô formula in continuous-time framework is invalid. In addition, most of the methods applied to discuss continuous-time systems cannot be directly adapted to discrete-time cases. Hence, it is necessary to employ a more characteristic and refined approach for investigating the discrete-time stochastic optimal control problems. The next issue entails formulating the discrete-time counterpart BSΔE as (1.1), which is distinctly different from the continuous-time BSDE. Lately, many authors have been devoted to considering BSΔE (see [24–26]). In general, there are two approaches to formulating BSΔE. One is driving by a finite state process (see [25]). In this work, we adopt another formulation as in [24], which is driven by a martingale with independent increments and the generator f in (1.1) relies on time \(t+1\). Based on these arguments, we could obtain the dual principle. It is worth mentioning that our paper differs from [19] in the following aspects. Firstly, our work is based on a weaker convexity assumption. Secondly, our results are obtained in the mean-field framework. Thirdly, we not only establish the SMP, but also derive, under additional assumptions, the SMP, which turns to be a sufficient condition. Finally, as an application, we present a Stackelberg game of mean-field BSΔE to demonstrate our results. To sum up, this is the first paper to discuss the discrete-time forward–backward stochastic optimal control problems of mean-field type under weaker convexity assumption, enabling us to establish the more general and constructive SMP. Our work generalizes and enhances the previously known SMP of [19, 23]. Meanwhile, it extends the classical results of [17, 23] to the mean-field theory as well as forward–backward system. It is interesting to remark that the results of our work also remain for multi-dimensional driving process; in addition, we could also consider a more general system, in which the mean-field terms are allowed to depend on some functional of the law (see [27]). There is no essential difficulty.
The reminder of the paper is organized as follows. The next section states some preliminaries. Section 3 is devoted to considering MF-FBSΔE (1.1). In Sect. 4, we establish the SMP and the sufficient condition for Problem \(\mathcal{A}\). In Sect. 5, a Stackelberg game of mean-field BSΔE is given to illustrate the theoretical results. Section 6 presents some perspectives and open problems.
2 Preliminaries
Let \(\mathbb{T}=\{0,1,\ldots ,T\}\). For a vector x, \(x'\) stands for its transpose. I represents the unit matrix with appropriate dimension. \(\mathbb{E}_{t}\) means the conditional mathematical expectation \(\mathbb{E}[\cdot |\mathfrak{F}_{t}]\). \(\mathfrak{F}_{0}=\{\emptyset ,\varOmega \}\) and \(\mathfrak{F}=\mathfrak{F}_{T}\). For \(t\in \{0,\ldots ,T-1\}\), \(\mathbb{E}_{t}[\Delta W_{t}]=\mathbb{E}[\Delta W_{t}]=0\), \(\mathbb{E}[\Delta W_{t}\Delta W_{t}']=I\). Now, we shall introduce some spaces to be used frequently in what follows.
In addition, we introduce the following admissible control set:
Definition 2.1
([28])
A point \(\bar{y}\in S\subset \mathbb{R}^{m}\) is called a relative interior point of S along the straight line \(l(\bar{y},\tilde{y}):=\{\tilde{y} \mid \tilde{y}=\bar{y}+\kappa ( \tilde{y}-\bar{y}), \tilde{y}\in S\setminus \{\bar{y}\}\neq \emptyset , \kappa \in \mathbb{R}\}\) if there exists \(\gamma =\gamma (\tilde{y})\in (0,1]\) such that \(\bar{y}+\varepsilon (\tilde{y}-\bar{y})\in S\) holds for all \(\varepsilon \in [-\gamma ,\gamma ]\). Besides, ȳ is called a relative interior point of S in a broad sense if ȳ is a relative interior point of S along every straight line in the set \(\{l(\bar{y},y):y\in S\setminus \{\bar{y}\}\}\). The totality of these points is called a relative interior of S in a broad sense and is denoted by riS. S is called relatively open in a broad sense if \(\operatorname{ri}S=S\).
Definition 2.2
([18])
A set \(S\subset \mathbb{R}^{m}\) is called γ-convex relative to a point \(y_{0}\in S\) if, for each point \(y\in S\), there exists \(\gamma =\gamma (y)\in (0,1]\) such that, for all \(\varepsilon \in [0,\gamma ]\), \(y_{0}+\varepsilon (y-y_{0})\in S\) holds. S is γ-convex if S is γ-convex relative to all of its points.
Definition 2.3
A set \(S\subset \mathbb{R}^{m}\) is called ±γ-convex relative to a point \(y_{0}\in S\) if, for each point \(y\in S\), there exists \(\gamma =\gamma (y)\in (0,1]\) such that, for all \(\varepsilon \in [0,\gamma ]\) or for all \(\varepsilon \in [-\gamma ,0]\), \(y_{0}+\varepsilon (y-y_{0})\in S\) holds. S is ±γ-convex if S is ±γ-convex relative to all of its points.
Remark 2.1
It is obvious that a relatively open set in a broad sense, convex and open sets are a γ-convex set. Besides, a γ-convex set is a ±γ-convex set. Nevertheless, the reverse does not always hold. For instance, \(M_{1}=[1,2)\cup (3,4]\) is γ-convex, but it is neither a convex set nor an open set. \(M_{2}=[1,2]\cup [3,4]\) is ±γ-convex, but it is not γ-convex.
We proceed to introducing some notations and basic assumptions which shall be assumed throughout the paper. Denote the usual inner product by \(\langle \cdot ,\cdot \rangle \) and the norm by \(|\cdot |\) of a Euclidean space. For \(\varGamma =(x,y,z,\tilde{x},\tilde{y},\tilde{z})\), define \(F(t,\varGamma ,u)=(-f(t,\varGamma ,u),b(t,\varGamma ,u), \sigma (t,\varGamma ,u))\) and \(\varLambda =b, \sigma , f, g\).
- (A1)
\(f(t,y,z,\tilde{y},\tilde{z})\) is uniformly Lipschitz continuous and independent of z, z̃ at \(t=T\), i.e., for any \(y, y_{1}, z, z_{1}\in \mathbb{R}^{n}\), there exists a constant \(c>0\) such that
$$ \textstyle\begin{cases} \vert f(T,y,z,\tilde{y},\tilde{z})-f(T,y_{1},z_{1},\tilde{y}_{1},\tilde{z}_{1}) \vert \leq c( \vert y-y_{1} \vert + \vert \tilde{y}-\tilde{y}_{1} \vert ),\quad t=T,\mathbb{P} \mbox{-a.s.}, \\ \vert f(t,y,z,\tilde{y},\tilde{z})-f(t,y_{1},z_{1},\tilde{y}_{1}, \tilde{z}_{1}) \vert \\ \quad \leq c( \vert y-y_{1} \vert + \vert \tilde{y}-\tilde{y}_{1} \vert + \vert z-z_{1} \vert + \vert \tilde{z}-\tilde{z}_{1} \vert ),\quad t\in \{1,2,\ldots ,T-1\}, \mathbb{P} \mbox{-a.s.}, \\ f(t,0,0,0,0)\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n}),\quad t \in \{1,2,\ldots ,T\},\mathbb{P}\mbox{-a.s.} \end{cases} $$ - (A2)
b, σ, f, g are uniformly Lipschitz continuous and differentiable on Γ, u; l, h are continuously differentiable on x, x̃, and all the derivatives are uniformly bounded. Moreover, f is independent of z, z̃ at \(t=T\).
- (A3)
∀Γ, \(u\in \mathscr{U}_{\mathrm{ad}}\), \(\varLambda (\cdot ,\varGamma ,u)\) is a \(\mathfrak{F}_{t}\)-adapted process, \(l(x,\tilde{x})\in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), \(h(x, \tilde{x})\in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R})\), and \(F(t,0,0)\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n} \times \mathbb{R}^{n}\times \mathbb{R}^{n})\), \(g(t,0,0) \in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R})\).
- (A4)
(Monotonic conditions)
For \(t\in \{1,\ldots ,T-1\}\),
$$\begin{aligned}& \mathbb{E}\bigl\langle F(t,\varGamma ,u)-F(t,\varGamma _{1},u),\varTheta - \varTheta _{1} \bigr\rangle \leq -\beta \mathbb{E} \vert \varTheta - \varTheta _{1} \vert ^{2},\quad \mathbb{P}\mbox{-a.s.}, \\& \quad \forall \varGamma =(x,y,z,\tilde{x},\tilde{y},\tilde{z}), \varGamma _{1}=(x_{1},y_{1},z_{1}, \tilde{x}_{1},\tilde{y}_{1},\tilde{z}_{1}), \varTheta =(x,y,z), \varTheta _{1}=(x_{1},y_{1},z_{1}). \end{aligned}$$For \(t=T\),
$$ \mathbb{E}\bigl\langle -f(T,x,y,\tilde{x},\tilde{y},u)+f(T,x_{1},y_{1}, \tilde{x}_{1},\tilde{y}_{1},u), x-x_{1}\bigr\rangle \leq -\beta \mathbb{E} \vert x-x_{1} \vert ^{2},\quad \mathbb{P}\mbox{-a.s.} $$For \(t=0\),
$$\begin{aligned} & \mathbb{E}\bigl\langle b(0,\varGamma ,u)-b(0,\varGamma _{1},u), y-y_{1} \bigr\rangle +\mathbb{E}\bigl\langle \sigma (0,\varGamma ,u)- \sigma (0,\varGamma _{1},u), z-z_{1} \bigr\rangle \\ &\quad \leq -\beta \mathbb{E} \bigl( \vert y-y_{1} \vert ^{2}+ \vert z-z_{1} \vert ^{2} \bigr),\quad \mathbb{P}\mbox{-a.s.} \end{aligned}$$Besides,
$$ \mathbb{E}\bigl\langle l(x,\tilde{x})-l(x_{1},\tilde{x}_{1}),x-x_{1} \bigr\rangle \geq c\mathbb{E} \vert x-x_{1} \vert ^{2},\mathbb{P}\mbox{-a.s.}, $$where c, β are nonnegative constants.
- (A5)
The set \(U_{t}\)\((t\in \mathbb{T})\) is ±γ-convex.
Throughout the paper, we formally denote \(f(0,\varGamma ,u)=b(T,\varGamma ,u)=\sigma (T,\varGamma ,u)=g(T,\varGamma ,u) \equiv 0\). Let \(\pi \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), we consider the following MF-BSΔE:
$$ \textstyle\begin{cases} \Delta Y_{t}=-f(t+1,Y_{t+1},Z_{t+1},\mathbb{E}Y_{t+1},\mathbb{E}Z_{t+1})+Z_{t} \Delta W_{t}+\Delta M_{t}, \\ Y_{T}=\pi . \end{cases} $$(2.1)
Definition 2.4
The triple of processes \((Y,Z,M)\in \mathcal{H}^{2}[0,T]\) is called a solution of MF-BSΔE (2.1) if it satisfies (2.1) for any \(t\in \{0,1,\ldots ,T-1\}\) and M is a martingale process strongly orthogonal to W.
Theorem 2.1
Assume that (A1) holds, then for any\(\pi \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), MF-BSΔE (2.1) admits a unique adapted solution\((Y,Z,M)\).
Proof
Firstly, we shall show the existence by using the backward induction method. From (A1) and \(\pi \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), we have \(f(T,\pi ,\mathbb{E}\pi )\in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\). Then \(\mathbb{E}\{|\mathbb{E}_{T-1}[\pi +f(T,\pi ,\mathbb{E}\pi )]|^{2}\}< \infty \). Hence, \(\pi +f(T,\pi ,\mathbb{E}\pi )-\mathbb{E}_{T-1}[\pi +f(T,\pi , \mathbb{E}\pi )]\) is a square integrable martingale difference. Further, by the Galtchouk–Kunita–Watanabe decomposition in [29], there are \(Z_{T-1}\in \mathfrak{F}_{T-1}\), \(Z_{T-1}\Delta W_{T-1}\in \mathcal{L}^{2}(\mathfrak{F}_{T-1}; \mathbb{R}^{n})\), and \(\Delta M_{T-1}\in \mathcal{L}^{2}(\mathfrak{F}_{T-1};\mathbb{R}^{n})\) such that \(\mathbb{E}_{T-1}[\Delta M_{T-1}]=\mathbb{E}_{T-1}[\Delta M_{T-1} \Delta W_{T-1}']=0\) and
Here, \(\Delta M_{T-1}\) is uniquely determined in that decomposition. Multiplying (2.2) by \(\Delta W_{T-1}'\) and then applying \(\mathbb{E}_{T-1}\) to both sides, we derive
We further obtain
This implies \(Y_{T-1}=\mathbb{E}_{T-1}[\pi +f(T,\pi ,\mathbb{E}\pi )]\in \mathcal{L}^{2}(\mathfrak{F}_{T-1};\mathbb{R}^{n})\). Thus, we determine \(Y_{T-1}\), \(Z_{T-1}\), \(\Delta M_{T-1}\).
We continue this backward procedure. Assume that \(Y_{t+1}\in \mathcal{L}^{2}(\mathfrak{F}_{t+1};\mathbb{R}^{n})\), \(t\in \{0,1,\ldots ,T-2\}\). Similarly to the above discussions, we know \(Z_{t}\in \mathfrak{F}_{t}\), \(Z_{t}\Delta W_{t}\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n})\), \(\Delta M_{t}\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n})\) such that \(\mathbb{E}_{t}[\Delta M_{t}]=\mathbb{E}_{t}[\Delta M_{t}\Delta W_{t}']=0\) and
In summary, we deduce \((Y_{t},Z_{t},\Delta M_{t})\in \mathcal{L}^{2}(\mathfrak{F}_{t}; \mathbb{R}^{n})\times \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n}) \times \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n})\), \(0\leq t\leq T-2\). Without loss of generality, let \(M_{0}=0\) and \(M_{t}=M_{0}+\sum_{s=0}^{t-1}\Delta M_{s}\), we see that (2.1) holds for \(t\in \{0,1,\ldots ,T-1\}\). In addition, M is a square integrable martingale process. Furthermore, since
we get that M is strongly orthogonal to W. The existence is finished.
Next, we shall prove the uniqueness. Assume that there are two solutions \((Y_{t}^{1},Z_{t}^{1},\tilde{Y}_{t}^{1}, \tilde{Z}_{t}^{1},M_{t}^{1} )\) and \((Y_{t}^{2},Z_{t}^{2},\tilde{Y}_{t}^{2},\tilde{Z}_{t}^{2},M_{t}^{2} )\) of MF-BSΔE (2.1). Then
Combining \(Z_{T-1}^{1}=\mathbb{E}_{T-1}[(\pi +f(T,\pi ,\mathbb{E}\pi ))\Delta W_{T-1}']=Z_{T-1}^{2}\) with (2.2), we have \(\Delta M_{T-1}^{1}=\Delta M_{T-1}^{2}\). Thus, using (2.3), we can immediately get that \(Y_{T-1}^{1}=Y_{T-1}^{2}\). The inductive method and \(Z_{0}^{1}=Z_{0}^{2}=M_{0}^{1}=M_{0}^{2}=0\) yield \((Y_{t}^{1},Z_{t}^{1},M_{t}^{1} )= (Y_{t}^{2},Z_{t}^{2},M_{t}^{2} )\) for \(t\in \{0,1,\ldots ,T-1\}\). □
3 Controlled MF-FBSΔEs
In this section, we focus on the fully coupled MF-FBSΔE (1.1). Let \(\bar{u}_{t}\) and \((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t} )\) be the optimal control and optimal trajectory of Problem \(\mathcal{A}\), respectively. Assume that (A2)–(A5) hold, firstly we define a multi-valued mapping:
where \(U^{+}_{t}\), \(U^{-}_{t}\) represent that the set \(U_{t}\) is γ-convex and −γ-convex, respectively. Notice that the set \(U_{t}\) is ±γ-convex, there exists \(\gamma >0\) such that, for all \(\varepsilon \in (0,\gamma ]\), we could find another admissible control
where \(\alpha (t)\in \mathcal{I}(t,v_{t})\). We construct a needle variation
where \(\alpha \in \mathcal{I}(\theta ,v_{\theta })\), \((\theta ,v_{\theta }) \in \mathbb{T}\times U_{\theta }\), and \(\varepsilon \in (0,\gamma ^{\ast }]\) with \(\gamma ^{\ast }=\frac{\gamma (v_{\theta })}{1+\alpha }\). Denote that \((X^{\varepsilon }_{t},Y^{\varepsilon }_{t},Z^{\varepsilon }_{t})\) is the state trajectory corresponding to the admissible control \(u^{\varepsilon }_{t}\). Now, we give the following existence and uniqueness theorem.
Theorem 3.1
Assume that (A2)–(A4) hold, then there exists a unique adapted solution\((X,Y,Z,M)\in \mathcal{M}^{2}[0,T]\)for mean-field system (1.1).
We shall apply the following two technical lemmas to prove the existence part of Theorem 3.1, and the proof of these lemmas shall be presented in the sequel.
Lemma 3.1
Suppose\((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), then the following linear MF-FBSΔE
has a unique solution\((X,Y,Z,M)\in \mathcal{M}^{2}[0,T]\).
Now, we define a family of MF-FBSΔEs parameterized by\(\mu \in [0,1]\)as follows:
where
Lemma 3.2
For a given\(\mu _{0}\in [0,1)\)and any\((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), MF-FBSΔEs (3.3) have a unique solution. Then there exists\(\delta _{0}\in (0,1)\)such that, for any\(\mu \in [\mu _{0},\mu _{0}+\delta _{0}]\)and\((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), MF-FBSΔEs (3.3) have a unique solution.
Proof of Theorem 3.1
Uniqueness. Suppose that \((X,Y,Z,M)\) and \((\check{X},\check{Y},\check{Z},\check{M} )\) are two solutions of (1.1), we denote
For \(t\in \{0,1,\ldots ,T-1\}\), it yields that
where
Notice that W, M, M̌ are square integrable martingale processes and M, M̌ are strongly orthogonal to W, we obtain \(\mathbb{E}[\varPsi _{t}]=0\). Furthermore,
Using the monotonic conditions, it follows that
which further implies
Besides, it is easy to see \(\mathbb{E} \vert Y_{T}-\check{Y}_{T} \vert ^{2}=0\) and \(\mathbb{E}\sum_{t=0}^{T} \vert M_{t}-\check{M}_{t} \vert ^{2}=0\). Thereby, \(\varTheta =\check{\varTheta }\).
Existence. By Lemma 3.1, we can immediately get that, when \(\mu =0\), for any \((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), MF-FBSΔEs (3.3) have a unique solution. By Lemma 3.2, for any \((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), (3.3) can be solved successively for \(\mu \in [0,\delta _{0}], [\delta _{0},2\delta _{0}], \ldots \) . Hence, we can deduce that when \(\mu =1\), for any \((r,\phi ,\varphi )\in \mathcal{N}^{2}[0,T]\), \(\lambda \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\), MF-FBSΔEs (3.3) have a unique solution. Let \(r_{t}=\phi _{t}=\varphi _{t}=\lambda =0\), we conclude that MF-FBSΔE (1.1) has a solution. □
Proof of Lemma 3.1
We consider the following BSΔE:
Using Theorem 2.1, the above equation admits a unique solution \((Y^{\ast },Z^{\ast },M^{\ast })\). Then we solve the following forward equation:
Let \(Y=Y^{\ast }+X\), \(Z=Z^{\ast }\), and \(M=M^{\ast }\), we can see that \((X,Y,Z,M)\) is a solution of (3.2). Thus, the existence is finished. With regards to the uniqueness, it suffices to apply the method of the proof of uniqueness in Theorem 3.1; here, we omit it. □
Proof of Lemma 3.2
Notice that
Set \(\varLambda ^{i}= (X^{i},Y^{i},\mathbb{E}X^{i},\mathbb{E}Y^{i} )\), \(\varTheta ^{i}= (X^{i},Y^{i},Z^{i} )\), \(\varGamma ^{i}= (X^{i},Y^{i},Z^{i},\mathbb{E}X^{i},\mathbb{E}Y^{i}, \mathbb{E}Z^{i} )\), and \(\varGamma ^{0}=0\) to solve iteratively the following equations:
Then we apply the product rule to \(\hat{X}_{t}^{i+1}\hat{Y}_{t}^{i+1}\) yielding
where \(\hat{\varGamma }_{t}^{i}=\varGamma _{t}^{i}-\varGamma _{t}^{i-1}\) and \(\hat{\varTheta }_{t}^{i}=\varTheta _{t}^{i}-\varTheta _{t}^{i-1}\). Set \(\beta _{0}=\min \{1,\beta \}\), we claim that
Let \(\varepsilon =\frac{\beta _{0}}{\delta (1+c)}\), by means of \(ab\leq \frac{a^{2}}{2\varepsilon }+\frac{\varepsilon b^{2}}{2}\), we have
which indicates
Then
where \(\beta _{1}>0\) and it only relies on c and T. Thus, there exists \(\beta _{2}>0\) relying on c, β, and T such that
Furthermore, there exists \(\bar{\delta }\in (0,1)\) relying on c, β, and T such that, for \(0<\delta \leq \bar{\delta }\),
By [30, Lemma 4.1], we see that \(\{\varTheta _{t}^{i}\}_{t=0}^{T-1}\) is a Cauchy sequence in \(\mathcal{K}^{2}[0,T]\). Denote its limit by \(\varTheta =(X,Y,Z)\). Taking the limit in (3.4), we can derive that, when \(0<\delta \leq \bar{\delta }\), \(\varTheta =(X,Y,Z)\) solves (3.3) for \(\mu =\mu _{0}+\delta \). The proof is completed. □
For simplicity, for \(\rho =b, \sigma , f, g\) and \(a=x, y, z, \tilde{x}, \tilde{y}, \tilde{z}, u\), we use the following abbreviations:
Let \((k,m,n,N)\) be a solution of the following variational equations:
We proceed to introducing the following adjoint equations:
Here, W, N, V are square integrable martingale processes and N, V are strongly orthogonal to W. Set
then we get
For MF-FBSΔE (1.1), we give the following estimates.
Lemma 3.3
Assume that (A2)–(A5) hold, we get
Proof
According to (3.7), we have
Combining the above equation with the monotonic conditions, we obtain
On the other hand, there is a constant \(c_{1}>0\) such that
Similarly, we can deduce
Using (3.8)–(3.9), we finally get
The proof is completed. □
Remark 3.1
Under (A2)–(A5), we have the following results.
If \(t=1,2,\ldots ,T-1\),
If \(t=0\),
If \(t=T\),
Consequently, the coefficients of (3.5) satisfy the monotonic conditions and there exists a unique solution \((k,m,n,N)\) to (3.5). Following the proof of Lemma 3.3, it is easy to check
Lemma 3.4
Assume that (A2)–(A5) hold, we get
Proof
Observe that
where
with \(a=x, y, z, \tilde{x}, \tilde{y}, \tilde{z}, u\), then we have
where
From (3.10), it yields that
where
Noticing Remark 3.1, we derive
On the other hand,
Let \(\varepsilon =\frac{1}{\beta }\), by virtue of \(ab\leq \frac{a^{2}}{8\varepsilon }+2\varepsilon b^{2}\), we have
Combining \(\lim_{\varepsilon \rightarrow 0} \Vert \tilde{f}_{\varGamma }(t)- \bar{f}_{\varGamma }(t) \Vert =0\) with Lemma 3.3, we obtain
In a similar way, we have
Thus,
The proof is completed. □
Remark 3.2
From Lemma 3.4, we see that \(k_{t}\) is the first-order variation of \(X_{t}\), and \(m_{t}\) is the first-order variation of \(Y_{t}\). It is easy to derive
4 Stochastic maximum principle
In this section, we are devoted to establishing the stochastic maximum principle for fully coupled MF-FBSΔE (1.1). Define the following Hamiltonian function:
Theorem 4.1
(Stochastic maximum principle)
Assume that (A2)–(A5) hold. Let\(\bar{u}_{t}\)be the optimal control and\((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t} )\)be the corresponding optimal trajectory of Problem\(\mathcal{A}\), then for\(v_{t}\in U_{t}\)and\(\alpha (t)\in \mathcal{I}(t,v_{t})\), one has
Proof
By (3.5)–(3.6), for \(t\in \{0,1,\ldots ,T-1\}\), we obtain
where
Since W, V are martingale processes and V is strongly orthogonal to W, then \(\mathbb{E}[\varPhi _{t}]=0\). Similarly,
where
We further derive
Accordingly,
Since \(k_{0}=0\), \(\xi _{0}=0\), then the above equation leads to
Combining \(\lim_{\varepsilon \rightarrow 0}\frac{1}{\varepsilon }[J(u^{ \varepsilon }_{t})-J(\bar{u}_{t})]\geq 0\) with (3.11), we have
then (4.1) holds owing to the arbitrariness of θ. □
Remark 4.1
Theorem 4.1 establishes a more general and constructive stochastic maximum principle under weakened convexity assumption. To be specific, if the set \(U_{t}\) is not convex, then the discrete-time stochastic maximum principles in [15, 21] are invalid. In this sense, our work generalizes and strengthens the discrete-time stochastic maximum principle of the existing works.
Corollary 4.1
Assume that (A2)–(A4) hold. Let\(\bar{u}_{t}\)be the optimal control and\((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t} )\)be the corresponding optimal trajectory of Problem\(\mathcal{A}\). Moreover, assume that the set\(U_{t}\)isγ-convex, \(t\in \mathbb{T}\), then for\(v_{t}\in U_{t}\), one has
Corollary 4.2
Assume that (A2)–(A4) hold. Let\(\bar{u}_{t}\)be the optimal control and\((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t} )\)be the corresponding optimal trajectory of Problem\(\mathcal{A}\). Moreover, assume that\(U_{t}=\operatorname{ri}U_{t}\), \(t\in \mathbb{T}\), then for\(v_{t}\in U_{t}\), one has
In what follows, we discuss assumptions, under which the necessary condition (4.1) turns into a sufficient one.
Theorem 4.2
(Sufficient conditions for optimality)
Under (A2)–(A5), assume that\(h(\cdot ,\cdot )\)is convex and\(\mathscr{H}(t,\cdot ,\cdot ,\cdot ,\cdot ,\cdot ,\cdot , \eta _{t}, \zeta _{t},\xi _{t},\cdot )\)is convex. Then\(\bar{u}_{t}\)is an optimal control of Problem\(\mathcal{A}\)if (4.1) holds.
Proof
Let \(u_{t}\) be an arbitrary admissible control and \((X_{t},\mathbb{E}X_{t},Y_{t},\mathbb{E}Y_{t},Z_{t},\mathbb{E}Z_{t})\) be the corresponding trajectory. Set \(\acute{X}_{t}=X_{t}-\bar{X}_{t}\) and \(\acute{Y}_{t}=Y_{t}-\bar{Y}_{t}\). Since \(\acute{X}_{0}=\acute{Y}_{T}=\xi _{0}=0\), it is derived that
Denote by \((\partial _{(x,y,z,\tilde{x},\tilde{y},\tilde{z},u)} \bar{\mathscr{H}} )(t)\), etc., the Clarke generalized gradients of \(\mathscr{H}\) evaluated at \((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t}, \mathbb{E}\bar{X}_{t},\mathbb{E}\bar{Y}_{t}, \mathbb{E}\bar{Z}_{t}, \bar{u}_{t} )\). Together with the stochastic maximum principle (4.1), it follows that
By [31, Lemma 2.3], \(( (\partial _{a}\bar{\mathscr{H}} )(t),0 ) \in (\partial _{a,u}\bar{\mathscr{H}} )(t)\), a.e. t. Besides, from [31, Lemma 2.2(4)], we get
Therefore, along with the Hamiltonian function, it yields that
On the other hand, using the convexity assumption on \(h(\cdot ,\cdot )\), we claim that
Thus,
The proof is completed. □
5 A Stackelberg game of MF-BSΔE
As an application, in this section, we consider a Stackelberg game of MF-BSΔE:
The cost functionals for the follower and the leader are given, respectively, as follows:
Here, \(u_{t}\), \(v_{t}\) denote the control processes of the follower and leader, respectively. The admissible control sets are given by
In this section, we make the following assumptions. Set \(\varGamma =(y,z,\tilde{y},\tilde{z})\), \(i=1, 2\).
- (H1)
- (a)
∀Γ, \(u\in \mathscr{U}_{1}[0,T]\), \(v\in \mathscr{U}_{2}[0,T]\), \(f(\cdot ,\varGamma ,u,v)\), \(g_{i}(\cdot ,\varGamma ,u,v)\) are \(\mathfrak{F}_{t}\)-adapted processes.
- (b)
\(g_{i}(t,0,0,0)\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R})\), \(f(t,0,0,0)\in \mathcal{L}^{2}(\mathfrak{F}_{t};\mathbb{R}^{n})\), \(h_{i}(y,\tilde{y})\in \mathcal{L}^{2}(\mathfrak{F}_{0};\mathbb{R})\).
- (c)
\(\forall t\in \mathbb{T}\), \(f(t,\cdot ,\cdot ,\cdot )\), \(g_{i}(t,\cdot ,\cdot ,\cdot )\) are uniformly Lipschitz continuous and differentiable on Γ, u, v; \(h_{i}(\cdot ,\cdot )\) are continuously differentiable on y, ỹ and all derivatives are uniformly bounded.
- (d)
The function f is independent of z, z̃ at \(t=T\).
- (a)
- (H2)
The functions \(f(t,\cdot ,\cdot ,\cdot )\), \(g_{i}(t,\cdot ,\cdot ,\cdot )\) are twice continuously differentiable on Γ, u, v; \(h_{i}(\cdot ,\cdot )\) are twice continuously differentiable on y, ỹ, and all derivatives are uniformly bounded.
- (H3)
The set \(U_{t}\)\((t\in \mathbb{T})\) is ±γ-convex.
Besides, throughout this section, we formally denote \(f(0,\varGamma ,u,v)=g_{i}(0,\varGamma ,u,v)\equiv 0\), \(i=1,2\). The optimal control problem to be solved can be stated in the following definition.
Definition 5.1
The pair \((\bar{u},\bar{v})\in \mathscr{U}_{1}[0,T]\times \mathscr{U}_{2}[0,T]\) is called an optimal solution to the Stackelberg game of MF-BSΔE if it satisfies the following statements:
- (a)
Given \(\kappa \in \mathcal{L}^{2} (\mathfrak{F};\mathbb{R}^{n} )\), there is a mapping \(l:\mathscr{U}_{2}[0,T]\times \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n}) \rightarrow \mathscr{U}_{1}[0,T]\) such that
$$ J_{1} \bigl(l(v_{t},\xi ),v_{t};\kappa \bigr)=\min_{u_{t}\in \mathscr{U}_{1}[0,T]}J_{1}(u_{t},v_{t}; \kappa ),\quad \forall v_{t}\in \mathscr{U}_{2}[0,T]. $$ - (b)
There exists unique \(\bar{v}_{t}\in \mathscr{U}_{2}[0,T]\) such that
$$ J_{2} \bigl(l(\bar{v}_{t},\kappa ), \bar{v}_{t};\kappa \bigr)=\min_{v_{t} \in \mathscr{U}_{2}[0,T]}J_{2} \bigl(l(\bar{v}_{t},\kappa ),v_{t}; \kappa \bigr). $$ - (c)
The optimal strategy of the follower is \(\bar{u}_{t}=l(\bar{v}_{t},\kappa )\).
5.1 Optimization for the follower
In view of the hierarchy property of the leader-follower game, the follower’s optimization problem is firstly considered in this subsection. Denote by \(\bar{u}_{t}\) and \((\bar{Y}_{t},\bar{Z}_{t} )\), respectively, the optimal control and optimal trajectory. To begin with, we consider the admissible control \(u^{\varepsilon }_{t}=\bar{u}_{t}+\alpha (t)\varepsilon (u_{t}- \bar{u}_{t})\), where \(u_{t}\in U_{t}\), \(\alpha (t)\in \mathcal{I}(t,u_{t})\). We construct a needle variation
where \(\alpha \in \mathcal{I}(\theta ,u_{\theta })\), \((\theta ,u_{\theta }) \in T\times U_{\theta }\), and \(\varepsilon \in (0,\gamma ^{\ast }_{1}]\). Let \((Y^{\varepsilon }_{t},Z^{\varepsilon }_{t})\) be the state trajectory corresponding to the control \(u^{\varepsilon }_{t}\). Set \(\varGamma _{t}=(Y_{t},\mathbb{E}Y_{t},Z_{t},\mathbb{E}Z_{t})\). We introduce the following variational equation:
and the adjoint equation
Define the Hamiltonian function
Using Theorems 4.1–4.2, we can immediately obtain the following statements.
Theorem 5.1
(Stochastic maximum principle)
Let (H1), (H3) hold and\(\kappa \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\). Given the leader’s strategy\(v_{t}\in \mathscr{U}_{2}[0,T]\), assume that\((\bar{Y}_{t},\bar{Z}_{t} )\)is the optimal trajectory and\(\bar{u}_{t}\)is the optimal control of the follower, then for any\(u_{t}\in U_{t}\), one has
Theorem 5.2
(Sufficient conditions for optimality)
Let (H1), (H3) hold and\(\kappa \in \mathcal{L}^{2}(\mathfrak{F};\mathbb{R}^{n})\). Given the leader’s strategy\(v_{t}\in \mathscr{U}_{2}[0,T]\), assume that\(h_{1}(\cdot ,\cdot )\)is convex and\(\mathscr{H}_{1}(t,\cdot ,\cdot ,\cdot ,\cdot , \cdot , v_{t},X_{t})\)is concave and Lipschitz continuous. Then\(\bar{u}_{t}\)is an optimal control of the follower’s problem if it satisfies (5.2).
5.2 Optimization for the leader
Notice that the follower’s optimal response \(\bar{u}_{t}\) can be determined by the leader, the state equation of the leader turns to be a MF-FBSΔE:
In this subsection, for \(a=y, \tilde{y}, z, \tilde{z}, v, x, \tilde{x}\), we use the following abbreviations:
Likewise, we consider the admissible control
where \(\alpha \in \mathcal{I}(\theta ,v_{\theta })\), \((\theta ,v_{\theta }) \in \mathbb{T}\times U_{\theta }\), and \(\varepsilon \in (0,\gamma ^{\ast }_{2}]\). Let \((X^{\varepsilon }_{t},Y^{\varepsilon }_{t},Z^{\varepsilon }_{t})\) be the state trajectory corresponding to the admissible control \(v^{\varepsilon }_{t}\). Let \((k,\eta ,\rho ,N)\) be a solution of the following variational equations:
We define the Hamiltonian
Similarly, we proceed to introducing the following adjoint equations associated with MF-FBSΔE (5.3):
Here, W, N, V are square integrable martingale processes and N, V are strongly orthogonal to W.
The following conclusions are straightforward with the aid of Theorems 4.1–4.2.
Theorem 5.3
(Stochastic maximum principle)
Assume that (H1)–(H3) hold. Let\(\bar{v}_{t}\)and\((\bar{X}_{t},\bar{Y}_{t},\bar{Z}_{t} )\)be the optimal control, optimal trajectory, respectively. Then, for\(v_{t}\in U_{t}\), one has
Theorem 5.4
(Sufficient conditions for optimality)
Under (H1)–(H3), assume that\(h_{2}(\cdot ,\cdot )\)is convex and\(\mathscr{H}_{2}(t,\cdot ,\cdot ,\cdot ,\cdot , \bar{u}_{t},\cdot , \cdot ,\cdot ,p_{t},\zeta _{t},q_{t})\)is convex, then\(\bar{v}_{t}\)is an optimal control of the leader’s problem if (5.4) holds.
6 Perspectives and open problems
In this section, we give a brief exposition on the prospects that are open to the researchers. The following topics shall be explored in our future works.
Firstly, we see that the effectiveness of optimality conditions obtained in this paper substantially relies on the structure of the set \(U_{t}\). So it is pregnant to discuss more general and essential convexity assumptions for the discrete-time forward–backward stochastic system.
Secondly, there are many more partially observable cases which are more constructive and inevitable for applications and are technologically demanding in their filtering procedure.
References
Yong, J.M., Zhou, X.Y.: Stochastic Controls, Hamiltonian Systems and HJB Equations. Springer, Berlin (1999)
Bismut, J.M.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20, 62–78 (1978)
Dokuchaev, N., Zhou, X.Y.: Stochastic controls with terminal contingent conditions. J. Math. Anal. Appl. 238(1), 143–165 (1999)
Ma, H.P., Liu, B.: Optimal control of mean-field jump-diffusion systems with noisy memory. Int. J. Control 92(4), 816–827 (2019)
Peng, S.G.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990)
Li, R.J., Liu, B.: A maximum principle for fully coupled stochastic control systems of mean-field type. J. Math. Anal. Appl. 415(2), 902–930 (2014)
Wu, Z.: A general maximum principle for optimal control problems of forward–backward stochastic control systems. Automatica 49, 1473–1480 (2013)
Xu, R.M., Zhang, L.Q.: Stochastic maximum principle for mean-field controls and non-zero sum mean-field game problems for forward–backward systems. arXiv:1207.4326
Yong, J.M.: Optimality variational principle for controlled forward–backward stochastic differential equations with mixed initial-terminal conditions. SIAM J. Control Optim. 48(6), 4119–4156 (2010)
Antonelli, F.: Backward–forward stochastic differential equations. Ann. Appl. Probab. 3, 777–793 (1993)
Ma, J., Yong, J.M.: Forward–Backward Stochastic Differential Equations and Their Applications. Springer, Berlin (1999)
Butkovskii, A.G.: On necessary and sufficient optimality conditions for impulse control systems. Avtom. Telemeh. 24(8), 1056–1064 (1963)
Pshenichnyi, B.N.: Necessary Conditions for an Extremum. Nauka, Moscow (1968). English translation in Dekker, New York (1971)
Mahmudov, N.I.: Necessary first-order and second-order optimality conditions in discrete-time stochastic systems. J. Optim. Theory Appl. 182, 1001–1018 (2019)
Lin, X.Y., Zhang, W.H.: A maximum principle for optimal control of discrete-time stochastic systems with multiplicative noise. IEEE Trans. Autom. Control 60(4), 1121–1126 (2015)
Xu, J.J., Zhang, H.S., Xie, L.H.: General linear forward and backward stochastic difference equations with applications. Automatica 96, 40–50 (2018)
Mahmudov, N.I.: Maximum principle for stochastic discrete-time Itô equations. In: Brownian Motion: Elements, Dynamics and Applications, Chap. 6, pp. 1–22 (2015)
Mardanov, M.J., Melikov, T.K.: A new discrete analogue of Pontryagin’s maximum principle. Dokl. Math. 98(3), 549–551 (2018)
Mardanov, M.J., Melikov, T.K., Malik, S.T.: On strengthening of optimality conditions in discrete control systems. Proc. Inst. Math. Mech. 44(1), 135–154 (2018)
Mardanov, M.J., Melikov, T.K., Malik, S.T., Malikov, K.: First- and second-order necessary conditions with respect to components for discrete optimal control problems. J. Comput. Appl. Math. 364, 112342 (2020)
Ji, S.L., Liu, H.D.: Maximum principle for stochastic optimal control problem of forward–backward stochastic difference systems. arXiv:1812.11283
Buckdahn, R., Djehiche, B., Li, J., Peng, S.G.: Mean-field backward stochastic differential equations: a limit approach. Ann. Probab. 37, 1524–1565 (2009)
Ma, H.P., Liu, B.: Maximum principle for partially observed risk-sensitive optimal control problems of mean-field type. Eur. J. Control 32, 16–23 (2016)
Bielecki, T.R., Cialenco, I., Chen, T.: Dynamic conic finance via backward stochastic difference equations. SIAM J. Financ. Math. 6(1), 1068–1122 (2015)
Cohen, S.N., Elliott, R.J.: A general theory of finite state backward stochastic difference equations. Stoch. Process. Appl. 120(4), 442–466 (2010)
Cohen, S.N., Elliott, R.J.: Backward stochastic difference equations and nearly time-consistent nonlinear expectations. SIAM J. Control Optim. 49(1), 125–139 (2011)
Andersson, A., Djehiche, B.: A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63, 341–356 (2011)
Leichtweis, K.: Convex Sets. Nauka, Moscow (1985)
Föllmer, H., Schied, A.: Stochastic Finance: An Introduction in Discrete Time, extended edn. de Gruyter Studies in Mathematics, vol. 27. de Gruyter, Berlin (2004)
Hu, Y., Peng, S.G.: Solution of forward–backward stochastic differential equations. Probab. Theory Relat. Fields 103(2), 273–283 (1995)
Zhou, X.Y.: Sufficient conditions of optimality for stochastic systems with controllable diffusions. IEEE Trans. Autom. Control 41, 1176–1179 (1996)
Acknowledgements
The authors would like to thank the anonymous referees and editor very much for helpful comments and suggestions which led to the improvement of presentation and quality of the work.
Availability of data and materials
Not applicable.
Funding
This work is partially supported by the National Natural Science Foundation of China (Grant No. 11971185).
Author information
Authors and Affiliations
Contributions
All authors contributed equally and significantly in this manuscript, and they read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Song, T., Liu, B. A maximum principle for fully coupled controlled forward–backward stochastic difference systems of mean-field type. Adv Differ Equ 2020, 188 (2020). https://doi.org/10.1186/s13662-020-02640-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13662-020-02640-x