- Research
- Open access
- Published:
Maximum principle for delayed stochastic mean-field control problem with state constraint
Advances in Difference Equations volume 2019, Article number: 348 (2019)
Abstract
In this paper, we consider the optimal control problem for the mean-field stochastic differential equations with delay and state constraint. By virtue of the classical Ekeland’s variational principle, the duality method and a new type of mean-field anticipated backward stochastic differential equation, we obtain the maximum principle of the optimal control for this problem. Our result can be applied to a harvest model from a mean-field system with delay.
1 Introduction
Mean-field stochastic differential equations (MFSDEs), also called McKean–Vlasov equations, were discussed by Kac [1]. And this study was initiated by McKean [2]. Here, we write a class of commonly used McKean–Vlasov equations as follows:
where \(\{B(t)\}_{t\geq 0}\) is an 1-dimensional standard Brownian motion, defined on a complete probability space \((\varOmega ,\mathcal{F}, \mathbb{P})\), denoting \(\mathbb{P}_{X}=\mathbb{P}\circ X^{-1}\) as the law of the random variables X. And the coefficients \((b,\sigma ):[0,T] \times \varOmega \times \mathbb{R}^{d}\times \mathcal{P}_{2}(\mathbb{R} ^{d})\mapsto \mathbb{R}\) are measurable functions. Here, \(\mathcal{P} _{2}(\mathbb{R}^{d})\) is the space of all probability measures on \(\mathbb{R}^{d}\), equipped with 2-Wassertein metric. For more details refer to [3].
Equation (1) has been adapted by many researchers, Buckdanhn, Li and Ma [4] studied the general stochastic control problem of Eq. (1). For other, related work, we refer to [5, 6]. Buckdahn, Li and Peng [7, 8] proposed a new kind of backward stochastic differential equations, called mean-field backward stochastic differential equations (MFBSDEs), coupled with MFSDEs. Carmona [9] studied the forward–backward mean-field stochastic differential equations (MFFBSDEs). The pioneering work of mean-field games was done by Lion and Lasry [3, 10]. They apply Eq. (1) to study the mean-field potential games in the large players and symmetric equilibrium. More about the mean-field games is in [3, 11].
In fact, Eq. (1) is too general to be used in the real world. Therefore, many researchers gave many different mean-field definitions. As pointed out by Buckdahn, Li and Ma [4], the main difference of these definitions is in the position of the expectation taken in the current literature. And they classified this conclusion in the following two types:
where φ is the coefficients function in Eq. (1), i.e. \(\varphi = b,\sigma \). φ̃ is a different function corresponding to b and σ. Li [12] studied the type (i) mean-field control problem. For type (ii) MFSDEs, Buckdahn, Djehiche and Li [7] proved the general stochastic maximum principle. Yong [13] solved a linear–quadratic (LQ) optimal problem for type (ii) MFSDEs by using a decoupling technique. Li et al. [14, 15] further studied the closed-loop optimal LQ control problem and the LQ problem in infinite horizon. Andersson and Djehiche [16] generalized this kind of mean-field definition and we give details for Eq. (3):
where ψ is some general nonlinear function. If we let \(\psi (x)=x\), then (3) degenerates to (ii). They obtained Pontryagin’s maximum principle for the optimal control problem. Hu and Øksandal [17] investigated this type singular optimal control, and they apply these results to prove the Nash equilibrium and zero-sum equilibrium. This kind of models can be used to describe the large population interacting system.
The study of the optimal control problem with delay also captured lots of attention. Many results were obtained, such as by Øksendal, Suem [18], Chen and Wu [19, 20]. The main reason is that real world systems not just depend on their current state, but also their previous history. Optimal control problems of the delayed systems are very difficult because of the infinite-dimensional state space structure. Therefore current research is divided into two kinds of methods to develop the stochastic maximum principle for delayed systems. One involves adjoint equation; it is given by the anticipated BSDE which was introduced by Peng and Yang [21]; see Chen and Wu [19, 20], Yu [22] and Zhang [23]. Another method is to derive a system of three-coupled adjoint equations, which consists of two BSDEs and one backwards ordinary stochastic equation; see Øksendal and Sulem [18].
Many studies of the real problems show that there are state constraints in stochastic optimal control problems. And then we need Ekeland’s variational principle to deal with the problem with state constraint; see [24, 25] and [23]. To the best of the authors’ knowledge, there are few results about the delayed stochastic optimal control problem. Reference [23] discussed this kind of problems, but only for the linear–quadratic (LQ) case. The aim of this article is to study the stochastic optimal control problem of mean-field system with delay and state constraints. We try to solve such problems in the light of finding the optimal controls. Both the delays and the state constraints will bring about trouble. Under some suitable conditions, we first prove the uniqueness and existence of solutions for the mean-field SDDE and mean-field anticipated BSDE. To develop our maximum principles, we follow the aforementioned first method to construct the adjoint equation. We adopt the non-convex control domain rather than the convex control domain. By Ekeland’s variational principle, we derive a necessary condition of the optimal control. The maximum principle differs from the classical one for the adjoint equation will be a new kind of mean-field anticipated BSDEs.
The rest of this paper is structured as follows: In Sect. 2,we give the preliminary results as regards mean-field SDDEs and mean-field anticipated BSDEs. The stochastic optimal control problem is formulated in Sect. 3. We derive the stochastic maximum principle for a stochastic control system in Sect. 4. We apply our result to real world model in Sect. 5 to finalize this paper.
2 Preliminary results
Let \((\varOmega ,\mathcal{F},\mathbb{F},P)\) be a complete filtered probability space. \(\{B(t)\}_{t\geq 0}\) is a standard 1-dimensional Brownian motion. \(\mathbb{F}=\{\mathcal{F}_{t}\}_{t\geq 0}\) is the natural filtration augmented by all P-null elements of \(\mathcal{F}\). \(T > 0\) and \(\delta \geq 0\) are given constants. For simplicity of notation, we only consider the 1-dimensional case in this paper and all results can be extended to multidimensional cases without difficulty. The norm in \(\mathbb{R}\) is denoted by \(\vert \cdot \vert \). We will also use the following notation for some positive integer n:
-
\(L^{n}(\mathcal{F}_{T},\mathbb{R}):= \{\xi :\xi \text{ is } \mathbb{R}\text{-valued }\mathcal{F}_{T}\text{-measurable random variable s.t. } \mathbb{E}\vert \xi \vert ^{n}<+\infty \}\);
-
\(L^{n}_{\mathbb{F}}(s,r;\mathbb{R}):= \{\psi (t):\{\psi (t),s \leq t\leq r\}\text{ is }\mathbb{R}\text{-valued adapted stochastic process s.t. } \mathbb{E}\int ^{r}_{s}\vert \psi (t) \vert ^{n}\,dt<+\infty \}\);
-
\(\mathcal{S}^{n}_{\mathbb{F}}(s,r;\mathbb{R}):= \{\psi (t): \{\psi (t),s\leq t\leq r\}\text{ is }\mathbb{R}\text{-valued adapted stochastic process s.t. } [4]\text{with right-continuous path and left limit s.t. } \mathbb{E}[ \sup_{s\leq t\leq r}\vert \psi (t)\vert ^{n}]<+\infty \}\).
2.1 Mean-field stochastic differential equation with delay
In this subsection, we will show some preliminary results on mean-field stochastic differential equations with delay (MFSDDEs) and mean-field anticipated backward stochastic differential equations (MABSDEs). These theoretical results include the existence and uniqueness of the solutions, some estimations of the solutions and the duality relationship between the MFSDDEs and MFABSDEs.
Firstly, we consider the following MFSDDE:
where
When there are no delays, the above MFSDDE will degenerate to the MFSDE in [16]. Let us introduce Assumption (H1), where x denotes the state variables, μ the expected value:
- (H1.1):
-
\(x,x_{\delta },\mu _{1},\mu _{1\delta },\mu _{2}, \mu _{2 \delta },\bar{x},\bar{x}_{\delta },\bar{\mu }_{1},\bar{\mu } _{1\delta },\bar{\mu }_{2},\bar{\mu }_{2\delta }\in \mathbb{R}\), there exists a constant C, s.t.
$$ \begin{aligned} & \bigl\vert b(t,x,x_{\delta },\mu _{1},\mu _{1\delta })-b(t,\bar{x},\bar{x} _{\delta }, \bar{\mu }_{1},\bar{\mu }_{1\delta }) \bigr\vert + \bigl\vert \sigma (t,x,x _{\delta },\mu _{2},\mu _{2\delta })-\sigma (t,\bar{x},\bar{x}_{\delta },\bar{\mu }_{2},\bar{\mu }_{2\delta }) \bigr\vert \\ &\quad \leq C\bigl( \vert x-\bar{x} \vert + \vert x_{\delta }- \bar{x}_{\delta } \vert + \vert \mu _{1}-\bar{\mu }_{1} \vert + \vert \mu _{1\delta }-\bar{ \mu }_{1\delta } \vert + \vert \mu _{2}-\bar{\mu }_{2} \vert + \vert \mu _{2\delta }-\bar{\mu }_{2\delta } \vert \bigr); \end{aligned} $$ - (H1.2):
-
for any \(x,x_{\delta },\mu _{1},\mu _{1\delta },\mu _{2}, \mu _{2\delta }\in \mathbb{R}\), the mappings \(b(\cdot ,x,x_{\delta }, \mu _{1},\mu _{1\delta })\) and \(\sigma (\cdot ,x,x_{\delta },\mu _{2}, \mu _{2\delta })\) are \(\mathbb{F}\)-adapted and \(b(\cdot ,0,0,0,0), \sigma (\cdot ,0,0,0,0) \in L^{2}_{\mathbb{F}}(0,T;\mathbb{R})\);
- (H1.3):
-
ψ and φ are continuously differentiable and their derivatives are bounded.
Theorem 2.1
Suppose that Assumption (H1) holds, and \(x_{0}(\cdot )\in L^{2}_{ \mathbb{F}}(-\delta ,0;\mathbb{R})\). Then MFSDDE (4) has a unique t-continuous solution \(x(t)\) and \(E\sup_{0\leq t \leq T} \vert x(t)\vert ^{2} < +\infty \).
Proof
Let us introduce a norm in Banach space \(L_{\mathbb{F}}^{2}(-\delta ,T; \mathbb{R})\)
Clearly it is equivalent to the original norm of \(L_{\mathbb{F}}^{2}(- \delta ,T;\mathbb{R})\). We define a mapping \(h:L_{\mathbb{F}}^{2}(- \delta ,T;\mathbb{R})\rightarrow L_{\mathbb{F}}^{2}(-\delta ,T; \mathbb{R})\) by the following equation s.t. \(h[X(\cdot )]=x(\cdot )\):
We desire to prove that h is a contraction mapping under the norm \(\Vert \cdot \Vert _{\beta }\). For arbitrary \(X(\cdot ),\bar{X}(\cdot )\in L_{\mathbb{F}}^{2}(-\delta ,T;\mathbb{R})\), set \(h[X(\cdot )]=x( \cdot )\), \(h[\bar{X}(\cdot )]=\bar{x}(\cdot )\), and \(\hat{X}(\cdot )=X( \cdot )-\bar{X}(\cdot )\), \(\hat{x}(\cdot )=x(\cdot )-\bar{x}(\cdot )\). Then \(\hat{x}(\cdot )\) satisfies
Denoting
applying the Itô formula to \(e^{-\beta t}\vert \hat{x}(t)\vert ^{2}\) on [0,T], we have
By (H1.1) and the Cauchy–Schwarz inequality we obtain
Here \(\triangle _{1}\leq 64C^{2}\mathbb{E}\int ^{T}_{-\delta }e^{- \beta t}\vert \hat{X}_{t}\vert ^{2}\,dt\), and by (H1.3) and the mean value theorem we also have
where \(X^{\eta }(t)\) is the value among \(X(t)\) and \(\bar{X}(t)\) and the constant C changes line by line. Similarly, \(\triangle _{3}\leq 32C ^{2}\mathbb{E}\int ^{T}_{-\delta }e^{-\beta t}\vert \hat{X}(t)\vert ^{2}\,dt\).
We select \(\beta =256C^{2}+1\); then
or
which shows that h is a strict contraction mapping. Then it follows from the fixed point theorem that the MFSDDE (4) has a unique solution in \(L^{2}_{\mathbb{F}}(-\delta ,T;\mathbb{R})\). Since b and σ satisfy (H1), we can easily derive that \(\mathbb{E} [\sup_{0\leq t\leq T}\vert x(t)\vert ^{2} ]<+\infty \). □
2.2 Mean-field anticipated backward stochastic differential equation
In this subsection, we consider the following MABSDE:
where the coefficients \((f,\xi ,\xi _{0},\eta _{0})\) satisfy Assumption (H2):
- (H2.1):
-
\(\xi _{0}(\cdot ),\eta _{0}(\cdot )\in L^{2}_{\mathbb{F}}(T,T+ \delta ; \mathbb{R})\) and \(\xi \in L^{2}(\mathcal{F}_{T},\mathbb{R})\);
- (H2.2):
-
for any \(t\in [0,T]\), \(y,z\in \mathbb{R}\), \(y_{\delta },z _{\delta }\in L^{2}(\varOmega ,\mathcal{F}_{t+\delta };\mathbb{R})\), \(f(t,y,y_{\delta },\mathbb{E}\psi (y),\mathbb{E}\psi (y_{\delta }), z, z_{\delta }, \mathbb{E}\varphi (z), \mathbb{E}\varphi (z _{\delta }))\) is \(\mathcal{F}_{t}\)-measurable;
- (H2.3):
-
\(f(\cdot ,0,0,0,0,0,0,0,0)\in L^{2}_{\mathbb{F}}(0,T; \mathbb{R})\);
- (H2.4):
-
for any \(t\in [0,T]\), \(y,\bar{y},\mu _{3},\mu _{3\delta },z, \bar{z},\mu _{4},\mu _{4\delta }\in \mathbb{R}\), \(\mu _{3\delta },\bar{ \mu }_{3\delta },z_{\delta },\bar{z}_{\delta },\bar{\mu }_{4},\bar{ \mu }_{4\delta }\in L^{2}(\varOmega , \mathcal{F}_{t+\delta };\mathbb{R})\), there exists a constant M, s.t.
$$ \begin{aligned} &\bigl\vert f(t,y,y_{\delta },\mu _{3},\mu _{3\delta },z,z_{\delta },\mu _{4}, \mu _{4\delta } \\ &\qquad {}-f(t,\bar{y},\bar{y}_{\delta },\bar{\mu }_{3}, \bar{ \mu }_{3\delta },\bar{z},\bar{z}_{\delta },\bar{\mu }_{4},\bar{\mu } _{4\delta })\bigr\vert \\ &\quad \leq M \bigl\{ \vert y-\bar{y} \vert + \vert z- \bar{z} \vert + \vert \mu _{3}-\bar{\mu }_{3} \vert + \bigl\vert \mathbb{E} \varphi (z)-\mathbb{E}\varphi (\bar{z}) \bigr\vert \\ &\qquad{}+\mathbb{E}^{ \mathcal{F}_{t}} \bigl[ \vert y_{\delta }- \bar{y}_{\delta } \vert + \vert z_{\delta }- \bar{z}_{\delta } \vert + \vert \mu _{3\delta }-\bar{\mu } _{3\delta } \vert + \vert \mu _{4\delta }-\bar{\mu }_{4\delta } \vert \bigr] \bigr\} ; \end{aligned} $$ - (H2.5):
-
ψ and φ are continuously differentiable and their derivatives are bounded.
Theorem 2.2
Suppose that Assumption (H2) holds, then the MABSDE (7) admits a unique solution \((y(\cdot ),z(\cdot ))\in \mathcal{S}^{2}_{ \mathbb{F}}(0,T;\mathbb{R})\times L^{2}_{\mathbb{F}}(0,T;\mathbb{R})\). Moreover, if there exist another set of coefficients \((\bar{f},\bar{ \xi },\bar{\xi }_{0},\bar{\eta }_{0})\) satisfying (H2), and we denote by \((\bar{y}(\cdot ),\bar{z}(\cdot ))\) the unique solution of MABSDE with coefficients \((\bar{f},\bar{\xi },\bar{\xi }_{0},\bar{\eta }_{0})\), then we have the following estimate:
where C is a constant depending on the Lipschitz constant M and T.
Proof
On the interval \([T-\delta ,T]\), Eq. (7) becomes
where \(\xi _{0}(\cdot )\), \(\eta _{0}(\cdot )\) are given, i.e. (7) is a mean-field BSDE without anticipation. According to Theorem 3.1 and Lemma 3.1 in [7], (7) admits a unique solution under Assumption (H2) and the estimate (8) also holds for \((y(\cdot ),z(\cdot ))\) on \([T-\delta ,T]\). We can proceed with this argument on \([T-2\delta ,T-\delta ]\), \([T-3\delta ,T-2\delta ]\) step by step. Hence the conclusions in Theorem 2.2 can be proved. □
3 Formulation of the optimal control problem
The purpose of this section is to discuss the optimal control for a mean field with delay system. We consider the system involving delay terms both in the state and the control variables.
Consider the following MFSDDE with control:
with
The control domain U is a nonempty bounded subset of \(\mathbb{R}\). We denote by \(\mathcal{U}=\{v(\cdot )\in L^{2}_{\mathbb{F}}(-\delta ,T; \mathbb{R})|v(t)\in U, \text{for any } t\in [-\delta ,T]\}\). \(\mathcal{U}\) is called a feasible control set. The cost functional is
Problem 3.1
The optimal control problem is to minimize \(J(v(\cdot ))\) over \(v(\cdot )\in \mathcal{U}\) subject to the following final state constraint:
Assumption (H3) will be in force throughout the rest of this paper:
- (H3.1):
-
\(x_{0}(\cdot )\in L^{2}_{\mathbb{F}}(-\delta ,0; \mathbb{R})\), \(v_{0}\in L^{2}_{\mathbb{F}}(-\delta ,0;\mathbb{R})\), and for any x, \(x_{\delta }\), \(\mu _{1}\), \(\mu _{1\delta }\), \(\mu _{2}\), \(\mu _{2\delta }\), \(\mu _{3},\mu _{3\delta },\mu _{4}, v,v_{\delta }\in \mathbb{R}\), the mappings \(b(\cdot , x,x_{\delta },\mu _{1},\mu _{1 \delta },v,v_{\delta })\), \(\sigma (\cdot ,x,x_{\delta },\mu _{2}, \mu _{2\delta })\) and \(L(\cdot , x,x_{\delta },\mu _{3}, \mu _{3\delta },v,v _{\delta })\) are \(\mathbb{F}\)-adapted. \(\varPhi (x,\mu _{4})\) is \(\mathcal{F}_{T}\)-measurable.
- (H3.2):
-
\(b(\cdot ,0,0,0,0,0,0),\sigma (\cdot ,0,0,0,0)\in L^{2} _{\mathbb{F}}(0,T;\mathbb{R})\), \(L(\cdot ,0,0,0,0,0,0)\in L^{1}_{ \mathbb{F}}(0,T;\mathbb{R})\) and \(\varPhi (0,0)\in L^{1}(\mathcal{F}_{T}; \mathbb{R})\).
- (H3.3):
-
b, σ and Φ are continuously differentiable with respect to their own variables and their derivatives are both continuous and uniformly bound. And there exists some constant C, s.t.
$$\begin{aligned}& \begin{aligned} & \bigl\vert b(t,x,x_{\delta },\mu _{1},\mu _{1\delta },v,v_{\delta }) \bigr\vert ^{2} + \bigl\vert L(t,x,x_{\delta },\mu _{3}, \mu _{3\delta },v,v_{\delta }) \bigr\vert ^{2} \\ &\quad \leq C\bigl(1+ \vert x \vert ^{2}+ \vert x_{\delta } \vert ^{2}+ \vert \mu _{1} \vert ^{2}+ \vert \mu _{1\delta } \vert ^{2}+ \vert \mu _{3} \vert ^{2}+ \vert \mu _{3\delta } \vert ^{2}+ \vert v \vert ^{2}+ \vert v_{\delta } \vert ^{2}\bigr), \end{aligned} \\& \bigl\vert \sigma (t,x,x_{\delta },\mu _{2},\mu _{2\delta }) \bigr\vert ^{2}\leq C\bigl(1+ \vert x \vert ^{2}+ \vert x_{\delta } \vert ^{2}+ \vert \mu _{2} \vert ^{2}+ \vert \mu _{2\delta } \vert ^{2}\bigr), \\& \bigl\vert \varPhi (x,\mu _{4}) \bigr\vert ^{2}\leq C \bigl(1+ \vert x \vert ^{2}+ \vert \mu _{4} \vert ^{2}\bigr). \end{aligned}$$ - (H3.4):
-
ψ, φ, ϕ and χ are continuously differentiable and their derivatives are uniformly bounded. And moreover,
$$ \bigl\vert \psi (x) \bigr\vert ^{2}+ \bigl\vert \varphi (x) \bigr\vert ^{2}+ \bigl\vert \phi (x) \bigr\vert ^{2}+ \bigl\vert \chi (x) \bigr\vert ^{2}\leq C \bigl(1+ \vert x \vert ^{2}\bigr), $$for some constant C.
- (H3.5):
-
\(G(x)\) is \(\mathcal{F}_{T}\)-measurable for all \(x\in \mathbb{R}\), \(\mathbb{E}|G(0)|\leq +\infty \), and G is continuously differentiable with bounded derivatives.
If \(v(\cdot )\) is a feasible control and Assumption (H2) holds, by Theorem 2.1, Eq. (9) admits a unique solution denoted by \(x^{v}(\cdot )\in \mathcal{S}^{2}_{\mathbb{F}}(0,T;\mathbb{R})\) and it is easy to check that the cost functional is well defined. If \(v(\cdot )\in \mathcal{U}\) also satisfies final state constraint (11), we will call it the admissible control. We denote the set of the admissible controls by \(\mathcal{U}_{\mathrm{ad}}\).
Remark 3.1
We shall postpone obtaining the maximum principle until the following bounded and continuous dependence for the solution of state function (9). These results will play an important role in exploring the maximum principle of Problem 3.1. One should note that our control domain U is bounded, while the case U is unbounded can be treated via the bounded case with a convergence technique as mentioned in Zhang [23].
Lemma 3.2
There exists a constant \(C>0\), such that, for any \(v(\cdot ),u(\cdot )\in \mathcal{U}\), we have
where we have used the abbreviated notations for \(w=v,u\),
Proof
Firstly, applying basic inequality and B-D-G inequality to Eq. (9), we deduce that
Since b, σ satisfy Assumption (H3.3) and ψ, φ satisfy Assumption (H3.4), we have
The second inequality is obtained by a change of variable. Thus, the required result (12) follows by applying Gronwall’s inequality and the boundedness of the control variables.
Next, let us turn to the continuous dependence result inequality (13). It is easy to obtain
For the first term of the right side of (15), we can get
Because the coefficient σ does not depend on the control variable, it follows that
Thus, we can group terms and apply Gronwall’s inequality to deduce that (13) holds. □
4 Maximum principle of the mean-field optimal control problem with delay
4.1 Variation of the trajectory
Now we let \((u(\cdot ),x^{u}(\cdot ))\) be an optimal solution of Problem 3.1. For the simplicity of notation, we denote \(x^{u}( \cdot )\) by \(x(\cdot )\) in the rest of this paper. Given any \(\tau \in [0,T)\) and \(v(\cdot )\in \mathcal{U}\), let us define the following spike variational control:
where \(0 < \epsilon < \delta \) is sufficiently small and \(\tau + \epsilon \leq T\). It is obvious that \(u^{\epsilon }(t)\in \mathcal{U}\). Let \(x^{\epsilon }(\cdot )\) be the trajectory of system (9) corresponding to \(u^{\epsilon }(\cdot )\). Similarly as above in (14) we will use the short-hand notation \(\varGamma ^{u}(t)\) and \(\varTheta ^{u}(t)\). We introduce the following linear variational equation:
It is easy to check that the variational equation is a linear MFSDDE and it admits a unique solution \(q(\cdot )\in \mathcal{S}^{2}(0,T; \mathbb{R})\). We have the following convergence result.
Lemma 4.1
Suppose (H3) holds. Then there exists \(C>0\), which is independent of ϵ such that
Proof
By the Cauchy–Schwartz inequality and B-D-G inequality we have
Since all the derivatives are bounded and the following results:
we can get
Applying Gronwall’s inequality, the desired conclusion is obtained. □
Lemma 4.2
Suppose (H3) holds, then we have
where \(C_{\epsilon }\) is nonnegative constant and \(C_{\epsilon } \rightarrow 0\) when \(\epsilon \rightarrow 0\).
Proof
Setting \(\tilde{x}(\cdot )=x^{\epsilon }(\cdot )-x(\cdot )-q(\cdot )\), and then \(x^{\epsilon }(\cdot )=\tilde{x}(\cdot )+x(\cdot )+q(\cdot )\).
We will use the following abbreviated notations:
By a Taylor expansion, we can rewrite Eq. (22) and we should point out that the expansion is more complex than the case without mean-field terms in Chen and Wu [20]. We have
where
Applying the conditions of b, σ, ψ, φ and the definition of \(u^{\epsilon }(\cdot )\), we can easily prove that
Here \(o(\epsilon ^{2})\) denotes the element such that \(\lim_{\epsilon \to 0}{\frac{o(\epsilon ^{2})}{\epsilon ^{2}}}=0\). Also, since all the derivatives are bounded, we can group all above terms to deduce that
Then, by the Grownwall inequality, we can prove Lemma 4.2. □
4.2 Necessary maximum principle
We will try to derive the necessary conditions of our optimal control problem in this subsection.
Since we have the final state constraint, we first introduce the following Ekeland variational principle in [26].
Lemma 4.3
(Ekeland’s variational principle)
Let \((S,d(\cdot ,\cdot ))\) be a completed metric space, and let \(F:S\rightarrow \mathbb{R}\) be lower-semicontinuous and bounded from below. If there exists \(u\in S\) such that \(F(u)\leq \inf_{v \in S}F(v)+\rho \) for some \(\rho >0\), then there exists \(u^{\rho } \in S\), such that \(F(u^{ \rho })\leq F(u)\), \(d(u^{\rho },u)\leq \sqrt{ \rho }\), and \(F(v)+\sqrt{\rho }d(u^{\rho },v)>F(u^{\rho })\) for any \(v\neq u^{ \rho }\).
Also we let \((u(\cdot ),x(\cdot ))\) be the optimal control and corresponding trajectory of (9)–(10) under the final state constraint (11). We define a metric d on \(\mathcal{U}\) by
where I is indicator function and then \((\mathcal{U},d)\) is a completed metric space.
Proposition 4.4
Let us define
with \(v(\cdot )\in \mathcal{U}\). \(x^{v}(\cdot )\) is the corresponding trajectory of \(v(\cdot )\). Then \(J_{\rho }(\cdot )\) is bounded and continuous on \(\mathcal{U}\).
Proof
By the definition of \(J(v(\cdot ))\) and (H3), the bounded of \(J_{\rho }(\cdot )\) is obvious. And moreover, for any \(v(\cdot ), \omega (\cdot )\in \mathcal{U}\),
Here
We see that terms within the first part on the right side of (30) are bounded by some constant C. By the continuity of L, Φ and their derivatives are bounded we have
Then, by Lemma 3.2, we can deduce
While \(\hat{J}_{2}\) can be expanded as
Combining (29)–(33), we can derive the continuous property of \(J_{\rho }(v(\cdot ))\). □
Remark 4.5
One can notice that \(J_{\rho }(\cdot )\) is defined on the feasible control set \(\mathcal{U}\) rather than the admissible control set \(\mathcal{U}_{\mathrm{ad}}\). It means that we can get rid of the state constraint by the new cost functional.
Now we consider the following free final state optimal control problem:
It is easy to verify that
and
According to Ekeland’s variational principle, there exists \(u^{\rho }( \cdot )\in \mathcal{U}\) such that
We can get the necessary conditions of \(u^{\rho }(\cdot )\) and then take \(\rho \downarrow 0\) to derive the proper conditions of \(u(\cdot )\).
Applying the “spike variation method”, we can construct a \(u^{\epsilon \rho }(\cdot )\in \mathcal{U}\) for any \(\epsilon >0\) as follows:
where \(0<\epsilon <\delta \) and \(\tau +\epsilon \leq T\). And then
Let \(x^{\rho }(\cdot )\) and \(x^{\epsilon \rho }(\cdot )\) be the solution of state function (9) under the control \(u^{\rho }(\cdot )\) and \(u^{\epsilon \rho }(\cdot )\). Following the variational equation (17), we introduce the following expression for \(q^{\epsilon \rho }(\cdot )\):
Here \(\varGamma ^{\rho }(t)\) and \(\varTheta ^{\rho }(t)\) are defined similarly to (14) with control variable \(u^{\rho }(\cdot )\), i.e.
According to Lemma 4.1 and Lemma 4.2, we have
Using the Taylor expansion we can check
By arguments analogous to the previous one,
with
and
Setting \(v(\cdot )=u^{\epsilon \rho }(\cdot )\) in (iii) of (36), we have
i.e.
By the formula of \(J_{\rho }\) we have
And
Then we can obtain
where
Obviously, \(\vert \alpha _{\rho }\vert ^{2}+\vert \gamma _{\rho }\vert ^{2}=1\). Therefore there exists a subsequence, still denoted by \((\alpha _{ \rho },\gamma _{\rho })\) such that \(\lim_{\rho \rightarrow 0}\alpha _{\rho }=\alpha \), \(\lim_{\rho \rightarrow 0}\gamma _{\rho }=\gamma \) in \(\mathbb{R}\) and \(\vert \alpha \vert ^{2}+\vert \gamma \vert ^{2}=1\). Next, we introduce the following adjoint equation:
Remark 4.6
\(b_{x_{\delta }}(\varGamma ^{\rho }(t),u^{\rho }(t),u^{\rho }(t-\delta )) \vert _{t+\delta }\) means that the value of derivative \(b_{x_{\delta }}\) at time \(t+\delta \) and other similar symbols have the same meanings. We can find that the adjoint equation is a MABSDE with a unique solution \((p^{\rho }(\cdot ),k^{\rho }(\cdot ))\) by Theorem 2.2. There is a dual relationship between (38) and (49). We should note that the derivatives of ψ, ϕ are in different positions in Eqs. (38) and (49).
Applying Ito’s formula to \(q^{\epsilon \rho }(t)p^{\rho }(t)\), we have
Let us assume without loss of generality that \(L_{x_{\delta }}(t)=L _{\mu _{\delta }}(t)=0\) on \([T,T+\delta ]\). Then Eq. (47) can be written as
By a change of variables, we can obtain
Considering the arbitrariness of \(\tau \in [0,T)\), dividing (52) by ϵ and taking \(\epsilon \rightarrow 0^{+}\), it follows that
Now let us take \(\rho \rightarrow 0^{+}\). Then \(\alpha ^{\rho }\rightarrow \alpha \), \(\gamma ^{\rho }\rightarrow \gamma \), and \(\vert \alpha \vert ^{2}+ \vert \gamma \vert ^{2}=1\). Simultaneously, from (36)(ii), we have \(d(u^{\rho }(\cdot ),u(\cdot ))\rightarrow 0\) when \(\rho \rightarrow 0^{+}\). Combining the continuous dependence of x on the control variables in Lemma 3.2, one can obtain
Let \((p(\cdot ),k(\cdot ))\) be the solution of the following equation:
We can check the convergence of \((p^{\rho }(\cdot ),k^{\rho }(\cdot ))\) by Theorem 2.2, i.e.,
where \((p^{\rho }(\cdot ),k^{\rho }(\cdot ))\) is the solution of the adjoint equation (49).
Consequently, letting \(\rho \rightarrow 0^{+}\) in (53), we have
We also need to drop the expectation in (57). In fact, (57) holds for all \(v(\cdot )\in \mathcal{U}\) since \(v(\cdot )\) is arbitrary. For any \(v\in U\) and \(A\in \mathcal{F}_{t}\), we define \(v(t)=vI_{A}+u(t)I_{\bar{A}}\). Then \(v(\cdot )\in \mathcal{U}\) and taking this \(v(\cdot )\) into (57), we have
Since \(A\in \mathcal{F}_{t}\) is chosen arbitrarily, it implies that
This will lead to
Let us introduce the following Hamiltonian:
Set
We have \(\mathcal{H}(t,v)\geq \mathcal{H}(t,u(t))\), \(\forall v\in U\), a.e. \(t\in [0,T]\), P-a.s.
In summary of the above analysis, we can get the following maximum principle.
Theorem 4.7
Let \((u(\cdot ),x(\cdot ))\) be an optimal solution of the Problem 3.1 and Assumption (H2) hold. Then there exist \(\alpha ,\gamma \in \mathbb{R}\) with \(\vert \alpha \vert ^{2}+\vert \gamma \vert ^{2}=1\) such that
Remark 4.8
When \(G(x)=0\), the state constraint will disappear and the results in our paper will degenerate to the case without state constraint.
5 Application
To conclude this paper, we apply our maximum principle to study a kind of optimal harvesting problem for a mean-field system. The model of our problem comes from Hu, Øksendal and Sulem [17]. We modify the model to be a delayed system with continuous harvesting as follows:
Here \(x(t)\) is the density of an unharvested population at time t and \(v(t)\) is the harvesting effort and \(\lambda _{1},\lambda _{2}>0\) are the given harvesting efficiency coefficients.
The performance functional is assumed to be of the form
where \(K=K(\omega )>0\) is \(\mathcal{F}_{T}\)-measurable, representing the salvage price. Our problem is to minimize above \(J(v(\cdot ))\) under the constraint (11). For simplicity, we assume all the coefficients \(b_{i}(t)\) (\(i=1,2,3\)), \(\sigma _{i}(t)\) (\(i=1,2\)) and \(L_{i}(t)>0\) (\(i=1,2,3\)) are deterministic functions on \([0,T]\).
Proposition 5.1
Assume \(\alpha >0\), \(\gamma >0\) and the constraint function \(G(x)\) is convex. Then the optimal control of the mean-field harvesting problem with delay is
where \(p_{0}(t)\) is solution of the following adjoint equation:
Proof
Applying the necessary condition (61) in Theorem 4.7, the control of the form (64) is a candidate of the optimal controls. We also need to prove the optimality of \(u(t)\). Suppose \(x(t)\) is the trajectory of \(u(t)\) and \(v(t)\in \mathcal{U}_{\mathrm{ad}}\) with \(x^{v}(t)\) being its corresponding trajectory. Let us denote \(\bar{x}(t)=x^{v}(t)-x(t)\). Applying Itô’s formula to \(\bar{x}(t)p_{0}(t)\) we have
For the mean-field optimal harvesting problem, we have
Using a change of time variables we conclude that
Thus, Eqs. (66) and (67) lead to
Furthermore, we can get
since we have the constraint condition \(\mathbb{E}[G(x(T))]= \mathbb{E}[G(x^{v}(T))]=0\) and G is convex. The proof of the optimality is completed. □
Remark 5.2
An example of the constraint function is \(G(x)=((N-x)^{+})^{2}\) with a fixed constant N. And this state constraint condition implies that \(x(T)\geq N\) a.s.
References
Kac, M.: Foundations of kinetic theory, Berkeley symposium on mathematical statistics and probability. In: Third Berkeley Symposium on Mathematical Statistics and Probability, pp. 101–110 (1956)
Mckean, H.P.: A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl. Acad. Sci. USA 56(6), 1907–1911 (1966)
Cardaliaguet, P.: Notes on mean field games. Technical report, (2010)
Buckdahn, R., Li, J., Ma, J.: A stochastic maximum principle for general mean-field systems. Appl. Math. Optim. 74(3), 507–534 (2016)
Agram, N., Øksendal, B.: Mean-field stochastic control with elephant memory in finite and infinite time horizon. arXiv preprint (2018). arXiv:1804.09918
Agram, N., Øksendal, B.: Stochastic control of memory mean-field processes. Appl. Math. Optim. 2, 1–24 (2017)
Buckdahn, R., Djehiche, B., Li, J.: A general stochastic maximum principle for sdes of mean-field type. Appl. Math. Optim. 64(2), 197–216 (2011)
Buckdahn, R., Li, J., Peng, S.: Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Process. Appl. 119(10), 3133–3154 (2009)
Carmona, R., Delarue, F., et al.: Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann. Probab. 43(5), 2647–2700 (2015)
Lasry, J.M., Lions, P.L.: Mean field games. Jpn. J. Math. 2(1), 229–260 (2007)
Carmona, R., Delarue, F.: Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51(4), 2705–2734 (2013)
Li, J.: Stochastic maximum principle in the mean-field controls. Automatica 48(2), 366–373 (2012)
Yong, J.: Linear–quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51(4), 2809–2838 (2013)
Huang, J., Li, X., Yong, J.: A linear–quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control Relat. Fields 5(1), 97–139 (2015)
Li, X., Sun, J., Yong, J.: Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probab. Uncertain. Quant. Risk 1(1), 1–24 (2016)
Andersson, D., Djehiche, B.: A maximum principle for sdes of mean-field type. Appl. Math. Lett. 63(3), 341–356 (2011)
Hu, Y., Øksendal, B., Sulem, A.: Singular mean-field control games. Stoch. Anal. Appl. 35(5), 1–29 (2017)
Menaldi, J.M., Rofman, E.: In: Sulem, A., Øksendal, B., Sulem, A. (eds.) Optimal Control and Partial Differential Equations—Innovations and Applications (2000)
Chen, L., Wu, Z.: Maximum principle for stochastic optimal control problem of forward–backward system with delay. In: IEEE Conference on Decision & Control, pp. 16–18 (2009). IEEE
Chen, L., Wu, Z.: Maximum principle for the stochastic optimal control problem with delay and application. Automatica 46(6), 1074–1080 (2009)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28(4), 966–979 (1990)
Yu, Z.: The stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls. Automatica 48(10), 2420–2432 (2012)
Zhang, F.: Maximum principle for delayed stochastic linear–quadratic control problem with state constraint. J. Appl. Math. 2013, Article ID 964765 (2013)
Ying, H., Peng, S.: Maximum principle for semilinear stochastic evolution control systems. Stoch. Int. J. Probab. Stoch. Process. 33(3–4), 159–180 (1990)
Yong, J., Zhou, X.: Stochastic controls: Hamiltonian systems and hjb equations. In: Karatzas, I., Yor, M. (eds.) Applications of Mathematics, 1st edn., vol. 43, pp. 101–156. Springer, New York (1999)
Ekeland, I.: On the variational principle. J. Math. Anal. Appl. 47(2), 324–353 (1974)
Funding
This work is supported by National Natural Science Foundation of China (11301530).
Author information
Authors and Affiliations
Contributions
All authors have contributed equally in this paper. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Chen, L., Wang, J. Maximum principle for delayed stochastic mean-field control problem with state constraint. Adv Differ Equ 2019, 348 (2019). https://doi.org/10.1186/s13662-019-2283-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13662-019-2283-1