Stability and control in a stochastic model of malaria population dynamics

Abstract

This article proves a stability theorem for the disease-free equilibrium of a stochastic differential equations model of malaria disease dynamics. The theorem is formulated in terms of an invariant which is similar to the basic reproduction number of a related deterministic model. Compared to the deterministic model, stability of the disease-free equilibrium holds more generally for the stochastic model. The optimal control theory is applied to the stochastic model, revealing some important new insights. Theoretical results are illustrated by way of simulations.

1 Introduction

Approximately half of the world’s population is at risk of malaria. A considerable amount of scientific effort has been directed to the fight against malaria. This includes the construction, analysis and application of mathematical models. Some of these models capture the effects of climate variables on malaria transmission, e.g., [1, 3, 9]. Some authors have introduced randomness into ordinary differential equations (ODE) compartmental models to reflect a multitude of uncertainties. Such stochastic differential equations (SDE) models have already been proposed for various diseases, and in particular for vector-borne diseases [8, 12, 18, 20].

We propose an SDE model for the population dynamics of a disease such as malaria or dengue fever. The underlying deterministic model of the present paper is the same as that in [18, 20] and is also similar to that of [12]. However, the stochastic perturbation of the present paper differs from those in both [18] and [12]. Also, the methodologies are quite different, both on the mathematical analysis and the simulation sides. The new model allows for an elimination theorem, that is, a stability theorem for the disease-free equilibrium, which takes a form comparable to its counterpart in the underlying deterministic model. The elimination theorem takes a simpler and more explicit form than the stability theorem of [20]. Ultimately, the stochastic perturbation in this new model permits stability of the disease-free equilibrium beyond the limitations of the underlying deterministic model, similar, for example, to [5, 19, 21].

We also perform optimal control analysis. Ishikawa, see [7], seems to be the first author to have worked on optimal control in SDE models of infectious disease dynamics. Other papers in this regard are [2, 11, 13, 22] etc. There are numerous papers in the literature on control in deterministic models of malaria dynamics, such as [3, 6, 10, 15]. In the current paper, we solve a stochastic control problem. Our control functions are linked to the use of bednets, insecticides, and isolation of infected humans. Approximate numerical solutions of the control problem are derived following a method similar to [22].

The remainder of this paper is organized as follows. Section 2 describes the malaria model. In Sect. 3 we show the existence and uniqueness of a global positive solution of the model. Section 4 presents an investigation of the asymptotic behavior of solutions to the stochastic model around the trivial equilibrium point. In Sect. 5, we run simulations to make future projections of the state of disease in the population and to illustrate the long-term behavior of the model. Finally, in Sect. 6, we provide a few concluding remarks and suggestions for future research.

2 The stochastic model of malaria

Stochasticity is introduced as follows. We assume having a complete filtered probability space $(\Omega , \mathcal{F}, \{\mathcal {F}_{t} \}_{t\geq 0}, \mathbb{P})$ with the filtration $\{\mathcal{F}_{t}\}_{t\ge 0} \subset \mathcal{F}$ satisfying the usual conditions (i.e., the filtration is right-continuous and $\mathcal{F}_{0}$ contains all the subsets having measure zero). As part of the fixed notation throughout the paper, we shall consider a pair of independent Wiener processes $B_{1}(t)$ and $B_{2}(t)$ on this probability space. We use the notation: for $n\in {\mathbb{N}}$, $\mathbb{R}_{+}^{n}=\{x \in \mathbb{R}^{n}: x_{i} >0 \text{ for each } i \}$.

We consider a stochastic malaria model based on the deterministic model in [16]. As in [16], the host population at time $t\ge 0$ is of size $N(t)$ and is subdivided into three compartments, and the vector disease population, of size $M(t)$, is subdivided into two compartments. We ambiguously use the same symbol for a population class and for its magnitude. The first compartment in the human population consists of those individuals who are uninfected, but susceptible to infection with the pathogen in point. We denote this class by $S(t)$. The second class, denoted by $I(t)$, consists of all the individuals who are infected with the pathogen. The third class consists of all the human individuals who recovered from the infection and have temporary immunity against the pathogen. This class of individuals is denoted by $R(t)$. The vector compartment is subdivided into two classes: susceptible V class and infected J class.

The human birth rate is denoted by A. There is no vertical transmission and all the newborns are susceptible. The per capita death rate (excluding death due to malaria) is assumed to be the same constant μ for all humans, and the rate of mortalities due to malaria is denoted by δ. The mosquito population has B and θ as the natural birth rate and per capita mortality rate, respectively. We assume that a susceptible individual bitten by an infected mosquito becomes infectious once successfully infected where α and β are the transmission probabilities from humans to mosquitoes and from mosquitoes to humans, respectively. The rate of infection of the human host in class S by infected vectors in J is dependent on the total number of humans available per infected vector. The per capita rate of transfer from the I-class to the R-class is k. The people who are immune lose their immunity at a (per capita) rate γ. The above description is depicted in the flow diagram in Fig. 1.

Figure 1

Flow diagram of the deterministic version of the model

After introducing stochastic perturbations, with $v_{1}$ and $v_{2}$ being non-negative constants, we obtain the following system of SDEs.

$$\begin{aligned}& dS(t) = \bigl[ A-\beta S(t)J(t)-\mu S(t)+\gamma R(t) \bigr]\,dt , \\& dI(t) = \bigl[\beta S(t)J(t) -\mu _{1} I(t) \bigr]\,dt + v_{1} I(t)\,dB _{1}(t), \\& dR(t) = \bigl[ kI(t) - (\mu +\gamma )R(t) \bigr]\,dt , \\& dV(t) = \bigl[ B-\alpha V(t)I(t)-\theta V(t) \bigr]\,dt , \\& dJ(t) = \bigl[ \alpha V(t)I(t)- \theta J(t) \bigr]\,dt +v_{2} J(t) \,dB _{2}(t), \end{aligned}$$

(1)

where $\mu _{1}=\mu +k +\delta$.

Stochasticity in real world phenomena is always prevalent. In modeling with ordinary differential equations, one can address what is arguably the most important effect of stochasticity. In some disease models, the authors consider the mortality rates as being affected most significantly, see [12, 13] for instance. In other models, such as [5, 19], it is the transmission rate of the disease that is regarded as most important with regard to random variations. In the current work, we consider the most serious source of the randomness to be the disease itself. Hence we introduce the perturbations as being linked to the infected classes of both the host population and the vector population, which leads to the SDE system above.

A solution of this system over a time interval D is a set of points

$$ X(t)=\bigl(S(t),I(t),R(t),V(t), J(t)\bigr),\quad t\in D. $$

The stochastic model has a unique equilibrium point $X_{*}$, the disease-free equilibrium,

$$ X_{*} = \biggl( \frac{A}{\mu}, 0,0, \frac{B}{\theta}, 0 \biggr) . $$

In the special case for which $v_{1}=0=v_{2}$, we refer to the system (1) as the underlying deterministic model.

Existence and uniqueness of positive solutions are covered by [18]. More precisely, we deduce the following results from [18].

Proposition 2.1

Consider a number $t_{1}>0$. Suppose that for every $0 < t < t_{1}$, we have S, I, R, V, J being positive. Then

• (a) If $S(0) + R(0)< \frac {A}{\mu}$, then $S(t) + R(t) < \frac {A}{\mu}$ for all $0< t\le t_{1}$.

• (b) If $V(0) < \frac {B}{\theta}$, then $V(t) < \frac {B}{\mu}$ for all $0< t\le t_{1}$.

Theorem 2.2

For any given initial value $X(0)\in {\mathbb{R}}^{5}_{+}$, there exist a unique positive solution $X(t)$ of (1) on $t\geq 0$ such that the solution remains in $\mathbb{R}_{+}^{5}$ with probability 1, namely $X(t) \in \mathbb{R}_{+}^{5}$ for all $t\geq 0$ almost surely.

3 Stability of the disease-free equilibrium

The underlying deterministic model is the same as in the model [16]. The basic reproduction number, ${\mathcal{R}}_{0}$, of the underlying deterministic model is an indicator of local asymptotic stability of the disease-free equilibrium. It was calculated in [16] as

$$ {\mathcal{R}}_{0} = \sqrt{\frac{\alpha \beta A B}{\theta ^{2}\mu \mu _{1}}}. $$

As an indicator of stability of the disease-free equilibrium of the stochastic model (1), we introduce an analogous invariant, which we denote by $\mathfrak{R}$, which is of a form similar to ${\mathcal{R}}_{0}$, and as the intensities of the perturbations tend to zero, then $\mathfrak{R}$ tends to ${\mathcal{R}}_{0}$. In Item 3.1 below, we briefly introduce a constant that appears in the formulation of $\mathfrak{R}$. Similar techniques appear in [17, 19], and [21].

Item 3.1

A function $h(x)$.

Let $v_{1}$ and $v_{2}$ be the perturbation intensities of model (1). We introduce a function h

$$ h: (0,1] \to {\mathbb{R}} ; \qquad x \mapsto \frac{1}{2x} \bigl[v_{1}^{2}x^{2}+v_{2}^{2}(1-x)^{2} \bigr]. $$

Then h is continuous. If $v_{1}>0$ or $v_{2}>0$, then $h(x) >0$ for all $x\in (0,1]$. If $v_{1}>0$, then:

$$ \lim_{x\rightarrow 0^{+}}h(x) = +\infty . $$

Thus h has an absolute minimum $h^{*}$ and $h^{*} >0$. We find that indeed h has such minimum value, at $x=x^{*}$ with

$$ x^{*}= \sqrt{\frac{v_{2}^{2}}{v_{1}^{2}+v_{2}^{2}}}, $$

and then we write: $h(x^{*})=h^{*}$.

The elimination theorem is formulated in terms of the following indicator:

$$ \mathfrak{R} = \sqrt{ \frac{\alpha \beta A B}{\theta ^{2}\mu (\mu _{1}+h^{*})}}. $$

(2)

For stochastic systems there are many different versions of the concept of stability and very sophisticated methods of stability analysis, see [14]. In the current paper we focus on almost sure exponential stability, which is also studied in [19].

Remark 3.2

Let us fix a positive constant q. We define the following stochastic processes $\{u(t)\}_{t\geq 0}$ and $\{z(t)\}_{t\geq 0}$:

$$ u(t) = I(t) + qJ(t) . $$

By Theorem 2.2, the following statement holds:

$$ \text{(a.s.)}, \quad u(t) >0 \quad \text{for all } t>0. $$

Thus, if $u(t)>0$ for all $t>0$, then we define

$$ z(t)=\ln u(t). $$

Following a similar methodology as in [19] to prove stability of the disease-free equilibrium, we note the following.

By the Itô formula, the stochastic process $\{z(X(t))\}_{t\geq 0}$ can be presented as:

$$ z\bigl(X(t)\bigr) = z\bigl(X(0)\bigr) + \int _{w=0}^{t}{\mathcal{L}}z\bigl(X(w)\bigr)\,dw + M_{1}(t)+M_{2}(t), $$

where $M_{1}(t)$ and $M_{1}(t)$ are the Itô integrals

$$ M_{1}(t)= \int _{0}^{t} \frac{1}{u(w)}{v_{1}I(w)} \,dB _{1}(w) \quad \text{and} \quad M_{2}(t)= \int _{0}^{t} \frac{q}{u(w)}{v_{2}J(w)} \,dB _{2}(w). $$

Since $|I(w)/u(w)|\leq 1$ and $|qJ(w)/u(w)|\leq 1$ (bounded), by application of the strong law of large numbers for local martingales, we can deduce that

$$ \lim_{t\to \infty}\frac{1}{t}M_{1}(t)=0 \quad\text{(a.s.)}\quad \text{and}\quad \lim_{t\to \infty} \frac{1}{t}M_{2}(t)=0. $$

Therefore it follows that

$$ \limsup_{t\to \infty}\frac{1}{t} z\bigl(X(t)\bigr)=\limsup _{t\to \infty} \frac{1}{t} \int _{0}^{t}{\mathcal{L}}z\bigl(X(w)\bigr)\,dw \quad\text{(a.s.)}. $$

Remark 3.3

From the above, in order to prove that the stochastic process $\{u(t)\}_{t\geq 0}$ converges exponentially to 0 (a.s.), it suffices to prove that

$$ \limsup_{t\to \infty}\frac{1}{t} \int _{0}^{t}{\mathcal{L}}z\bigl(X(w)\bigr)\,dw < 0 \quad \text{(a.s.)}. $$

We can calculate $\mathcal{L}z(t)$ as

$$ \mathcal{L}z(t) = \frac {1}{u(t)} \bigl[\beta S(t)J(t) -\mu _{1} I(t) \bigr] + \frac {q}{u(t)} \bigl[\alpha V(t)I(t)- \theta J(t) \bigr] - E(t), $$

where $E(t)$ is given by

$$ E(t) = \frac {1}{2u^{2}(t)} \bigl(\bigl(v_{1}I(t) \bigr)^{2}+\bigl(qv_{2}J(t)\bigr)^{2} \bigr). $$

Note that for every $t>0$, $I(t)/u(t) + qJ(t)/u(t) =1$ and thus, $qJ(t)/u(t) =1-I(t)/u(t)$. Therefore we can write

$$\begin{aligned} E(t) =& \frac {1}{2u^{2}(t)} \bigl(\bigl(v_{1}I(t) \bigr)^{2}+\bigl(qv_{2}J(t)\bigr)^{2} \bigr) \\ =& \frac{1}{2} \biggl[(v_{1})^{2} \biggl(\frac {I(t)}{u(t)} \biggr)^{2} + (v_{2})^{2} \biggl(q\frac {J(t)}{u(t)} \biggr)^{2} \biggr] \\ =& \frac{1}{2} \biggl[(v_{1})^{2} \biggl(\frac {I(t)}{u(t)} \biggr)^{2} + (v_{2})^{2} \biggl(1 - \frac {I(t)}{u(t)} \biggr)^{2} \biggr] \\ =& \frac {I(t)}{u(t)} h \biggl(\frac {I(t)}{u(t)} \biggr). \end{aligned}$$

(3)

In particular then

$$ E(t)\geq h^{*}\frac{I(t)}{u(t)}, \quad\text{i.e.,} \quad {-}E(t)\leq -h^{*} \frac{I(t)}{u(t)}. $$

Now we obtain the inequality:

$$\begin{aligned} \mathcal{L}z\bigl(X(t)\bigr) =& \frac {I(t)}{u(t)} \bigl[q\alpha V(t) -\mu _{1} \bigr] +\frac {J(t)}{u(t)} \bigl[\beta S(t)-q\theta \bigr]-E(t) \\ \leq & \frac {I(t)}{u(t)} \biggl[\frac{q\alpha B}{\theta} -\mu _{1} \biggr] +\frac {J(t)}{u(t)} \biggl[\frac{\beta A}{\mu}-q \theta \biggr] - h^{*} \frac{I(t)}{u(t)} \\ =& \frac {I(t)}{u(t)} \biggl[\frac{q\alpha B}{\theta} -\bigl(\mu _{1}+h^{*}\bigr) \biggr] +\frac {J(t)}{u(t)} \biggl[ \frac{\beta A}{\mu}-q\theta \biggr]. \end{aligned}$$

We are interested in $\limsup_{t\to \infty} z(t)$ and we introduce the necessary notation for this analysis. For a stochastic process $\{x(t)\}_{t\geq 0}$, we write

$$ \langle x\rangle _{t}=\frac{1}{t} \int _{0}^{t} x(s) \,ds . $$

We note that for every $w\in \Omega$ there exists an unbounded increasing sequence of positive numbers $(t_{n})$ with the property that $\lim_{n\to \infty}\mathcal{L}z(t_{n}) = \limsup_{t\to \infty} \mathcal{L} z(t)$, and such that the following sequences are convergent:

$$ \langle I/u \rangle _{t_{n}} , \qquad \langle J/u \rangle _{t_{n}}. $$

The latter two limits will be denoted by i, j respectively, and we write

$$ \Gamma =\limsup_{t\to \infty} \langle \mathcal{L}z\rangle _{t}. $$

We note that these values above depend on q. Now $\limsup_{t\to \infty} \langle {\mathcal {L}}z \rangle _{t}$ can be seen to satisfy the inequality

$$ \Gamma \leq i \biggl[\frac{q\alpha B}{\theta} -\bigl(\mu _{1}+h^{*} \bigr) \biggr] +j \biggl[\frac{\beta A}{\mu}-q\theta \biggr] . $$

(4)

Theorem 3.4

If $\mathfrak{R} < 1$, then $I(t)$ and $J(t)$ converges exponentially to 0 (a.s.).

Proof

Let us assume that $\mathfrak{R} < 1$. The condition $\mathfrak{R} < 1$ is equivalent to the inequality

$$ \frac {\beta A}{\mu \theta} \biggl(\frac {\alpha B}{\theta} \biggr) -\bigl(\mu _{1}+h^{*}\bigr) < 0. $$

We can find a number ϵ such that $0<\epsilon <1$ and

$$ \biggl(\frac {\beta A}{\mu \theta} +\epsilon \biggr) \biggl( \frac {\alpha B}{\theta} \biggr) -\bigl(\mu _{1}+h^{*}\bigr) < 0. $$

(5)

Now let q be given by

$$ q=\frac {\beta A}{\mu \theta} +\epsilon . $$

For the given value of q, we now consider $u(t)$ and $z(t)$. To prove our theorem, it suffices to prove that $u(t)$ converges exponentially to zero (a.s.). Thus by Remark 3.3, it is sufficient to prove that $\Gamma <0$. We write the inequality (4) as

$$ \Gamma \leq Q_{1}i +Q_{2}j, $$

with

$$ Q_{1}=\frac{q\alpha B}{\theta} -\bigl(\mu _{1}+h^{*}\bigr)=\bigl(\mu _{1}+h^{*}\bigr)[ \mathfrak{R}-1]< 0, $$

and

$$ Q_{2}= \frac{\beta A}{\mu}-q\theta = - \epsilon \theta < 0. $$

The coefficients of i and j on the right-hand side of the inequality are negative and constant. Note that

$$ i+qj = 1. $$

Therefore at least one of the quantities i or j must be non-zero. Thus,

$$ \limsup_{t\to \infty} < {\mathcal{L}}z >_{t} < 0 \quad \text{(a.s.)} $$

and the proof is complete. □

Remark 3.5

By Theorem 3.4, we have that if $\mathfrak{R} < 1$, then starting from any initial value, $I(t)$ and $J(t)$ converge exponentially to 0. Therefore the theorem shows that for parameter values with ${\mathcal{R}}_{0}$ slightly bigger than 1, stability of the disease-free equilibrium (i.e., elimination of the disease) is enhanced by the stochastic perturbations.

The proof of the following theorem, that supplements Theorem 3.4, goes along similar arguments as the proof of [21, Theorem 4.4(b)].

Theorem 3.6

If $I(t)$ and $J(t)$ converges to 0 (a.s.), then $\lim_{t\to \infty} X(t) = X_{*}$ almost surely.

Proof

Let $\Omega _{1}$ be the subset of all paths ω in Ω for which both $I(t,\omega )$ and $J(t,\omega )$ converge to 0. In the rest of the proof, we assume working along some path in $\Omega _{1}$, which is arbitrarily chosen, but we suppress the path for simplicity of notation.

• We prove by contradiction that $R(t)$ converges to 0 (a.s.). Suppose that for a given sample path we have:

$$\begin{aligned} r:=\limsup_{t\to \infty}R(t)>0. \end{aligned}$$

(6)

Since $I(t)$ converges to 0, there exists $t_{0}>0$ such that whenever $t\geq t_{0}$, then

$$ I(t) < \frac{r(\mu +\gamma )}{2k} \quad\text{(a.s.)}. $$

Consider any $t_{1}>t_{0}$. If $R(t_{1})\geq r/2$, then

$$ R'(t_{1}) = kI(t_{1})-(\mu +\gamma )R(t_{1})\leq k \biggl( \frac{r(\mu +\gamma )}{2k} \biggr)-(\mu +\gamma ) \frac{r}{2}=0. $$

In particular, if for some $t_{1}>t_{0}$ we have $R(t_{1}) \geq r/2$, then $R(t)\leq R(t_{1})$ for all $t>t_{1}$. Then it follows that in order that $\limsup_{t\to \infty}R(t)=r$, we must have that $R(t) \geq r$ for all $t>t_{0}$. This means (with $I(t)$ converging to 0) that eventually $R'(t)\leq 0$. Thus, over some interval $[t_{2}, \infty )$ the function R is monotone. Then the bounded function R is also convergent (to r), and also $R'(t)$ converges to 0. Since $R'(t)$ converges to 0, it follows that $r=0$, and this is a contradiction. This completes the proof that $R(t)$ converges to 0 (a.s.).

• We now prove by contradiction that $S(t)$ converges to $A/\mu$ (a.s.). Suppose to the contrary that (for a given sample path in $\Omega _{1}$) we have:

$$\begin{aligned} z_{0}:=\liminf_{t\to \infty}S(t)< A/\mu . \end{aligned}$$

(7)

Now consider any $z_{1} \in (z_{0}, A/\mu )$. In particular then, $A-\mu z_{1}>0$. Since both $J(t)$ and $R(t)$ are convergent to 0, and $S(t)$ is bounded, there exists $t_{0}>0$ such that whenever $t\geq t_{0}$, then

$$ \beta S(t)J(t)-\gamma R(t) < A-\mu z_{1}. $$

Consider any $t_{1}>t_{0}$. If $S(t_{1})\leq z_{1}$, then

$$ S'(t_{1}) = A-\mu S(t_{1})-\bigl(\beta S(t_{1})J(t_{1})-\gamma R(t)\bigr)> A- \mu z_{1}-(A-\mu z_{1})=0. $$

In particular, if for some $t_{1}>t_{0}$ we have $S(t_{1}) > z_{1}$, then $S(t)\geq z_{1}$ for all $t>t_{1}$. Since $z_{1}$ can be chosen arbitrarily close to $z_{0}$, we can in fact deduce that $S(t)$ actually converges to $z_{0}$. Feeding this limit into the expression for $S'(t)$ we find that eventually $S'(t)$ is consistently positive. With $S'(t)$ bounded above, it means that $\lim S'(t)=0$. This means that $r=A/\mu$, which is a contradiction. Thus we have proved that $\lim S'(t)=0$.

• Similarly as we proved convergence of $S(t)$, we can prove that $V(t)$ converges to $B/\theta$ (a.s.). We omit the detail. □

4 The stochastic optimal control problem

In this section we formulate the stochastic optimization problem and describe its solution. Control functions $u(t)=(u_{1}(t), u_{2}(t), u_{3}(t))$ are introduced into the model (1). For fixed constants $w_{1}$, $w_{2}$, and $w_{3}$, we assume that

$$ 0< u_{i}< w_{i}\quad \text{for each }i\in \{ 1,2,3\}. $$

The control functions are: $u_{1}$ which represents the extent to which bednets are used, $u_{2}$ indicates the isolation of infected human beings, and $u_{3}$ measures the increase in the mortality rate of vectors due to indoor residual spraying (IRS). We assume that the death due to IRS affects mostly the infected vectors, while the susceptible vectors are essentially unaffected. The resulting set of equations takes the form:

$$\begin{aligned}& dS(t) = \bigl[ A-\beta \bigl(1-u_{1}(t)\bigr)S(t)J(t)-\mu S(t)+ \gamma R(t) \bigr]\,dt , \\& dI(t) = \bigl[\beta \bigl(1-u_{1}(t)\bigr)S(t)J(t) -\mu _{1} I(t) \bigr]\,dt + v_{1} I(t)\,dB _{1}(t), \\& dR(t) = \bigl[ kI(t) - (\mu +\gamma )R(t) \bigr]\,dt , \\& dV(t) = \bigl[ B-\alpha V(t)I(t) \bigl(1-u_{2}(t)\bigr)-\theta V(t) \bigr]\,dt , \\& dJ(t) = \bigl[ \alpha V(t)I(t) \bigl(1-u_{2}(t)\bigr)- \theta \bigl(1+u_{3}(t)\bigr) J(t) \bigr]\,dt +v_{2} J(t)\,dB _{2}(t), \end{aligned}$$

(8)

where $\mu _{1}=\mu +k +\delta$.

Control Problem 4.1

Fix the number $T>0$. For $t\in [0,T]$ and with $z_{t}\in {\mathbb{R}}^{5}$ being a possible state of the system at time t, we write

$$ J(t,z_{t};u)=\mathbb{E}_{t,z_{t}} \biggl[ \int _{0}^{T} \bigl(c_{1}u_{1}^{2}(s)+ c_{2}u_{2}^{2}(s)+c_{3}u_{3}^{2}(s)+c_{4} I(s) \bigr) \,ds \biggr]. $$

Here the expectation is on the condition that at time t, the state of the system is $z_{t}$. Our objective is to find a control strategy $u^{*}(t)$ which minimizes the expected value of the objective functional (for fixed $z_{0}$),

$$ J(0,z_{0};u). $$

The class of admissible control laws is

$$ \mathcal{A} = \bigl\{ u(\cdot ): u \ge 0,\ u \text{ is bounded and adapted, and such that } u \geq 0 \text{ a.s.} \bigr\} . $$

(9)

We define the value function as

$$ U(t,x) = \inf_{u(\cdot )\in \mathcal{A}}J(t,x; u) = J\bigl(t,x; u^{*}\bigr). $$

A solution to the optimal control problem stated in Problem 4.1 is obtained via the dynamic programming approach. Let us write the system (7) as

$$\begin{aligned}& dS(t) = f_{1}(X) \,dt , \\& dI(t) = f_{2}(X)\,dt + v_{1} I(t)\,dB _{1}(t), \\& dR(t) = f_{3}(X)\,dt , \\& dV(t) = f_{4}(X)\,dt , \\& dJ(t) = f_{5}(X)\,dt +v_{2} J(t)\,dB _{2}(t). \end{aligned}$$

(10)

We write down $\mathcal{L}U(t)$:

$$\begin{aligned} \mathcal{L}U(t) =&f_{1}(t)U_{S}(t)+f_{2}(t)U_{I}(t)+f_{3}(t)U_{R}(t)+f_{4}(t)U_{V}(t)+f_{3}(t)U_{J}(t) \\ &{} +\frac{1}{2}\bigl[\bigl(v_{1}I(t)\bigr)^{2}U_{II}(t)+2v_{1}v_{2}U_{IJ}(t)+(v_{2}J)^{2}U_{JJ}(t) \bigr]. \end{aligned}$$

Then by applying the Hamilton–Jacobi–Bellmann theory (see, for instance, [4]) we must minimize:

$$ \inf_{u \in \mathcal{A}} \bigl[c_{1}u_{1}^{2}+ c_{2}u_{2}^{2}+c_{3}u_{3}^{2}+c_{4} I(s) + \mathcal{L}U(t) \bigr]. $$

This entails partial differentiation, with respect to the control variables, of the given expression. If we make these derivatives zero, we obtain the following equations.

$$\begin{aligned}& 2c_{1} u_{1}(t) +\beta S(t)J(t)\bigl[U_{S}(t) + U_{I}(t)\bigr] = 0, \\& 2c_{2} u_{2}(t) +\alpha V(t)I(t)\bigl[U_{V}(t) -U_{J}(t)\bigr] = 0, \\& 2c_{3} u_{3}(t) - \theta J(t)U_{J}(t) = 0. \end{aligned}$$

(11)

This leads to an optimal control of the form

$$\begin{aligned}& u_{1}^{*}(t) = \max \biggl\{ 0, \min \biggl[w_{1}, \frac{1}{2c_{1}} \beta S(t)J(t)\bigl[U_{I}(t) - U_{S}(t)\bigr]\biggr]\biggr\} , \\& u_{2}^{*}(t) = \max \biggl\{ 0, \min \biggl[w_{2}, \frac{1}{2c_{2}} \alpha V(t)I(t)\bigl[U_{J}(t) -U_{V}(t)\bigr]\biggr]\biggr\} , \\& u_{3}^{*}(t)= \max \biggl\{ 0, \min \biggl[w_{3}, \frac{1}{2c_{3}}\theta J(t)U_{J}(t) \biggr] \biggr\} . \end{aligned}$$

Numerical solution of the optimal system is rather complex. We opt to follow the same approximation method as was done in [2, 13, 22]. For this purpose we require the solution of the deterministic optimal control problem, which emerges as a special case of Problem 4.1. For the deterministic optimal control problem, the co-state variables satisfy the following system of ODEs:

$$\begin{aligned}& \lambda _{1}'(t) = \bigl(\beta \bigl(1-u_{1}(t)\bigr)J(t)+\mu \bigr)\lambda _{1}(t)- \beta \bigl(1-u_{1}(t)\bigr)J(t)\lambda _{2}(t), \\& \lambda _{2}'(t) = -c4 +\mu _{1}\lambda _{2}(t)-k\lambda _{3}(t)+ \alpha V(t) \bigl(1-u_{1}(t)\bigr)\lambda _{4}(t)-\alpha V(t) \bigl(1-u_{1}(t)\bigr) \lambda _{5}(t), \\& \lambda _{3}'(t) = \gamma \lambda _{1}(t)+(\mu +\gamma )\lambda _{3}(t), \\& \lambda _{4}'(t) = \bigl(\alpha I(t) \bigl(1-u_{2}(t)\bigr)+\theta \bigr)\lambda _{4}(t)- \alpha I(t) \bigl(1-u_{2}(t)\bigr)\lambda _{5}(t), \\& \lambda _{5}'(t) = \beta \bigl(1-u_{1}(t) \bigr)S(t)\lambda _{1}(t)-\beta \bigl(1-u_{1}(t) \bigr)S(t) \lambda _{2}(t)+\theta \bigl(1+u_{3}(t)\bigr) \lambda _{5}(t). \end{aligned}$$

The solution $u_{d}^{*}$ of the deterministic optimal control problem takes the form

$$\begin{aligned}& u_{d1}^{*}(t) = \max \biggl\{ 0, \min \biggl[w_{1}, \frac{1}{2c_{1}} \beta S(t)J(t)\bigl[ \lambda _{2}(t) - \lambda _{1}(t)\bigr]\biggr]\biggr\} , \\& u_{d2}^{*}(t) = \max \biggl\{ 0, \min \biggl[w_{2}, \frac{1}{2c_{2}} \alpha V(t)I(t)\bigl[ \lambda _{5}(t) - \lambda _{4}(t)\bigr]\biggr]\biggr\} , \\& u_{d3}^{*}(t) = \max \biggl\{ 0, \min \biggl[w_{3}, \frac{1}{2c_{3}}\theta J(t) \lambda _{5}(t)\biggr]\biggr\} , \end{aligned}$$

with $\lambda (t)$ being the co-state variable.

The heuristic argument is that for sufficiently small values of the perturbation parameters, we can approximate the first-order partial derivatives of U by the corresponding coordinate of λ. We numerically solve for λ using a fourth-order Runge–Kutta scheme.

5 Numerical simulations

We apply this model to malaria disease over a population, which we assume to have a population of 10⁵ when malaria-free. In the simulations, we use parameter values from Table 1 or minor modifications of it. These values are sourced from [20] and it applies to malaria disease. The values of the perturbation parameters $v_{1}$ and $v_{2}$ are specified at each graph. We work with the following nominally chosen initial values:

$$\begin{aligned}& S(0)= 84{,}000 , \qquad I(0)= 4000,\qquad R(0)= 12{,}000, \\& V(0)= 250{,}000 ,\qquad J(0)= 50{,}000 . \end{aligned}$$

Table 1 Numerical values of parameters

5.1 Long-term behavior

To illustrate the stability theorem, we make a change in the values of α and β, choosing $\alpha =6.375\times 10^{-7}$ and $\beta =1.275\times 10^{-8}$, and then we calculate the value of the basic reproduction number as ${\mathcal{R}}_{0}=1.022$, while $\mathfrak{R}=0.9276<1$ for $v_{1}=v_{2}=0.05$. The deterministic model will have the disease-free equilibrium being unstable. Theorem 3.6, on the other hand, guarantees the almost sure stability of the disease-free equilibrium of the stochastic model. The graph in Fig. 2 seems to confirm the finding of Theorem 3.6. In Fig. 2 we have the comparison of the mean I-path of 1000 sample paths of the stochastic model compared with the deterministic I-path. Figure 3 shows the trajectories of the same variables, but they are run with different initial conditions and a longer time horizon.

Figure 2

The mean I-path of 1000 sample paths of the stochastic model compared with the deterministic I-path, [$v_{1}=v_{2}=0.05$]

Figure 3

The mean I-path of 1000 sample paths of the stochastic model compared with the deterministic I-path, [$v_{1}=v_{2}=0.05$]

5.2 Optimal control

We present some simulations on stochastic optimal controls. Approximate solutions are obtained in a similar way as in [22] and subsequently in [2, 13]. This amounts to appropriate utilization of formulas that solve the deterministic control problem are used to obtain approximate solutions for the stochastic control problem.

In Figs. 4–7, we show these stochastic optimal controls corresponding to a single path $\omega _{0}$ of the process $\{B_{1},B_{2}\}_{t\geq 1}$. All three controls are shown in Fig. 4 and then compared with their deterministic counterparts in Figs. 4–7. In Fig. 4, it is observed that initially and for some time, all controls are kept at their maximum values. As a result, the prevalence of malaria is reduced relatively fast (as can be seen in Fig. 8). In comparison with the deterministic case, a significant but not drastic difference can be seen. In Fig. 8, we notice that with optimal control on both the stochastic model and the deterministic model, the I-classes in the two cases follow the same trajectory. However, we stress here that optimal control must be maintained on both sides. Therefore, the control manager in particular must monitor the system in order to apply the stochastic control accurately.

Figure 4

The control functions of the stochastic model [$v_{1}=v_{2}=0.005$] for a sample path $\omega _{0}$ of the stochastic process $\{B_{1},B_{2}\}_{t\geq 1}$

Figure 5

Comparison of the optimal control $u_{1}$ for the sample path $\omega _{0}$ of the process $\{B_{1},B_{2}\}_{t\geq 1}$, with the corresponding deterministic control

Figure 6

Comparison of the optimal control function $u_{2}$ for the sample path $\omega _{0}$, with its deterministic counterpart

Figure 7

Comparison of the optimal control function $u_{3}$ for the sample path $\omega _{0}$, with its deterministic counterpart

Figure 8

The trajectory of the I-class under optimal control compared with the mean of 1000 runs of the stochastic optimally controlled I

6 Conclusions

Our analysis of the stochastic model produced a threshold for stability of the disease-free equilibrium, which is similar to the basic reproduction number. Stability of the disease-free equilibrium of our model is described in terms of invariant $\mathfrak{R}$, which is not higher than ${\mathcal{R}}_{0}$. Our stability theorem has a simple form consistent with the deterministic theory. Since exponential stability is almost certain as long as $\mathfrak{R}<1$, it follows that the stochastic perturbation enhances the stability of the disease-free equilibrium. This translates into a better chance of the disease being eliminated from the population. Simulations suggest (as expected) that in general, the expectation of $I(t)$-values in the stochastic model is lower than the corresponding $I(t)$-values in the underlying deterministic model.

The control study revealed some very useful insights too. We have observed that for optimal control, it is important that the manager of the operation, must constantly monitor the dynamical system in order to apply the control accurately, taking into account disturbances on the system. Even for the modeler who prefers to work with deterministic models, it is good to know that minor stochastic perturbations will not lead to disastrous deviations from the deterministic model. In particular, if a system experiences perturbations in reality, then surveillance is important for optimal adjustment of the controls in response to the perturbations.

The cost function that was used in the control problem was chosen just for explorative purposes. If the function can be chosen more realistically then the results obtained in this work become valuable for public health purposes.