 Research
 Open Access
 Published:
Stability analysis of preypredator system with Holling type functional response and prey refuge
Advances in Difference Equations volume 2017, Article number: 243 (2017)
Abstract
In this paper, a predatorprey system with Holling type function response incorporating prey refuge is presented. By applying the analytical approaches, the dynamics behavior of the considered system is investigated, including stability, limit cycle and bifurcation. The results show that the shape of the functional response plays an important role in determining the dynamics of the system. Especially, the interesting conclusion is that the prey refuge has a destabilizing effect under some certain conditions.
Introduction
Researches on predation systems are always a popular issue in contemporary theoretical ecology and applied mathematics [1–12]. Results based on nonspatial systems have shown that the effect of prey refuge played an important role in determining the dynamical consequences of predatorprey systems [1, 3, 4, 7, 8, 10, 13–19]. Incorporating the effect of prey refuge into the considered predation system is initially done by modifying the originally functional response of predator to prey population, the functional response describes the per capita consumption rates of predators depending on prey density, and quantifies the energy transfer between trophic levels, like Holling I, II, III and IV functional response [20, 21]. The most widely reported conclusions are the community/interior/positive/coexistent equilibrium of the considered predation system being stabilized and the equilibrium density of prey and/or predator was enhanced by the addition of prey refuge [3, 8, 12, 19, 22, 23].
This paper is based on the following predatorprey system with prey selflimitation and the population growth of prey is logistic in the absence of predators:
Here, \(x(t)\) and \(y(t)\) are the density of prey and predator populations at time t, respectively, and they are all positive numbers. The other parameters have the following biological meanings: r is the intrinsic per capita growth rate of prey population; K is the prey environmental carrying capacity; c is the efficiency with which predators convert consumed prey into new predators; d is the per capita death rate of predators. The function \(f(x)\) denotes a generalized functional response and represents the amount of prey killed per unit time by an individual predator. This paper applies a generalized representation of the functional response
which is introduced by Real [21] and defined as Holling type functional response, where h is the handling time of predators, λ is the attack efficiency of predator to prey population, the exponent n describes the shape of the functional response, including the Holling II functional response for \(n=1\) and Holling III functional response for \(n=2\).
Incorporating a Holling type functional response into the system (1.1), the following predatorprey system could be obtained:
In this paper, we extend the above model by incorporating the effect of prey refuge. Kot [24] gave a definition for the total number of prey caught V:
where λ is the attack coefficient, h is the handling time required per prey, T is the total time and all parameters are positive. This definition proposes that the total number of prey caught by its predators is linearly proportional to prey’s density. However, most cases may be included in this representation. Hence, without loss of generality, this paper assumes that the total number of prey caught, \(V(x)\), is
where \(T_{s}(x)\) is the available search time and the other parameters are similar to Kot’s definition [24].
According to Maynard Simth [11] and Ma et al. [23], there exists a quantity βx of prey population which occupy refuges. By modifying the total number of prey caught V, it is given that
Solving \(V(x)\) from the above equations, we have
Thus, the modified functional response which incorporates the effect of prey refuge is given by
Based on the above analysis, the system (1.1) with the effect of prey refuge gets the following form:
Before proceeding, we use the following change of variables:
and rewriting x̄, ȳ, t̄ as x, y, t, we obtain the equivalent form of the system (1.3)
We have only three parameters, where \(A=\frac{1}{K^{n} \lambda h(1\beta)^{n}}>0\), \(B=\frac{c}{hr}>0\), \(C=\frac{dh}{c}>0\).
Equilibria
The equilibrium points of the system (1.5) are \(E_{0}(0,0)\), \(E_{K}(K,0)\), \(\tilde{E}(\tilde{x},\tilde{y})\), where
The equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) is positive if and only if
If \(\beta>1\frac{1}{K}(\frac{d}{\lambda(cdh)})^{1/n}\), the equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) collapses with the point \(E_{K}(K,0)\).
Now, differentiating ỹ with respect to β, we observe that ỹ attains its maximum value at
and then it decreases with further increases in β. Also we see that x̃ increases with β.
Again, in order to investigate the local stability and the existence of limit cycle, we should give the equilibrium points of the system (1.6). The equilibrium points of the system (1.6) can be obtained by solving the following equations:
Clearly, it has three equilibrium points \(E_{0}(0,0)\), \(E_{1}(1,0)\), \(\bar{E}(\bar{x},\bar{y})\), where
The equilibrium point \(\bar{E}(\bar{x},\bar{y})\) is positive if and only if \(A<\frac{1C}{C}\).
If \(A=\frac{1C}{C}\), the positive equilibrium point collapses with the equilibrium point \(E_{1}(1,0)\).
The equilibrium point \(\bar{E}(\bar{x},\bar{y})\) lies in the fourth quadrant when \(A>\frac{1C}{C}\).
Positivity and boundedness of the solutions
In order to study the positivity and boundedness for the solutions of system (1.5), we denote the function on the right hand of system (1.5) as \(\mathbf{G}=(xg_{1},yg_{2})\) in which
Clearly, \(G\in C^{1}(R^{2}_{+})\). Thus \(\mathbf{G}:R^{2}_{+} \rightarrow R^{2}\) is locally Lipschitz on \(R^{2}_{+}=\{(x,y)x>0,y>0\}\). Hence the fundamental theorem of existence and uniqueness ensures existence and uniqueness of solution of the system (1.5) with the given initial conditions. The state space of the system is the nonnegative cone in \(R^{2}_{+}\). In the theoretical ecology, positivity and boundedness of the system establishes the biological well behaved nature of the system.
Theorem 3.1
All the solutions of the system (1.5) with the given initial conditions are always positive and bounded.
Proof
Firstly, we wish to prove that \((x(t),y(t))\in R^{2}_{+}\) for all \(t \in[0,+\infty]\). We show this by the method of contradiction. Suppose this is not true. Hence, there must exist one \(\bar{t} \in[0,+\infty]\), such that \(x(\bar{t})\leq0\) and \(y(\bar{t})\leq 0\). From the system (1.5), we have
Since \((x(t),y(t))\) are well defined and continuous on \([0,\bar{t}]\), there must exist a \(M>0\) such that \(\forall t \in[0,\bar{t}]\)
It is clear that if we have the limit \(t \rightarrow\bar{t}\), we obtain
which is a contradiction. Hence, all the solutions of the system (1.5) are always positive.
Secondly, we will prove the boundedness.
Letting \(V(t)=x(t)+\frac{1}{c}y(t)\), then we obtain
Integrating both sides of the above equation and applying the theorem of the differential inequality, we have
and \(\lim_{t \rightarrow+\infty} V(t)\leq\frac{(d+r)K}{d}\). □
Stability analysis
Local stability
The Jacobian matrix of the system (1.6) at the equilibrium point \(E_{1}(1,0)\) is given by
The two eigenvalues of matrix \(J_{1}\) are \((A+1)\) and \(B(1CAC)\).
Hence, if \(A<\frac{1C}{C}\), the equilibrium point \(E_{1}(1,0)\) is a saddle point. Otherwise, the equilibrium point \(E_{1}(1,0)\) is locally asymptotically stable.
For the positive equilibrium point \(\bar{E}(\bar{x},\bar{y})\), the Jacobian matrix is as follows:
Here
Clearly \(\operatorname{Det}J_{2}=A_{12}A_{21}>0\). Therefore, the sign of the eigenvalues of Jacobian matrix \(J_{2}\) depends only on \(\operatorname{Tr}J_{2}=A_{11}\).
Hence, the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) is locally asymptotically stable if and only if
Case 1: If \(0< n<\frac{1}{1C}\), then \(2n(1C)>0\) and \(1n(1C)>0\).
The inequality (4.1) can easily be solved as follows:
In other words
Hence, the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) is locally asymptotically stable.
The results of GonzálezOlivares et al. [6] and Huang et al. [8] are special cases of ours for \(n=1\) and \(n=2\), respectively.
Case 2: If \(\frac{1}{1C}\leq n \leq\frac{2}{1C}\), then the inequality (4.1) holds.
Therefore, the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) is always asymptotically stable whenever the proportion of prey refuge is.
Case 3: If \(n>\frac{2}{1C}\), then \(2n(1C)<0\) and \(1n(1C)<0\).
Hence, the inequality (4.1) is equivalent to the following condition:
It is easy to show that
Therefore, the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) is locally asymptotically stable under these assumptions.
Existence of limit cycle
Let us rewrite the system (1.6) in the following form:
Here \(g(x)=(1x)(A+x^{n})\), \(p(x)=x^{n}\), \(q(x)=(1C)p(x)\).
We present a lemma [17] regarding uniqueness of limit cycle of the above system.
Lemma
[17]
Suppose the system (4.2) obeys
in \(0\leq x< \bar{x}\) and \(\bar{x}< x\leq1\). Then the system (4.2) has exactly one limit cycle which is globally asymptotically stable with respect to the set \(\{(x,y)x>0,y>0\}\backslash \{E_{2}(\bar{x},\bar{y})\}\).
By employing the above lemma, we have
Here
This will be equivalent to prove that \(\varphi(x)\leq0\) for all \(x>0\).
Noticing that \(\varphi(0)=A^{2}C(n2)\leq0\) for \(n\geq2\) and
It is easy to show that \(x=0\) and \(x=\bar{x}\) are the solutions of the equation \(\varphi'(x)=0\).
Again \(\varphi''(\bar{x})=nA\bar{x}^{n2}[2C(1+n)+(n1)(n2)(1C)]<0\).
Thus, \(x=\bar{x}\) is the maximum value point of the function \(\varphi(x)\).
Now, in order to prove \(\varphi(x)\leq0\), it is enough to show that \(\varphi(\bar{x})\leq0\).
That is,
Clearly, it is exactly the condition of the instability of the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\).
Global stability
In this section, we will prove the global stability of the positive equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) of the system (1.5).
We first choose a Lyapunov function defined as follows:
By a simple computation, we obtain
Selecting \(p=\frac{\lambda(1\beta)^{n}\tilde{x}^{n1}}{n c \lambda (1\beta)^{n}\tilde{x}^{n1}(1+\lambda h(1\beta)^{n}\tilde{x}^{n})}>0\), then we have
Thus, \(\frac{dW}{dt}<0\) if \(n \geq 1\). Hence, the positive equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) of the system (1.5) is globally asymptotically stable.
Main results
According to the above analysis, we can obtain the following results.
Theorem 5.1
Assuming that \(2\leq n<\frac{1}{1C}\), then we have:

(1)
If \(0< A<\frac{1C}{C}[\frac{1n(1C)}{2n(1C)}]^{n}\), the system (1.4) has a unique globally stable limit cycle surrounding the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) which is unstable.

(2)
If \(\frac{1C}{C}[\frac{1n(1C)}{2n(1C)}]^{n}< A<\frac{1C}{C}\), the system (1.4) has a globally asymptotically stable equilibrium point \(\bar{E}(\bar{x},\bar{y})\) at the first quadrant.
Theorem 5.2
Assuming that \(n>\frac{2}{1C}\), then we have:

(1)
If \(0< A<\frac{1C}{C}[\frac{1n(1C)}{2n(1C)}]^{n}\), the system (1.4) has a globally asymptotically stable equilibrium point \(\bar{E}(\bar{x},\bar{y})\) in the first quadrant.

(2)
If \(\frac{1C}{C}[\frac{1n(1C)}{2n(1C)}]^{n}< A<\frac{1C}{C}\), the system (1.4) has a unique globally stable limit cycle surrounding the interior equilibrium point \(\bar{E}(\bar{x},\bar{y})\) which is unstable.
In reference to the original parameters of the system (1.5), the above results can be expressed as follows.
Theorem 5.3
Assuming that \(2\leq n<\frac{c}{cdh}\), then we have:

(1)
If \(0<\beta<1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac {2cn(cdh)}{cn(cdh)}]\), the prey and predator populations stably oscillate around the unique interior equilibrium point.

(2)
If \(1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac {2cn(cdh)}{cn(cdh)}]<\beta<1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}\), the two populations tend to reach a globally asymptotically stable equilibrium point at the first quadrant.
Theorem 5.4
Assuming that \(n>\frac{2c}{cdh}\), then we have:

(1)
If \(0<\beta<1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac {2cn(cdh)}{cn(cdh)}]\), the two populations tend to reach a globally asymptotically stable equilibrium point in the first quadrant.

(2)
If \(1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac {2cn(cdh)}{cn(cdh)}]<\beta<1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}\), the prey and predator populations stably oscillate around the unique interior equilibrium point.
Bifurcation analysis
To obtain a complete classification of the qualitative behavior of the system (1.5), we analyze the bifurcation pattern and illustrate the results with one parameter, saying the effect of prey refuge β. The classification requires up to two codimensionone bifurcations: (i) Hopfbifurcation point in which the coexistence equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) exchanges stability, (ii) the bifurcation point tracking a transcritical bifurcation between the coexistence equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) and the prey only equilibrium \(E_{K}(K,0)\), where these two equilibria coincide and exchange their stability to each other.
Hopf bifurcation
Onedimensional bifurcation analysis reveals the behavior of the system (1.5) when a particular system parameter is varied over a long range. Here we observe the behavior of the system (1.5) when the prey refuge intensity is varied.
By simple computation, the characteristic equation of the system (1.5) at the coexistence equilibrium point \(\tilde{E}(\tilde{x},\tilde{y})\) is
in which
It can easily be observed from the characteristic equation (6.1) that the roots become purely imaginary when \(\tilde{a}_{11}=0\), i.e. \(\beta=\beta_{c}= 1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac{2cn(cdh)}{cn(cdh)}]\). In this case, \(\operatorname{Re}(\lambda)_{\beta=\beta_{c}}=0\), \(\operatorname{Im}(\lambda)_{\beta=\beta_{c}} \neq0\) and \(\frac{d}{d \beta}\operatorname{Re}(\lambda)_{\beta=\beta_{c}}<0\) (we use the standard package of Mathematica to get these results) and hence the transversality condition for a Hopf bifurcation is satisfied. Therefore, there exists a Hopf bifurcation at \(\beta=\beta_{c}\). The negative sign of \(\frac{d}{d \beta}\operatorname{Re}(\lambda)_{\beta=\beta_{c}}<0\) implies that the oscillations in the population densities dampen as the effect of prey refuge passes from lower value to higher value through \(\beta=\beta_{c}\).
Hence, we obtain the following results.
Theorem 6.1
The system (1.5) undergoes a Hopf bifurcation at \(\tilde{E}(\tilde{x},\tilde{y})\) when the effect of prey refuge β passes the threshold value \(\beta_{c}= 1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac{2cn(cdh)}{cn(cdh)}]\).
Transcritical bifurcation
In this section, we will consider the existence of a transcritical bifurcation for the system (1.5). In order to do this, we select the effect of prey refuge β as the bifurcation parameter. According to the analysis in Section 3, if \(1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}[\frac{2cn(cdh)}{cn(cdh)}] <\beta<1\frac{1}{K}(\frac{d}{\lambda(cdh)})^{1/n}\), the coexistence equilibrium Ẽ is stable but the axial equilibrium \(E_{K}\) is unstable. The two equilibria coincide a \(\beta=\beta_{0}=1\frac{1}{K}(\frac{d}{\lambda(cdh)})^{1/n}\) and exchange their stability when \(1\frac{1}{K}(\frac{d}{\lambda(cdh)})^{1/n}<\beta<1\).
Now, we will prove that the system (1.5) undergoes a transcritical bifurcation by using Sotomayer’s theorem [2]. For \(\beta=\beta_{0}\), the system (1.5) has only one axial equilibrium point \(E_{K}\). The Jacobian matrix evaluated at \(E_{K}\) is
J has an eigenvalue \(\lambda=0\). Let V and W be the eigenvectors corresponding to the eigenvalue \(\lambda=0 \) for J and \(J^{T} \), respectively, then one can calculate
Using the expressions for V and W, we get
Hence, according to Sotomayor’s theorem, the system (1.5) undergoes a transcritical bifurcation when the effect of prey refuge β passes through the threshold value \(\beta_{0}\).
Hence, we obtain the following results.
Theorem 6.2
The system (1.5) undergoes a transcritical bifurcation when the effect of prey refuge β passes the threshold value \(\beta_{0}=1\frac{1}{K}(\frac{d}{\lambda(cdh)})^{1/n}\).
Discussion
In this paper, we have considered a predatorprey system with a general functional response incorporating a prey refuge. Our analysis reveals that

(i)
The equilibrium point \(E_{K}(K,0)\) is locally asymptotically stable if the proportion of refuge using by prey is larger than \(1\frac{1}{K}[\frac{d}{\lambda(cdh)}]^{1/n}\). Therefore, when the refuge using by prey is high, the system predicts that the prey population reaches its carrying capacity and the predators go extinct, a dynamics also observed by Collings [5] for some certain parameters.

(ii)
The shape of the functional response plays an important role in determining the dynamic behavior of the system. If \(0< n<\frac{c}{cdh}\), the effect of prey refuge has a stabilizing effect, which is consistent with results of GonzálezOlivares and RamosJiliberto [6] who has found a clear stabilizing effect on their considered system. Here, stabilization or the increase of stability refers to cases where a community equilibrium point changes from repeller to an attractor due to changes in the value of a control parameter [6]. However, if the exponent n is larger than \(\frac{2c}{cdh}\), the stability of the interior equilibrium point changes from the globally asymptotically stable state to the unstable state surrounding a globally stable limit cycle as the refuge using by prey increases. The prey refuge can decrease the stability of the interior equilibrium point. We call this a destabilizing effect.
References
 1.
Ruxton, GD: Short term refuge use and stability of predatorprey models. Theor. Popul. Biol. 47, 117 (1995)
 2.
Collings, JB: Bifurcation and stability analysis of a temperaturedependent mite predatorprey interaction model incorporating a prey refuge. Bull. Math. Biol. 57(1), 6376 (1995)
 3.
GonzálezOlivares, E, RamosJiliberto, R: Dynamic consequences of prey refuges in a simple model system: more prey, fewer predators and enhanced stability. Ecol. Model. 166, 135146 (2003)
 4.
Jana, D, Agrawal, R, Upadhyay, RK: Dynamics of generalist predator in a stochastic environment: effect of delayed growth and prey refuge. Appl. Math. Comput. 268, 10721094 (2015)
 5.
Wang, J, Pan, L: Qualitative analysis of a harvested predatorprey system with Hollingtype III functional response incorporating a prey refuge. Adv. Differ. Equ. 2012, 96 (2012)
 6.
Harrison, GW: Global stability of predatorprey interactions. J. Math. Biol. 8, 159171 (1979)
 7.
Huang, Y, Chen, F, Li, Z: Stability analysis of preypredator model with Holling type response function incorporating a prey refuge. Appl. Math. Comput. 182, 672683 (2006)
 8.
Kar, TK: Stability analysis of a preypredator model incorporating a prey refuge. Commun. Nonlinear Sci. Numer. Simul. 10, 681691 (2005)
 9.
Jana, D, Bairagi, N: Habitat complexity, dispersal and metapopulations: macroscopic study of a predatorprey system. Ecol. Complex. 17, 131139 (2014)
 10.
Jana, D, Ray, S: Impact of physical and behavioral prey refuge on the stability and bifurcation of Gause type Filippov preypredator system. Model. Earth Syst. Environ. 2, 24 (2016)
 11.
Jana, D: Chaotic dynamics of a discrete predatorprey system with prey refuge. Appl. Math. Comput. 224, 848865 (2013)
 12.
Sih, A: Prey refuges and predatorprey stability. Theor. Popul. Biol. 31, 112 (1987)
 13.
Ives, AR, Dobson, AP: Antipredator behavior and the population dynamics of simple predatorprey systems. Am. Nat. 130, 431447 (1987)
 14.
Jana, D, Agrawal, R, Upadhyay, RK: Dynamics of generalist predator in a stochastic environment: effect of delayed growth and prey refuge. Appl. Math. Comput. 268, 10721094 (2015)
 15.
Maynard Smith, J: Models in Ecology. Cambridge University Press, Cambridge (1974)
 16.
Mukherjee, D: The effect of refuge and immigration in a predatorprey system in the presence of a competitor for the prey. Nonlinear Anal., Real World Appl. 31, 277287 (2016)
 17.
Strogatz, SH: Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Perseus Publishing, Cambridge (1994)
 18.
Ghosh, J, Sahoo, B, Poria, S: Preypredator dynamics with prey refuge providing additional food to predator. Chaos Solitons Fractals 96, 110119 (2017)
 19.
McNair, JM: The effects of refuges on predatorprey interactions: a reconsideration. Theor. Popul. Biol. 29, 3863 (1986)
 20.
Taylor, RJ: Predation. Chapman & Hall, New York (1984)
 21.
Real, LA: The kinetics of functional response. Am. Nat. 111, 289300 (1977)
 22.
Murdoch, WW, Oaten, A: Predation and population stability. Adv. Ecol. Res. 9, 2132 (1975)
 23.
Ma, Z, Li, W, Zhao, Y, Wang, W, Zhang, H, Li, Z: Effects of prey refuges on a predatorprey model with a class of functional responses: the role of refuges. Math. Biosci. 218, 7379 (2009)
 24.
Kot, M: Elements of Mathematical Ecology. Cambridge University Press, Cambridge (2011)
Acknowledgements
We would like to thank the editor and the anonymous referees very much for their valuable comments and suggestions. This work was supported by the National Natural Science Foundation of China (No. 11301238) and the Fundamental Research Funds for the Central Universities (No. lzujbky2017166).
Author information
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors contributed equally and significantly in this paper. All authors read and approved the final manuscript.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
MSC
 37C75
 34K18
 92B05
Keywords
 predatorprey system
 prey refuge
 stability
 limit cycle
 bifurcation
 destabilizing effect