Theory and Modern Applications

# Stability and bifurcation analysis of an amensalism system with Allee effect

## Abstract

In this work, we propose and study a new amensalism system with Allee effect on the first species. First, we investigate the existence and stability of all possible coexistence equilibrium points and boundary equilibrium points of this system. Then, applying the Sotomayor theorem, we prove that there exists a saddle-node bifurcation under some suitable parameter conditions. Finally, we provide a specific example with corresponding numerical simulations to further demonstrate our theoretical results.

## Introduction

The interaction between two or more species has been a central problem in ecology and biology since the famous Lotka–Volterra model was proposed. The interaction between different species generates a complicated dynamics of biological species and exhibits the complexity and diversity. Amensalism, a typical type of interaction between the species, has been intensively considered in the last decades. Amensalism describes a basic biological interaction in nature, where one species inflicts harm on another not affected by the former, which means that it does not receive any costs or benefits to itself. The first pioneer work for the investigations of amensalism model is due to Sun , who, in 2003, proposed the following two-species amensalism model:

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}=r_{1}x (\frac{k_{1}-x-T y}{k_{1}} ), \\ \frac{dy}{dt}=r_{2}y(\frac{k_{2}-y}{k_{2}}), \end{cases}\displaystyle \end{aligned}
(1)

where $$x=x(t)$$ and $$y=y(t)$$ represent the population densities of two species at time t, respectively, and $$r_{1}$$, $$r_{2}$$, $$k_{1}$$, $$k_{2}$$, and T are positive real numbers. In  the author investigated the stability properties of the equilibrium points of this system.

By rescaling we can see that model (1) can be rewritten as

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}=x (r_{1}-a_{11}x-a_{12} y ), \\ \frac{dy}{dt}=y(r_{2}-a_{22}y), \end{cases}\displaystyle \end{aligned}
(2)

where the parameters $$r_{1}$$, $$r_{2}$$, $$a_{11}$$, $$a_{12}$$, and $$a_{22}$$ are positive constants.

Since the first amensalism model was presented, there are numerous relevant works that focus on the complicated dynamics of amensalism models from different aspects . For example, in  the author discussed the dynamical properties of a two-species amensalism system with nonmonotonic function response. Guan et al.  considered a two-species amensalism model with Beddington–DeAngelis functional response. The two-species amensalism model with Michaelis–Menten-type harvesting and a cover for the first species was proposed in , where the stability and bifurcation of this system were investigated.

In the nature world the dynamics of population is inevitably affected by difficulties in finding mate, predator avoidance, the evading natural enemies, and resource defense , where the Allee effect on species occurs. The Allee effect, as a significant phenomenon in population dynamics, was intensively investigated in the last decades. It was found that the Allee effect has a very important impact on the biological system and exhibits complex dynamics. Generally speaking, a population is said to have the Allee effect if the per capita population growth rate and population density have a positive correlation in a small population [17, 18]. Moreover, the Allee effect has two types, weak and strong[19, 20]. The weak Allee effect indicates that per capita growth rate is smaller at low species density, but not negative. The strong Allee effect means that the per capita growth rate is negative at low species density. Nowadays, a lot of efforts have been made to investigate the influence of the Allee effect on the dynamical behavior of biological systems [12, 2126]. Especially, according to system (2), Wei et al.  recently proposed an amensalism model with weak Allee effect for the second species and studied its stability and bifurcation.

Inspired by the previous works, we naturally want to know: for an amensalism system, what about its dynamical properties when an Allee effect is introduced to the first species? Hence, based on system (2), we consider the following amensalism system with a weak Allee effect on the first species:

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}=x (\frac{r_{1}x}{m+x}-a_{11}x-a_{12}y ), \\ \frac{dy}{dt}=y(r_{2}-a_{22}y), \end{cases}\displaystyle \end{aligned}
(3)

where all parameters $$r_{1}$$, $$r_{2}$$, $$a_{11}$$, $$a_{12}$$, $$a_{22}$$, and m are positive constants, and the term $$K(x)=\frac{ x}{m + x}$$ represents a weak Allee effect, where m describes the intense of Allee effect on the first species; this function satisfies the following properties :

1. (i)

$$\lim_{x \to 0}K(x) = 0$$, that is, there is no reproduction without partners;

2. (ii)

$$K'(x) > 0$$ for all $$x \in (0,\infty )$$, that is, the Allee effect decreases as population density increases;

3. (iii)

$$\lim_{x \to \infty } K(x)=1$$, which means that the Allee effect disappears at high population densities.

Setting

\begin{aligned} \overline{t}=r_{2}t,\qquad \overline{x}=\frac{a_{11}}{r_{2}}x,\qquad \overline{y}= \frac{a_{22}}{r_{2}}y \end{aligned}

and dropping the bars, we transform system (3) into

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}=x (\frac{\alpha x}{\gamma +x}-x-\beta y ), \\ \frac{dy}{dt}=y(1-y), \end{cases}\displaystyle \end{aligned}
(4)

where $$\alpha =\frac{r_{1}}{r_{2}}$$, $$\beta =\frac{a_{12}}{a_{22}}$$, and $$\gamma =\frac{ma_{11}}{r_{2}}$$.

Our main purpose in this paper is investigating the local stability property of the possible equilibrium points and the saddle-node bifurcation of system (4). The rest of this paper is arranged as follows. In Sect. 2, we present the local dynamical behaviors, including the distribution of possible equilibrium points of system (4) and their stability. In Sect. 3, we prove the existence of saddle-node bifurcation when the coefficient α is chosen as a bifurcation parameter. In Sect. 4, we give an example with specific parameter values and the corresponding numerical simulations to further illustrate the validity of the main results. Finally, we end this paper with a brief conclusion in Sect. 5.

## Existence and stability of equilibrium points

### Existence of equilibrium points

The equilibrium points of system (4) satisfy the following equations:

\begin{aligned} \textstyle\begin{cases} x (\frac{\alpha x}{\gamma +x}-x-\beta y )=0, \\ y(1-y)=0. \end{cases}\displaystyle \end{aligned}
(5)

By a simple computation we derive that system (4) has boundary equilibrium points $$P_{0}(0, 0)$$ and $$P_{1}(0, 1)$$. When $$\alpha >\gamma$$, there exists another boundary equilibrium point $$P_{2}(\alpha -\gamma ,0)$$. Moreover, if $$x\neq 0$$ and $$y\neq 0$$, then there exists a coexistence (positive) equilibrium point $$P_{3}(x_{3}, 1)$$, where $$x_{3}$$ is the root of the following equation:

\begin{aligned} f(x)=x^{2}+(\beta +\gamma -\alpha )x+\beta \gamma =0. \end{aligned}

Set the discriminant of this equation,

\begin{aligned} \Delta (\alpha )=(\beta +\gamma -\alpha )^{2}-4\beta \gamma =\alpha ^{2}-2( \beta +\gamma )\alpha +(\beta -\gamma )^{2}, \end{aligned}
(6)

and denote the roots of $$\Delta (\alpha )=0$$ by $$\alpha _{1}$$ and $$\alpha _{2}$$. Then

\begin{aligned} \alpha _{1}= \beta +\gamma -2\sqrt{\beta \gamma }, \qquad \alpha _{2}= \beta +\gamma +2\sqrt{\beta \gamma }. \end{aligned}
(7)

Because the parameters β and γ are positive, using the mean value theorem, we can get that $$\alpha _{2} > \alpha _{1}\geq 0$$.

Considering all possible coexistence and boundary equilibria, we obtain the following results.

### Theorem 2.1

For system (4), there always are the boundary equilibrium points$$P_{0}(0, 0)$$and$$P_{1}(0, 1)$$. Furthermore, we have:

1. (1)

if$$\alpha >\gamma$$, then system (4) has another boundary equilibrium point$$P_{2}(\alpha -\gamma ,0)$$;

2. (2)

For the possible coexistence equilibrium points,

1. (i)

if$$\alpha <\alpha _{2}$$, then system (4) has no coexistence equilibrium point;

2. (ii)

if$$\alpha =\alpha _{2}$$, then there is a unique coexistence equilibrium point$$P_{31}(x_{31},1)$$, where$$x_{31}=\sqrt{\beta \gamma }$$;

3. (iii)

if$$\alpha >\alpha _{2}$$, then there are two coexistence equilibrium points$$P_{32}(x_{32},1)$$and$$P_{33}(x_{33},1)$$, where$$x_{32,33}=\frac{\alpha -\beta -\gamma \mp \sqrt{\Delta }}{2}$$.

### Proof

It is obvious that (1) holds. If $$\alpha _{1}<\alpha <\alpha _{2}$$, then $$\Delta (\alpha ) < 0$$, which implies that $$f(x)$$ has no real roots; if $$\alpha \leq \alpha _{1}$$, then $$f(0) = \beta \gamma > 0$$, and the symmetry axis of $$f(x)$$ is $$x= \frac{\alpha -\beta -\gamma }{2}\leq \frac{\alpha _{1}-\beta -\gamma }{2}= -\sqrt{\beta \gamma } < 0$$, so there is no positive real solution for $$f (x) = 0$$. Thus we complete the proof of (2)(i).

When $$\alpha =\alpha _{2}$$, then $$\Delta (\alpha ) = 0$$, which means that $$f (x) = 0$$ has a unique positive solution $$x_{31}=\sqrt{\beta \gamma }$$. Hence the proof of (2)(ii) is completed.

When $$\alpha >\alpha _{2}$$, then $$\Delta (\alpha ) > 0$$. Combing this with $$f (0) = \beta \gamma > 0$$ and the symmetry axis $$x= \frac{\alpha -\beta -\gamma }{2}\geq \frac{\alpha _{2}-\beta -\gamma }{2}= \sqrt{\beta \gamma } >0$$, we can get that $$f (x) = 0$$ has two positive roots $$x_{32}=\frac{\alpha -\beta -\gamma - \sqrt{\Delta }}{2}$$ and $$x_{33}=\frac{\alpha -\beta -\gamma + \sqrt{\Delta }}{2}$$. Thus the proof of (2)(iii) is finished, and we complete the proof of Theorem 2.1. □

### Stability of the equilibrium points

The Jacobian matrix of system (4) evaluated at any equilibrium point is

\begin{aligned} J(x,y)= \begin{pmatrix} H(x, y)&-\beta x \\ 0&1-2y \end{pmatrix}, \end{aligned}
(8)

where

\begin{aligned} H(x, y)=\frac{2\alpha \gamma x+\alpha x^{2}}{(\gamma +x)^{2}}-2 x- \beta y. \end{aligned}

Consider the stability of boundary equilibria $$P_{0}$$, $$P_{1}$$, and $$P_{2}$$, we have the following results.

### Theorem 2.2

For system (4), the following statements are true.

1. (1)

For the equilibrium point$$P_{0}$$:

1. (i)

If$$\alpha \neq \gamma$$, then$$P_{0}$$is a saddle node. That is, $$S_{\varepsilon }(P_{0})$$is divided into two parts by two separatrices that tend to$$P_{0}$$along the upside and the underneath of$$P_{0}$$, where$$S_{\varepsilon }(P_{0})$$is a neighborhood of$$P_{0}$$with sufficient small radiusε. One part consists of two hyperbolic sectors, and the other part is a parabolic sector. Moreover, if$$\alpha > \gamma$$ (or$$\alpha < \gamma$$), then the parabolic sector is on the right (or left) half-plane.

2. (ii)

If$$\alpha = \gamma$$, then$$P_{0}$$is a nonhyperbolic saddle.

2. (2)

The equilibrium point$$P_{1}$$is a hyperbolic stable node. Moreover, $$P_{1}$$is globally asymptotically stable for$$\alpha <\gamma$$.

3. (3)

If$$\alpha >\gamma$$, then the equilibrium point$$P_{2}$$is a hyperbolic saddle.

### Proof

(1) The Jacobian matrix of system (4) evaluated at the equilibrium point $$P_{0}$$ is given by

\begin{aligned} J(P_{0})= \begin{pmatrix} 0&0 \\ 0&1 \end{pmatrix}, \end{aligned}
(9)

and the two eigenvalues of $$J(P_{0})$$ are $$\lambda _{1}(P_{0})=0$$ and $$\lambda _{2}(P_{0})=1>0$$. Obviously, the equilibrium $$P_{0}$$ is nonhyperbolic, so it is hard to directly judge its type from the eigenvalues. We further discuss its stability properties by applying Theorem 7.1 in Chap. 2 in .

To change system (4) into a standard form, we expand system (4) in power series up to the fourth order around the origin:

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}= (\frac{\alpha }{\gamma }-1 )x^{2}-\beta x y- \frac{\alpha }{\gamma ^{2}}x^{3}+\frac{\alpha }{\gamma ^{3}}x^{4}+Q_{0}(x)= P(x, y), \\ \frac{dy}{dt}=y-y^{2}= y+Q(x,y), \end{cases}\displaystyle \end{aligned}
(10)

where $$Q_{0}(x)$$ represents a power series with terms $$x^{i}$$ ($$i\geq 5$$).

Combining the implicit function theorem with the second equation of (10), we get that there is a unique function $$y=\varphi (x)=0$$ such that $$\varphi (x)+Q(x, \varphi (x)) = 0$$ and $$\varphi (0)=\varphi '(0)=0$$. Then substituting $$y=\varphi (x)=0$$ into the first equation of (10), we get that

\begin{aligned} \frac{dx}{dt}= \biggl(\frac{\alpha }{\gamma }-1 \biggr)x^{2}- \frac{\alpha }{\gamma ^{2}}x^{3}+\frac{\alpha }{\gamma ^{3}}x^{4}+Q_{0}(x). \end{aligned}
(11)

When $$\alpha \neq \gamma$$, we obtain that the coefficient at $$x^{2}$$ is $$\frac{\alpha }{\gamma }-1\neq 0$$. So, based on Theorem 7.1 in Chap. 2 in , the equilibrium point $$P_{0}$$ is a saddle node. This means that a neighborhood $$S_{\varepsilon }(P_{0})$$ (ε is a sufficiently small radius) is divided into two parts by two separatrices that tend to $$P_{0}$$ along the upside and the underneath of $$P_{0}$$. One part is a parabolic sector, and the other part consists of two hyperbolic sectors. Furthermore, if $$\alpha > \gamma$$ (or $$\alpha < \gamma$$), then the parabolic sector is on the right (or left) half-plane.

When $$\alpha =\gamma$$, (11) becomes

\begin{aligned} \frac{dx}{dt}=-\frac{1}{\gamma }x^{3}+ \frac{1}{\gamma ^{2}}x^{4}+Q_{0}(x). \end{aligned}
(12)

Applying the notations of Theorem 7.1 in Chap. 2 in , we have $$m=3$$ and $$a_{m}=\frac{1}{\gamma }<0$$, so the equilibrium point $$P_{0}$$ is a nonhyperbolic saddle.

(2) For the equilibrium point $$P_{1}$$, the Jacobian matrix is

\begin{aligned} J(P_{1})= \begin{pmatrix} -\beta &0 \\ 0&-1 \end{pmatrix}. \end{aligned}
(13)

The two eigenvalues of this matrix $$J(P_{1})$$ are $$\lambda _{1}(P_{1})=-\beta <0$$ and $$\lambda _{2}(P_{1})=-1<0$$, and thus $$P_{1}$$ is a hyperbolic stable node.

When $$\alpha <\gamma$$, there are two equilibrium points $$P_{0}$$, which is unstable, and $$P_{1}$$, which is locally asymptotically stable. To ensure that $$P_{1}$$ is globally asymptotically stable, we consider the Lyapunov function $$V(x, y) = x$$. Then

\begin{aligned} \frac{dV}{dt}=x \biggl(\frac{\alpha x}{\gamma +x}-x-\beta y \biggr) = \frac{\alpha -\gamma - x}{\gamma +x}x^{2}-\beta x y. \end{aligned}

Obviously, if $$\alpha <\gamma$$, then $$\frac{dV}{dt}<0$$ for all $$x>0$$, $$y > 0$$. So $$P_{1}$$ is globally asymptotically stable for $$\alpha <\gamma$$.

(3) The Jacobian matrix of system (4) at equilibrium point $$P_{2}$$ is

\begin{aligned} J(P_{2})= \begin{pmatrix} -\frac{\alpha ^{2}-2\alpha \gamma +\gamma ^{2}}{\alpha }&-\beta ( \alpha -\gamma ) \\ 0&1 \end{pmatrix}. \end{aligned}
(14)

When $$\alpha >\gamma$$, it is obvious that $$J(P_{1})$$ has two characteristic roots $$\lambda _{1}(P_{1})=- \frac{\alpha ^{2}-2\alpha \gamma +\gamma ^{2}}{\alpha }<0$$ and $$\lambda _{2}(P_{1})=1>0$$, so the equilibrium point $$P_{2}$$ is a hyperbolic saddle.

This ends the proof of Theorem 2.2. □

Discussing the stability of the coexistence equilibria $$P_{31}$$, $$P_{32}$$, and $$P_{33}$$, we have the following results.

### Theorem 2.3

For system (4), we have the following statements.

1. (1)

If$$\alpha =\alpha _{2}$$, then the equilibrium point$$P_{31}$$is a saddle node, that is, $$S_{\varepsilon }(P_{31})$$is divided into two parts by two separatrices that tend to$$P_{31}$$along the upside and the underneath of$$P_{31}$$, where$$S_{\varepsilon }(P_{31})$$is a neighborhood of$$P_{31}$$with sufficient small radius ε. One part consists of two hyperbolic sectors, and the other part is a parabolic sector. Moreover, the parabolic sector is on the right half-plane.

2. (2)

If$$\alpha >\alpha _{2}$$, then the equilibrium point$$P_{32}$$is a hyperbolic saddle, and the equilibrium point$$P_{33}$$is a hyperbolic stable node.

### Proof

(1) The Jacobian matrix of system (4) at the equilibrium point $$P_{31}$$ is given by

\begin{aligned} J(P_{31})= \begin{pmatrix} H(x_{31},1)&-\beta \sqrt{\beta \gamma } \\ 0&-1 \end{pmatrix}. \end{aligned}
(15)

Note that $$P_{31}(\sqrt{\beta \gamma },1)$$ satisfies the equation $$\frac{\alpha x_{31}}{\gamma +x_{31}}=x_{31}+\beta$$. By some calculations we can derive that $$H(x_{31},1)=\frac{\alpha _{2}\gamma x_{31}}{(\gamma +x_{31})^{2}}-x_{31}=0$$. Then two characteristic roots of $$J(P_{31})$$ are $$\lambda _{1}(P_{31})=0$$ and $$\lambda _{2}(P_{31})=-1<0$$, so the equilibrium point $$P_{31}$$ is nonhyperbolic, and its stability cannot be given directly from the eigenvalues. We next analyze its stability behavior by using Theorem 7.1 in Chap. 2 in .

Letting $$X=x-x_{31}$$ and $$Y=y-1$$, we translate the equilibrium point $$P_{31}$$ to the origin and then expand the corresponding system in power series up to the third order around the origin, which makes system (4) to be the following form:

\begin{aligned} \textstyle\begin{cases} \frac{dX}{dt}=c_{0}+c_{1} X- \beta x_{31}Y+c_{2}X^{2}- \beta X Y+c_{3}X^{3}+Q_{1}(X), \\ \frac{dY}{dt}=-Y-Y^{2}, \end{cases}\displaystyle \end{aligned}
(16)

where

\begin{aligned}& c_{0}=\frac{\alpha _{2}\gamma ^{2}}{\gamma +x_{31}}+\alpha _{2}(x_{31}- \gamma )-\beta x_{31}-x_{31}^{2}=0, \\& c_{1}=\frac{1}{\gamma +x_{31}} \bigl(\beta \gamma -x_{31}^{2} \bigr)=0,\qquad c_{2}=\frac{\alpha _{2}\gamma ^{2}}{(\gamma +x_{31})^{3}}-1,\qquad c_{3}=- \frac{\alpha _{2}\gamma ^{2}}{(\gamma +x_{31})^{4}}, \end{aligned}

and $$Q_{1}(X)$$ represents a power series with terms $$X^{i}$$ ($$i\geq 4$$).

To transform the Jacobian matrix into a standard form, we use the invertible translation

\begin{aligned} \begin{pmatrix} u \\ v \end{pmatrix} = \begin{pmatrix} 1&-\beta x_{31} \\ 0&1 \end{pmatrix} \begin{pmatrix} X \\ Y \end{pmatrix}. \end{aligned}
(17)

Then system (16) becomes

\begin{aligned} \textstyle\begin{cases} \frac{du}{dt}=d_{20}u^{2}+d_{11}uv+d_{02}v^{2}+d_{30}u^{3}+d_{21}u^{2}v+d_{12}uv^{2}+d_{03}v^{3} \\ \hphantom{\frac{du}{dt}=}{}+Q_{2}(u,v), \\ \frac{dv}{dt}=-v-v^{2}, \end{cases}\displaystyle \end{aligned}
(18)

where

\begin{aligned}& d_{20}=-1+\frac{\alpha _{2}\gamma ^{2}}{(\gamma +x_{31})^{3}},\qquad d_{11}=- \beta \biggl(1+2 x_{31}- \frac{2\alpha _{2}\gamma ^{2} x_{31}}{(\gamma +x_{31})^{3}} \biggr), \\& d_{02}=\beta x_{31} \biggl(1-\beta -\beta x_{31}+ \frac{\alpha _{2}\beta \gamma ^{2}x_{31}}{(\gamma +x_{31})^{3}} \biggr),\qquad d_{30}=- \frac{\alpha _{2}\gamma ^{2}}{(\gamma +x_{31})^{4}}, \\& d_{21}=- \frac{3\alpha _{2}\beta x_{31}\gamma ^{2}}{(\gamma +x_{31})^{4}},\qquad d_{12}=- \frac{3\alpha _{2}\gamma ^{2}\beta ^{2} x_{31}^{2}}{(\gamma +x_{31})^{4}},\qquad d_{03}=- \frac{\alpha _{2}\gamma ^{2}\beta ^{3}x_{31}^{3}}{(\gamma +x_{31})^{4}}, \end{aligned}

and $$Q_{2}(u, v)$$ is a power series with terms $$u^{i}v^{j}$$ ($$i+j \geq 4$$).

By introducing a new time variable $$T= -t$$ we get

\begin{aligned} \textstyle\begin{cases} \frac{du}{dT}=-d_{20}u^{2}-d_{11}u v-d_{02} v^{2}-d_{30} u^{3}-d_{21} u^{2}v-d_{12}uv^{2}-d_{03}v^{3} \\ \hphantom{\frac{du}{dT}=}{}-Q_{2}(u,v)=P(u,v), \\ \frac{dv}{dT}=v+v^{2}=v+Q(u,v). \end{cases}\displaystyle \end{aligned}
(19)

Based on the implicit function theorem, from $$\frac{dv}{dT}=0$$ we can deduce a unique function $$v=\phi (u)=0$$ satisfying $$\phi (0)=\phi '(0)=0$$ and $$\phi (u) + Q(u, \phi (u))=0$$. Then substituting it into the first equation of (19), we have that

\begin{aligned} \frac{du}{dT}=-d_{20}u^{2}-d_{30}u^{3}+Q_{3}(u), \end{aligned}

where $$Q_{3}(u)$$ is a power series with terms $$u^{i}$$ ($$i\geq 4$$).

Thus we can derive that the coefficient at the term $$u^{2}$$ is

\begin{aligned} -d_{20}=-\frac{\alpha _{2}\gamma ^{2}}{(\gamma +x_{31})^{3}}+1=- \frac{\gamma ^{2}}{(\gamma +x_{31})^{3}} \biggl( \frac{\beta }{x_{31}}- \frac{2x_{31}}{\gamma }-\frac{x_{31}^{2}}{\gamma ^{2}} \biggr)>0. \end{aligned}

Then, according to Theorem 7.1 in Chap. 2 in , we obtain that the equilibrium point $$P_{31}$$ is a saddle node. This means that a neighborhood $$S_{\varepsilon }(P_{31})$$ (ε is a sufficiently small radius) of $$P_{31}$$ is divided into two parts by two separatrices that tend to $$P_{31}$$ along the upside and the underneath of $$P_{31}$$. One part consists of two hyperbolic sectors, and the other part is a parabolic sector. Moreover, the parabolic sector is on the right half-plane because the coefficient at the term $$u^{2}$$ is greater than zero.

(2) If $$\alpha >\alpha _{2}$$, then we have $$f (\sqrt{\beta \gamma } )=\beta \gamma +(\beta +\gamma - \alpha )\sqrt{\beta \gamma }+\beta \gamma <0$$. Hence $$x_{32}<\sqrt{\beta \gamma }$$ and $$x_{33}>\sqrt{\beta \gamma }$$.

The Jacobian matrix of system (4) evaluated at the equilibrium $$P_{32}$$ is calculated as

\begin{aligned} J(P_{32})= \begin{pmatrix} H(x_{32},1)&-\beta \sqrt{\beta \gamma } \\ 0&-1 \end{pmatrix}. \end{aligned}
(20)

Then from $$\frac{\alpha x_{32}}{\gamma +x_{32}}=x_{32}+\beta$$ it follows that

\begin{aligned} H(x_{32},1)= \frac{2\alpha \gamma x_{32}+\alpha x_{32}^{2}}{(\gamma +x_{32})^{2}}-2 x_{32}-\beta = \frac{\beta \gamma -x_{32}^{2}}{\gamma +x_{32}}>0. \end{aligned}

Therefore we get that two characteristic roots of the matrix $$J(P_{32})$$ are $$\lambda _{1}(P_{32})=\frac{\beta \gamma -x_{32}^{2}}{\gamma +x_{32}}>0$$ and $$\lambda _{2}(P_{32})=-1<0$$, implying that $$P_{32}$$ is a hyperbolic saddle.

Similarly to the proof of the former $$P_{32}$$, we can deduce that the two eigenvalues of $$J(P_{33})$$ are $$\lambda _{1}(P_{33})=\frac{\beta \gamma -x_{33}^{2}}{\gamma +x_{33}}<0$$ and $$\lambda _{2}(P_{33})=-1<0$$, which means that $$P_{33}$$ is a hyperbolic stable node.

This completes the proof of Theorem 2.3. □

In Sect. 2.1, we have derived the parameter conditions for the existence of the coexistence equilibrium point $$P_{31}$$. We next show that system (4) experiences a saddle-node bifurcation at equilibrium $$P_{31}$$ as the parameter α passes through the bifurcation value $$\alpha =\alpha _{SN}= \beta +\gamma +2\sqrt{\beta \gamma }$$ by applying the Sotomayor theorem . For saddle-node bifurcation, we formulate the following results.

### Theorem 3.1

System (4) undergoes a saddle-node bifurcation at the equilibrium point$$P_{31}$$if$$\alpha =\alpha _{SN}= \beta +\gamma +2\sqrt{\beta \gamma }$$. Moreover, two equilibrium points bifurcate from$$P_{31}$$for$$\alpha >\alpha _{SN}$$, coalesce as the equilibrium point$$P_{31}$$for$$\alpha =\alpha _{SN}$$, and disappear for$$\alpha <\alpha _{SN}$$.

### Proof

Now we will prove the transversality condition for the occurrence of a saddle-node bifurcation at $$\alpha =\alpha _{SN}$$ by utilizing the Sotomayor theorem. From Sect. 2.1 we see that the two eigenvalues of $$J(P_{31})$$ are $$\lambda _{1}(P_{31})=0$$ and $$\lambda _{2}(P_{31})=-1<0$$. Denote by V and W the eigenvectors corresponding to the eigenvalue $$\lambda _{1}(P_{31})$$ for the matrices $$J (P_{31})$$ and $$J (P_{31})^{T}$$, respectively. Some computations yield

\begin{aligned} V= \begin{pmatrix} v_{1} \\ v_{2} \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \end{pmatrix};\qquad W= \begin{pmatrix} w_{1} \\ w_{2} \end{pmatrix} = \begin{pmatrix} 1 \\ -\beta \sqrt{\beta \gamma } \end{pmatrix}. \end{aligned}

Furthermore, we obtain

\begin{aligned} F_{\alpha }(P_{31};\alpha _{SN})= \begin{pmatrix} \frac{ x^{2}}{\gamma +x} \\ 0 \end{pmatrix}_{(P_{31};\alpha _{SN})} = \begin{pmatrix} \frac{x_{31}^{2}}{\gamma +x_{31}} \\ 0 \end{pmatrix} \end{aligned}
(21)

and

\begin{aligned} D^{2}F(P_{31};\alpha _{SN}) (V, V) =& \begin{pmatrix} \frac{\partial ^{2}F_{1}}{\partial x^{2}}v_{1}^{2}+2 \frac{\partial ^{2}F_{1}}{\partial x\,\partial y}v_{1}v_{2} + \frac{\partial ^{2}F_{1}}{\partial y^{2}}v_{2}^{2} \\ \frac{\partial ^{2}F_{2}}{\partial x^{2}}v_{1}^{2}+2 \frac{\partial ^{2}F_{2}}{\partial x\,\partial y}v_{1}v_{2} + \frac{\partial ^{2}F_{2}}{\partial y^{2}}v_{2}^{2} \end{pmatrix}_{(P_{31}; \alpha _{SN})} \\ =& \begin{pmatrix} \frac{2\alpha _{SN}\gamma ^{2}}{(\gamma +x_{31})^{3}}-2 \\ 0 \end{pmatrix}. \end{aligned}
(22)

Obviously, the vectors V and W satisfy

\begin{aligned} W^{T} F_{\alpha }(P_{31}; \alpha _{SN})= \frac{ x_{31}^{2}}{\gamma +x_{31}}\neq 0 \end{aligned}

and

\begin{aligned} W^{T} \bigl[D^{2} F(P_{31};\alpha _{SN}) (V, V) \bigr]= \frac{2\alpha _{SN}\gamma ^{2}}{(\gamma +x_{31})^{3}}-2 =- \frac{2\sqrt{\beta \gamma }}{\gamma +\sqrt{\beta \gamma }}\neq 0. \end{aligned}

Hence, by the Sotomayor theorem, when $$\alpha = \alpha _{SN}$$, system (4) undergoes a saddle-node bifurcation at nonhyperbolic critical point $$P_{31}$$. The number of positive equilibrium points of system (4) changes from zero to two as α passes from the left of $$\alpha = \alpha _{SN}$$ to the right. More concretely, there are no equilibrium points when $$\alpha <\alpha _{SN}$$, there is one equilibrium point $$P_{31}$$ when $$\alpha =\alpha _{SN}$$, and there are two equilibrium points $$P_{32}$$ and $$P_{33}$$ when $$\alpha >\alpha _{SN}$$. Thus the proof of Theorem 3.1 is completed. □

## Numerical simulations

In this section, by numerical simulations we give some phase portraits of the amensalism model (4) to further illustrate the previous theoretical analysis. Without loss of generality, we consider the following example.

### Example 4.1

Consider the system

\begin{aligned} \textstyle\begin{cases} \frac{dx}{dt}=x (\frac{\alpha x}{0.2+x}-x-0.8 y ), \\ \frac{dy}{dt}=y(1-y). \end{cases}\displaystyle \end{aligned}
(23)

In system (4), we give specific parameter values $$\beta =0.8$$ and $$\gamma =0.2$$. Then we get system (23). By some calculations we have $$\alpha _{1}=\beta +\gamma -2\sqrt{\beta \gamma } =0.2$$ and $$\alpha _{2}=\beta +\gamma +2\sqrt{\beta \gamma }=1.8$$. According to Theorems 2.1 and 2.2, for system (4), there exist two boundary equilibrium points $$P_{0}(0, 0)$$ and $$P_{1}(0, 1)$$ for all positive parameters; $$P_{0}(0, 0)$$ is a saddle node, and $$P_{1}(0, 1)$$ is a stable node. Next, we consider the following four cases by choosing various parameters α.

Case 1. Fix $$\alpha =0.12$$. Then for system (23), there is no other equilibrium point, as shown in Fig. 1(a). Furthermore, we choose the initial condition $$(x(0),y(0)) = (1,0.6)$$. From Fig. 3(a) we can see that as t increases, the trajectories tend to $$P_{1}$$, which is a stable node.

Case 2. Fix $$\alpha = 1.2$$. Then for system (23), there is a boundary equilibrium point $$P_{2}=(1,0)$$ that is a saddle depicted in Fig. 1(b). Furthermore, we take the initial value $$(x(0),y(0)) = (0.3,0.2)$$, and from Fig. 3(b) it follows that the orbits converge to the stable node $$P_{1}$$ as t increases.

Case 3. Fix $$\alpha =1.8$$. Then system (23) has one boundary equilibrium point $$P_{2}=(1.6, 0)$$, which is a saddle, and one coexistence equilibrium point $$P_{31}=(0.4,1)$$. From Theorem 2.3(1) we know that $$P_{31}$$ is a saddle node, and the above dynamical behaviors are shown in Fig. 1(c). From Theorem 3.1 we have that system (4) undergoes a saddle-node bifurcation at the equilibrium $$P_{31}$$ for $$\alpha =\alpha _{SN}=1.8$$; the corresponding illustrative bifurcation diagram is depicted in Fig. 2. Moreover, we take the initial value $$(x(0),y(0)) = (1,0.6)$$, it follows from Fig. 3(c) that the trajectories approach the saddle node $$P_{31}$$ as t increases.

Case 4. Fix $$\alpha = 2.4$$. Then system (23) has one boundary equilibrium point $$P_{2}=(2.2, 0)$$, which is a saddle, and two positive equilibrium points $$P_{32}\approx (0.1255, 1)$$ and $$P_{33}\approx (1.2745, 1)$$. According to Theorem 2.3(2), we have that $$P_{32}$$ is a saddle, $$P_{33}$$ is a stable node, as displayed in Fig. 1(d). Moreover, we choose the initial value $$(x(0), y(0)) = (0.3, 0.2)$$. From Fig. 3(d) we can observe that the orbits tend to the stable node $$P_{33}$$ as t increases.

Figs. 3(a)–(d) display the trajectories of system (4) corresponding to Figs. 1(a)–(d) with the different initial values.

## Conclusions

In this paper, we proposed a new amensalism system with Allee effect on the first species and studied the stability and bifurcation of this model. We discussed the distribution and stability of all boundary equilibrium points and the coexistence equilibrium points. The dynamical properties of the amensalism model become complex when the amensalism system subjects to the Allee effect. By comparing with system (1), we can see the system (4) possesses some new dynamical phenomena as follows.

1. (i)

There exist at most five equilibrium points for system (4), including two distinct interior points, whereas for system (1), there are at most four equilibrium points, including one interior point.

2. (ii)

As the number of equilibrium points increases, the dynamical properties of the system (4) become more complicated; for example, system (4) undergoes a saddle-node bifurcation at equilibrium point $$P_{31}$$.

3. (iii)

The stability of coexistence equilibrium points is more complex. In  the coexistence equilibrium point (if it exists) of system (1) is globally stable, whereas for system (4), the equilibrium point $$P_{31}$$ is a saddle node when $$\alpha = \alpha _{2}$$.

## References

1. Sun, G.: Qualitative analysis on two populations amensalism model. J. Jiamusi Univ. (Natl. Sci. Ed.) 21(3), 283–286 (2003)

2. Zhang, Z.: Stability and bifurcation analysis for a amensalism system with delays. Math. Numer. Sin. 30(2), 213–224 (2008)

3. Zhang, J.: Bifurcated periodic solutions in an amensalism system with strong generic delay kernel. Math. Methods Appl. Sci. 36, 113–124 (2013)

4. Chen, F., Zhang, M., Han, R.: Existence of positive periodic solution of a discrete Lotka–Volterra amensalism model. J. Shengyang Univ. (Natl. Sci.) 27(3), 251–254 (2015)

5. Wu, R., Zhao, L., Lin, Q.: Stability analysis of a two species amensalism model with Holling II functional response and a cover for the first species. J. Nonlinear Funct. Anal. 2016(46), 1–15 (2016)

6. Wu, R.: A two species amensalism model with non-monotonic functional response. Commun. Math. Biol. Neurosci. 2016(19), 1–10 (2016)

7. Xie, X., Chen, F., He, M.: Dynamic behaviors of two species amensalism model with a cover for the first species. J. Math. Comput. Sci. 16, 395–401 (2016)

8. Lin, Q., Zhou, X.: On the existence of positive periodic solution of a amensalism model with Holling II functional response. Commun. Math. Biol. Neurosci. 2017(3), 1–12 (2017)

9. Liu, Y., Zhao, L., Huang, X.Y., Deng, H.: Stability and bifurcation analysis of two species amensalism model with Michaelis–Menten type harvesting and a cover for the first species. Adv. Differ. Equ. 2018, 295, 1–19 (2018)

10. Chen, B.: Dynamic behaviors of a non-selective harvesting Lotka–Volterra amensalism model incorporating partial closure for the populations. Adv. Differ. Equ. 2018, 111, 1–14 (2018)

11. Guan, X.Y., Chen, F.D.: Dynamical analysis of a two species amensalism model with Beddington–DeAngelis functional response and Allee effect on the second species. Nonlinear Anal., Real World Appl. 48, 71–93 (2019)

12. Wei, Z., Xia, Y.H., Zhang, T.H.: Stability and bifurcation analysis of an Amensalism model with weak Allee effect. Qual. Theory Dyn. Syst. 19, 23, 1–15 (2020)

13. Murray, J.D.: Mathematical Biology. Springer, New York (1993)

14. Schreiber, S.: Allee effects, extinctions, and chaotic transients in simple population models. Theor. Popul. Biol. 64, 201–209 (2003)

15. Courchamp, F., Berec, L., Gascoigne, J.: Allee Effects in Ecology and Conservation. Oxford University Press, London (2008)

16. Mccarthy, M.A.: The Allee effect, finding mates and theoretical models. Ecol. Model. 103(1), 99–102 (1997)

17. Allee, W.C.: Animal Aggregations: A Study in General Sociology. University of Chicago Press, Chicago (1931)

18. Allee, W.C.: The Social Life of Animals. William Heinemann, London (1938)

19. Dennis, B.: Allee effect: population growth, critical density, and chance of extinction. Nat. Resour. Model. 3, 481–538 (1989)

20. Wang, M.H., Kot, M.: Speeds of invasion in a model with strong or weak Allee effects. Math. Biosci. 171(1), 83–97 (2001)

21. Scheuring, I.: Allee effect increases the dynamics stability of populations. J. Theor. Biol. 199, 407–414 (1999)

22. Sen, M., Banerjee, M.: Bifurcation analysis of a radio-dependent prey–predator model with the Allee effect. Ecol. Complex. 11, 12–27 (2012)

23. Zhao, M., Du, Y.: Stability of a discrete-time predator–prey system with Allee effect. Nonlinear Anal. Differ. Equ. 4(5), 225–233 (2016)

24. Kangalgil, F.: The local stability analysis of a nonlinear discrete-time population model with delay and Allee effect. Cumhuriyet Sci. J. 38(3), 480–487 (2017)

25. Hu, D., Cao, H.: Stability and bifurcation analysis in a predator–prey system with Michaelis–Menten type predator harvesting. Nonlinear Anal., Real World Appl. 33, 58–82 (2017)

26. Eduardo, G.O., Alejandro, R.P., Betsab, G.Y.: Multiple limit cycles in a Leslie–Gower-type predator–prey model considering weak Allee effect on prey. Nonlinear Anal., Model. Control 2017, 347–365 (2017)

27. Zhang, Z., Ding, T., Huang, W., Dong, Z.: Qualitative Theory of Differential Equation. Science Press, Beijing (1997)

28. Perko, L.: Differential Equations and Dynamical Systems. Springer, NewYork (2001)

### Acknowledgements

The authors would like to thank the reviewers and the editor for very helpful suggestions and comments, which led to improvements of our original paper.

Not applicable.

## Funding

The first author was supported by the Fundamental Research Funds for the Central Universities (No. 2652018055), the China Scholarship Council (No. 201906405015), and the National Natural Science Foundation of China (No. 11771033). The second author was supported by the Initial Funding for the Doctoral Program of BIGC (No. 27170119000/001).

## Author information

Authors

### Contributions

Both authors contributed equally to the writing of this paper. Both authors read and approved the final manuscript.

### Corresponding author

Correspondence to Ming Zhao.

## Ethics declarations

### Competing interests

The authors declare that they have no competing interests.

## Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions 