 Research
 Open Access
 Published:
Dynamics of a twodimensional competitive system of rational difference equations with quadratic terms
Advances in Difference Equations volume 2014, Article number: 301 (2014)
Abstract
We investigate global dynamics of the following systems of difference equations:
where the parameters ${b}_{1}$, ${a}_{2}$, ${A}_{1}$, ${c}_{2}$ are positive numbers and the initial condition ${y}_{0}$ is an arbitrary nonnegative number and ${x}_{0}$ is a positive number. We show that this system has rich dynamics which depends on the part of a parametric space. We find precisely the basins of attraction of all attractors including the points at ∞.
MSC:39A10, 39A30, 37E99, 37D10.
1 Introduction
In this paper we study the global dynamics of the following rational system of difference equations:
where the parameters ${b}_{1}$, ${a}_{2}$, ${A}_{1}$, ${c}_{2}$ are positive numbers and the initial condition ${y}_{0}$ is an arbitrary nonnegative number and ${x}_{0}$ is a positive number.
The related system of difference equations
where the parameters ${A}_{1}$, ${\beta}_{1}$, ${\alpha}_{2}$ and ${\gamma}_{2}$ are positive numbers and the initial conditions ${x}_{0}>0$, ${y}_{0}\ge 0$, was considered in [1], where it was shown that this system has simple dynamics. Precisely, it was shown that system (2) has no equilibrium points if ${\beta}_{1}\le {A}_{1}$ and that it has a unique equilibrium point if ${\beta}_{1}>{A}_{1}$, in which case this equilibrium point is a saddle point. Furthermore, the following result describes the global dynamics of system (2).
Theorem 1 Consider system (2).
(1) Assume that ${\beta}_{1}>{A}_{1}$ and ${\gamma}_{2}{A}_{1}\ne {\alpha}_{2}$. Then there exist a set $\mathcal{C}\subset \mathcal{R}$ which is invariant and a subset of the basin of attraction of E. The set $\mathcal{C}$ is a graph of a strictly increasing continuous function of the first variable on an interval (and so is a manifold) and separates ℛ into two connected and invariant components, namely
which satisfy:
and
(2) Assume that ${\gamma}_{2}{A}_{1}={\alpha}_{2}$. Then system (2) can be decoupled as follows:
and every solution of this system (depending of the choice of the initial condition $({x}_{0},{y}_{0})$) is either bounded and converges to an equilibrium point, or increases monotonically to infinity.
(3) Assume that ${\beta}_{1}\le {A}_{1}$ and ${\gamma}_{2}{A}_{1}\ne {\alpha}_{2}$. Every solution $\{({x}_{n},{y}_{n})\}$ of system (2), with ${x}_{0}>0$, ${y}_{0}\ge 0$, satisfies
Thus every solution of system (2) either converges to the unique equilibrium point or is asymptotic to one of the points at infinity, precisely to either $(0,\mathrm{\infty})$ or to $(\mathrm{\infty},0)$. In all cases, either solution is eventually monotonic or the subsequences of even indexed and odd indexed terms are eventually monotonic. Introduction of quadratic terms into the system will substantially change the dynamics by introducing new equilibrium points (up to three) with different local character and minimal periodtwo solutions (up to 11). Again, most of the solutions of system (1) will be asymptotic to $(\mathrm{\infty},0)$ or $(0,\mathrm{\infty})$, but the separatrix between the two basins of attraction may consist of several global stable manifolds of either saddle point equilibrium points or nonhyperbolic equilibrium points or minimal periodtwo solutions. In one case, when there exists a unique nonhyperbolic equilibrium point, it is possible that this point will have a basin of attraction of positive Lebesgue measure.
System (1) is a competitive system, and our results are based on recent results about competitive systems in the plane, see [2, 3]. System (1) can be used as a mathematical model for competition in population dynamics. The first systematic study of a specific competitive system with quadratic terms was performed in [4] where system of the form
where the parameters $a,b>0$ and the initial conditions ${x}_{0},{y}_{0}\ge 0$, was considered. It was shown that the dynamics of system (3) is very similar to the dynamics of the corresponding linear fractional system
with the same conditions on parameters and initial conditions. Both systems have nine parametric regions with different dynamical behavior.
As noted, the introduction of quadratic terms in system (2) dramatically changes the dynamics. The techniques used to study system (2) were straightforward calculations, while the techniques which will be used to study system (1) are a combination of techniques for studying real algebraic curves and implicit function theorem as neither equilibrium points nor periodtwo solutions are explicitly computable. Some of our calculations are performed by using Mathematica and outputs are included in the Appendix.
The paper is organized as follows. Section 2 contains some necessary results on competitive systems in the plane. Section 3 provides some basic facts about the equilibrium points and injectivity of the map associated with system (1). Section 4 contains local stability analysis of both equilibrium solutions and minimal periodtwo solutions. Section 5 gives global dynamics in different cases.
2 Preliminaries
A firstorder system of difference equations
where $\mathcal{S}\subset {\mathbb{R}}^{2}$, $(f,g):\mathcal{S}\to \mathcal{S}$, f, g are continuous functions, is competitive if $f(x,y)$ is nondecreasing in x and nonincreasing in y, and $g(x,y)$ is nonincreasing in x and nondecreasing in y. If both f and g are nondecreasing in x and y, system (4) is cooperative. Competitive and cooperative maps are defined similarly. Strongly competitive systems of difference equations or strongly competitive maps are those for which the functions f and g are coordinatewise strictly monotone. Competitive and cooperative systems have been investigated by many authors, see [2, 3, 5–14]. Special attention to discrete competitive and cooperative systems in the plane was given in [2, 3, 5, 6, 9, 10, 14–23]. One of the reasons for paying special attention to twodimensional discrete competitive and cooperative systems is their applicability to mathematical models in biology and economics, the former involves competition or cooperation between two species. Another reason is that the theory of twodimensional discrete competitive and cooperative systems is very well developed, unlike such theory for three and higher dimensional systems. Part of the reason for this situation is de MottoniSchiaffino theorem given below, which provides relatively simple scenarios for possible behavior of many twodimensional discrete competitive and cooperative systems. However, this does not mean that one cannot encounter chaos in such systems as has been shown by Smith, see [14].
If $\mathbf{v}=(u,v)\in {\mathbb{R}}^{2}$, we denote by ${\mathcal{Q}}_{\ell}(\mathbf{v})$, $\ell \in \{1,2,3,4\}$ the four quadrants in ${\mathbb{R}}^{2}$ relative to v, i.e., ${\mathcal{Q}}_{1}(\mathbf{v})=\{(x,y)\in {\mathbb{R}}^{2}:x\ge u,y\ge v\}$, ${\mathcal{Q}}_{2}(\mathbf{v})=\{(x,y)\in {\mathbb{R}}^{2}:x\le u,y\ge v\}$, and so on. Define the southeast partial order ${\u2aaf}_{se}$ on ${\mathbb{R}}^{2}$ by $(x,y){\u2aaf}_{se}(s,t)$ if and only if $x\le s$ and $y\ge t$. Similarly, we define the northeast partial order ${\u2aaf}_{ne}$ on ${\mathbb{R}}^{2}$ by $(x,y){\u2aaf}_{ne}(s,t)$ if and only if $x\le s$ and $y\le t$. For $\mathcal{A}\subset {\mathbb{R}}^{2}$ and $\mathrm{x}\in {\mathbb{R}}^{2}$, define the distance from x to $\mathcal{A}$ as $dist(\mathrm{x},\mathcal{A}):=inf\{\parallel \mathrm{x}\mathrm{y}\parallel :\mathrm{y}\in \mathcal{A}\}$. By $int\mathcal{A}$ we denote the interior of a set $\mathcal{A}$.
It is easy to show that a map F is competitive if it is nondecreasing with respect to the southeast partial order, that is, if the following holds:
For standard definitions of attracting fixed point, saddle point, stable manifold, and related notions, see [8, 24].
We now state three results for competitive maps in the plane. The following definition is from [14].
Definition 1 Let $\mathcal{S}$ be a nonempty subset of ${\mathbb{R}}^{2}$. A competitive map $T:\mathcal{S}\to \mathcal{S}$ is said to satisfy condition $(O+)$ if for every x, y in $\mathcal{S}$, $T(x){\u2aaf}_{ne}T(y)$ implies $x{\u2aaf}_{ne}y$, and T is said to satisfy condition $(O)$ if for every x, y in $\mathcal{S}$, $T(x){\u2aaf}_{ne}T(y)$ implies $y{\u2aaf}_{ne}x$.
The following theorem was proved by de Mottoni and Schiaffino [19] for the Poincaré map of a periodic competitive LotkaVolterra system of differential equations. Smith generalized the proof to competitive and cooperative maps [12].
Theorem 2 Let $\mathcal{S}$ be a nonempty subset of ${\mathbb{R}}^{2}$. If T is a competitive map for which $(O+)$ holds, then, for all $x\in \mathcal{S}$, $\{{T}^{n}(x)\}$ is eventually componentwise monotone. If the orbit of x has compact closure, then it converges to a fixed point of T. If instead $(O)$ holds, then, for all $x\in \mathcal{S}$, $\{{T}^{2n}(x)\}$ is eventually componentwise monotone. If the orbit of x has compact closure in $\mathcal{S}$, then its omega limit set is either a periodtwo orbit or a fixed point.
The following result is from [14], with the domain of the map specialized to be the cartesian product of intervals of real numbers. It gives a sufficient condition for conditions $(O+)$ and $(O)$.
Theorem 3 Let $\mathcal{R}\subset {\mathbb{R}}^{2}$ be the cartesian product of two intervals in ℝ. Let $T:\mathcal{R}\to \mathcal{R}$ be a ${C}^{1}$ competitive map. If T is injective and $det{J}_{T}(x)>0$ for all $x\in \mathcal{R}$, then T satisfies $(O+)$. If T is injective and $det{J}_{T}(x)<0$ for all $x\in \mathcal{R}$, then T satisfies $(O)$.
The following result is a direct consequence of the trichotomy theorem of Dancer and Hess, see [2] and [25], and is helpful for determining the basins of attraction of the equilibrium points.
Corollary 1 If the nonnegative cone of ⪯ is a generalized quadrant in ${\mathbb{R}}^{n}$, and if T has no fixed points in $\u301a{u}_{1},{u}_{2}\u301b$ other than ${u}_{1}$ and ${u}_{2}$, then the interior of $\u301a{u}_{1},{u}_{2}\u301b$ is either a subset of the basin of attraction of ${u}_{1}$ or a subset of the basin of attraction of ${u}_{2}$.
The next result is a wellknown global attractivity result which holds in partially ordered Banach spaces as well, see [25].
Theorem 4 Let T be a monotone map on a closed and bounded rectangular region $\mathcal{R}\subset {\mathbb{R}}^{2}$. Suppose that T has a unique fixed point $\overline{\mathbf{e}}$ in ℛ. Then $\overline{\mathbf{e}}$ is a global attractor of T on ℛ.
The following theorems were proved by Kulenović and Merino [3] for competitive systems in the plane, when one of the eigenvalues of the linearized system at an equilibrium (hyperbolic or nonhyperbolic) is by an absolute value smaller than 1, while the other has an arbitrary value. These results are useful for determining basins of attraction of fixed points of competitive maps.
Theorem 5 Let T be a competitive map on a rectangular region $\mathcal{R}\subset {\mathbb{R}}^{2}$. Let $\overline{\mathrm{x}}\in \mathcal{R}$ be a fixed point of T such that $\mathrm{\Delta}:=\mathcal{R}\cap int({\mathcal{Q}}_{1}(\overline{x})\cup {\mathcal{Q}}_{3}(\overline{\mathrm{x}}))$ is nonempty (i.e., $\overline{\mathrm{x}}$ is not the NW or SE vertex of ℛ), and T is strongly competitive on Δ. Suppose that the following statements are true.

(a)
The map T has a ${C}^{1}$ extension to a neighborhood of $\overline{\mathrm{x}}$.

(b)
The Jacobian ${J}_{T}(\overline{\mathrm{x}})$ of T at $\overline{\mathrm{x}}$ has real eigenvalues λ, μ such that $0<\lambda <\mu $, where $\lambda <1$, and the eigenspace ${E}^{\lambda}$ associated with λ is not a coordinate axis.
Then there exists a curve $\mathcal{C}\subset \mathcal{R}$ through $\overline{\mathrm{x}}$ that is invariant and a subset of the basin of attraction of $\overline{\mathrm{x}}$ such that $\mathcal{C}$ is tangential to the eigenspace ${E}^{\lambda}$ at $\overline{\mathrm{x}}$, and $\mathcal{C}$ is the graph of a strictly increasing continuous function of the first coordinate on an interval. Any endpoints of $\mathcal{C}$ in the interior of ℛ are either fixed points or minimal periodtwo points. In the latter case, the set of endpoints of $\mathcal{C}$ is a minimal periodtwo orbit of T.
The situation where the endpoints of $\mathcal{C}$ are boundary points of ℛ is of interest. The following result gives a sufficient condition for this case.
Theorem 6 For the curve $\mathcal{C}$ of Theorem 5 to have endpoints in $\partial \mathcal{R}$, it is sufficient that at least one of the following conditions is satisfied.

(i)
The map T has no fixed points nor periodic points of minimal period two in Δ.

(ii)
The map T has no fixed points in Δ, $det{J}_{T}(\overline{\mathrm{x}})>0$, and $T(\mathrm{x})=\overline{\mathrm{x}}$ has no solutions $\mathrm{x}\in \mathrm{\Delta}$.

(iii)
The map T has no points of minimal period two in Δ, $det{J}_{T}(\overline{\mathrm{x}})<0$, and $T(\mathrm{x})=\overline{\mathrm{x}}$ has no solutions $\mathrm{x}\in \mathrm{\Delta}$.
The next result is useful for determining basins of attraction of fixed points of competitive maps.
Theorem 7 (A) Assume the hypotheses of Theorem 5, and let $\mathcal{C}$ be the curve whose existence is guaranteed by Theorem 5. If the endpoints of $\mathcal{C}$ belong to $\partial \mathcal{R}$, then $\mathcal{C}$ separates ℛ into two connected components, namely
such that the following statements are true.

(i)
${\mathcal{W}}_{}$ is invariant, and $dist({T}^{n}(\mathrm{x}),{\mathcal{Q}}_{2}(\overline{\mathrm{x}}))\to 0$ as $n\to \mathrm{\infty}$ for every $\mathrm{x}\in {\mathcal{W}}_{}$.

(ii)
${\mathcal{W}}_{+}$ is invariant, and $dist({T}^{n}(\mathrm{x}),{\mathcal{Q}}_{4}(\overline{\mathrm{x}}))\to 0$ as $n\to \mathrm{\infty}$ for every $\mathrm{x}\in {\mathcal{W}}_{+}$.

(B)
If, in addition to the hypotheses of part (A), $\overline{\mathrm{x}}$ is an interior point of ℛ and T is ${C}^{2}$ and strongly competitive in a neighborhood of $\overline{\mathrm{x}}$, then T has no periodic points in the boundary of ${\mathcal{Q}}_{1}(\overline{\mathrm{x}})\cup {\mathcal{Q}}_{3}(\overline{\mathrm{x}})$ except for $\overline{\mathrm{x}}$, and the following statements are true.

(iii)
For every $\mathrm{x}\in {\mathcal{W}}_{}$, there exists ${n}_{0}\in \mathbb{N}$ such that ${T}^{n}(\mathrm{x})\in int{\mathcal{Q}}_{2}(\overline{\mathrm{x}})$ for $n\ge {n}_{0}$.

(iv)
For every $\mathrm{x}\in {\mathcal{W}}_{+}$, there exists ${n}_{0}\in \mathbb{N}$ such that ${T}^{n}(\mathrm{x})\in int{\mathcal{Q}}_{4}(\overline{\mathrm{x}})$ for $n\ge {n}_{0}$.
If T is a map on a set ℛ and if $\overline{\mathrm{x}}$ is a fixed point of T, the stable set ${\mathcal{W}}^{s}(\overline{\mathrm{x}})$ of $\overline{\mathrm{x}}$ is the set $\{x\in \mathcal{R}:{T}^{n}(\mathrm{x})\to \overline{\mathrm{x}}\}$ and unstable set ${\mathcal{W}}^{u}(\overline{\mathrm{x}})$ of $\overline{\mathrm{x}}$ is the set
When T is noninvertible, the set ${\mathcal{W}}^{s}(\overline{\mathrm{x}})$ may not be connected and made up of infinitely many curves, or ${\mathcal{W}}^{u}(\overline{\mathrm{x}})$ may not be a manifold. The following result gives a description of the stable and unstable sets of a saddle point of a competitive map. If the map is a diffeomorphism on ℛ, the sets ${\mathcal{W}}^{s}(\overline{\mathrm{x}})$ and ${\mathcal{W}}^{u}(\overline{\mathrm{x}})$ are the stable and unstable manifolds of $\overline{x}$.
Theorem 8 In addition to the hypotheses of part (B) of Theorem 7, suppose that $\mu >1$ and that the eigenspace ${E}^{\mu}$ associated with μ is not a coordinate axis. If the curve $\mathcal{C}$ of Theorem 5 has endpoints in $\partial \mathcal{R}$, then $\mathcal{C}$ is the stable set ${\mathcal{W}}^{s}(\overline{\mathrm{x}})$ of $\overline{\mathrm{x}}$, and the unstable set ${\mathcal{W}}^{u}(\overline{\mathrm{x}})$ of $\overline{x}$ is a curve in ℛ that is tangential to ${E}^{\mu}$ at $\overline{\mathrm{x}}$ and such that it is the graph of a strictly decreasing function of the first coordinate on an interval. Any endpoints of ${\mathcal{W}}^{u}(\overline{\mathrm{x}})$ in ℛ are fixed points of T.
The following result gives information on local dynamics near a fixed point of a map when there exists a characteristic vector whose coordinates have negative product and such that the associated eigenvalue is hyperbolic. This is a wellknown result, valid in a much more general setting: we include it here for completeness. A point $(x,y)$ is a subsolution if $T(x,y){\u2aaf}_{se}(x,y)$, and $(x,y)$ is a supersolution if $(x,y){\u2aaf}_{se}T(x,y)$. An order interval $\u301a(a,b),(c,d)\u301b$ is the cartesian product of the two compact intervals $[a,c]$ and $[b,d]$.
Theorem 9 Let T be a competitive map on a rectangular set $\mathcal{R}\subset {\mathbb{R}}^{2}$ with an isolated fixed point $\overline{\mathrm{x}}\in \mathcal{R}$ such that $\mathcal{R}\cap int({\mathcal{Q}}_{2}(\overline{\mathrm{x}})\cup {\mathcal{Q}}_{4}(\overline{\mathrm{x}}))\ne \mathrm{\varnothing}$. Suppose that T has a ${C}^{1}$ extension to a neighborhood of $\overline{\mathrm{x}}$. Let $\mathrm{v}=({\mathrm{v}}^{(1)},{\mathrm{v}}^{(2)})\in {\mathbb{R}}^{2}$ be an eigenvector of the Jacobian of T at $\overline{\mathrm{x}}$, with associated eigenvalue $\mu \in \mathbb{R}$. If ${\mathrm{v}}^{(1)}{\mathrm{v}}^{(2)}<0$, then there exists an order interval ℐ which is also a relative neighborhood of $\overline{\mathrm{x}}$ such that for every relative neighborhood $\mathcal{U}\subset \mathcal{I}$ of $\overline{\mathrm{x}}$ the following statements are true.

(i)
If $\mu >1$, then $\mathcal{U}\cap int{\mathcal{Q}}_{2}(\overline{\mathrm{x}})$ contains a subsolution and $\mathcal{U}\cap int{\mathcal{Q}}_{4}(\overline{\mathrm{x}})$ contains a supersolution. In this case, for every $\mathrm{x}\in \mathcal{I}\cap int({\mathcal{Q}}_{2}(\overline{\mathrm{x}})\cup {\mathcal{Q}}_{4}(\overline{\mathrm{x}}))$, there exists N such that ${T}^{n}(\mathrm{x})\notin \mathcal{I}$ for $n\ge N$.

(ii)
If $\mu <1$, then $\mathcal{U}\cap int{\mathcal{Q}}_{2}(\overline{\mathrm{x}})$ contains a supersolution and $\mathcal{U}\cap int{\mathcal{Q}}_{4}(\overline{\mathrm{x}})$ contains a subsolution. In this case ${T}^{n}(\mathrm{x})\to \overline{\mathrm{x}}$ for every $\mathrm{x}\in \mathcal{I}$.
3 Some basic facts
In this section we give some basic facts which will be used later. The map T associated to system (1) is given by
Let
3.1 Equilibrium points
The equilibrium points $(\overline{x},\overline{y})$ of system (1) satisfy the equations
By eliminating $\overline{x}\ne 0$ from (6), we get
Similarly, we can eliminate variable $\overline{y}$ from system (6) to obtain
Lemma 1 Let
and
Then the following statements hold:

(a)
Consider equation (7). Then all its real roots are positive numbers. Furthermore, equation (7) has one, two, or three real roots.

(b)
If ${\mathrm{\Delta}}_{1}>0$, then equation (7) has one real root and two pairs of distinct conjugate imaginary roots.

(c)
If ${\mathrm{\Delta}}_{1}<0$, then equation (7) has three distinct real roots and one pair of conjugate imaginary roots.

(d)
If ${\mathrm{\Delta}}_{1}=0$ and ${\mathrm{\Delta}}_{2}\ne 0$, then equation (7) has one pair of conjugate imaginary roots and two real roots, one real root of multiplicity one and other one of multiplicity two.

(e)
If ${\mathrm{\Delta}}_{1}=0$ and ${\mathrm{\Delta}}_{2}=0$, then equation (7) has one pair of conjugate imaginary roots and one real root of multiplicity three.
Proof The proof of (a) follows from Descartes’ rule of signs.
Let
The following matrix, called the discrimination matrix of $\tilde{f}(y)$ and ${\tilde{f}}^{\prime}(y)$ in [26], is actually the Sylvester matrix of $\tilde{f}(y)$ and ${\tilde{f}}^{\prime}(y)$ with some permuted rows.
Let ${D}_{k}$ denote the determinant of the submatrix of $Discr(\tilde{f})$, formed by the first 2k row and the first 2k columns, for $k=1,\dots ,m$. So, by a straightforward calculation, one can see that
Assume that ${D}_{5}>0$. The sign list of the sequence $\{{D}_{1},{D}_{2},{D}_{3},{D}_{4},{D}_{5}\}$ is given by
from which it follows that the number of sign changes of the revised sign list of list (9) is two. Now, statement (b) follows in view of Theorem 1 [26]. Assume that ${D}_{5}<0$. If ${D}_{4}\ge 0$, then we obtain that $\tilde{f}(y)$ has three pairs of conjugate imaginary roots, which is a contradiction. Hence, ${D}_{4}<0$. The sign list of the sequence $\{{D}_{1},{D}_{2},{D}_{3},{D}_{4},{D}_{5}\}$ is given by
which implies that the number of sign changes of the revised sign list of (10) is one. Now, statement (c) follows in view of Theorem 1 [26]. Similarly, one can prove statements (d) and (e). □
3.2 Injectivity, $(O+)$ and $(O)$
Lemma 2 Assume that $(\overline{x},\overline{y})$ is an equilibrium of the map T. Then the following hold:

(1)
If ${a}_{2}\ne {A}_{1}{c}_{2}$, then T is injective.

(2)
If ${a}_{2}={A}_{1}{c}_{2}$, then the curve
$${b}_{1}{x}^{2}=\overline{x}({A}_{1}+{y}^{2})$$is invariant under the map T. Furthermore, the following holds:
$$T(x,\sqrt{\frac{{b}_{1}{x}^{2}{A}_{1}\overline{x}}{\overline{x}}})=(\overline{x},\overline{y})\phantom{\rule{1em}{0ex}}\mathit{\text{for}}x\ge \sqrt{\frac{{A}_{1}\overline{x}}{{b}_{1}}}.$$ 
(3)
If ${A}_{1}{c}_{2}>{a}_{2}$, then T satisfies $(O+)$, in which case $\{{T}^{n}({x}_{0},{y}_{0})\}$ is asymptotic to either $(0,\mathrm{\infty})$ or $(\mathrm{\infty},0)$, or to an equilibrium point, for all $({x}_{0},{y}_{0})\in \mathcal{R}$.

(4)
If ${A}_{1}{c}_{2}<{a}_{2}$, then T satisfies $(O)$, in which case $\{{T}^{n}({x}_{0},{y}_{0})\}$ is asymptotic to either $(0,\mathrm{\infty})$ or $(\mathrm{\infty},0)$, or to a periodtwo point, for all $({x}_{0},{y}_{0})\in \mathcal{R}$.
Proof (1) Assume that $T({x}_{1},{y}_{1})=T({x}_{2},{y}_{2})$. Then we have
Equation (11) is equivalent to
Equation (12) implies
By substituting this into equation (13), we obtain
from which it follows that ${y}_{1}={y}_{2}$ since ${a}_{2}\ne {A}_{1}{c}_{2}$. From (14) we have ${x}_{1}={x}_{2}$, which completes the proof of statement (a).
(2)One can see that
Since ${a}_{2}={A}_{1}{c}_{2}$, equations (7) and (8) become
and
From (15) we have ${A}_{1}=\frac{{b}_{1}^{2}{c}_{2}{\overline{y}}^{3}}{\overline{y}}$. By substituting this into (16) we get
which implies $\overline{x}\overline{y}{b}_{1}{c}_{2}=0$, from which the proof follows.
(3) The Jacobian matrix of the map T has the form
The determinant of (17) at any point is equal to
The proof of (3) and (4) follows from Theorem 3. □
4 Linearized stability analysis
The determinant of (17) at the equilibrium point is given by
The trace of (17) at the equilibrium point is given by
The characteristic equation has the form
Equilibrium curves ${\mathcal{C}}_{f}=\{(x,y)\in \mathcal{R}:f(x,y)=x\}$ and ${\mathcal{C}}_{g}=\{(x,y)\in \mathcal{R}:g(x,y)=y\}$ can be given explicitly as functions of y:
Note that ${x}_{{g}^{}}(x)$ is always negative. Let ${x}_{g}(y)$ denote ${x}_{{g}^{+}}(y)$. We consider only ${x}_{f}(y)$ and ${x}_{g}(y)$. Let
Lemma 3 Let $T=(f,g)$ be the map defined by (5). Then ${f}_{x}^{\prime}(x,y)>1$, and the following is true:
Proof The first derivative of ${x}_{f}(y)$ is given by
Since ${f}_{y}^{\prime}(x,y)<0$, we get ${f}_{x}^{\prime}(x,y)>1$. Further,
Now, the proof follows from
□
Lemma 4 Let T be the map defined by (5), and let
be the Jacobian matrix of T at a fixed point $(\overline{x},\overline{y})$. Then the Jacobian matrix (19) has real and distinct eigenvalues ${\lambda}_{1}$ and ${\lambda}_{2}$ such that ${\lambda}_{1}<{\lambda}_{2}$ and ${\lambda}_{2}>1$. Furthermore, the following holds:
Proof Implicit differentiation of the equations defining ${C}_{f}$ and ${C}_{g}$ at $(\overline{x},\overline{y})$ gives
The characteristic equation associated with the Jacobian matrix of T at $(\overline{x},\overline{y})$ is given by
Since the map T is competitive, then the eigenvalues of the Jacobian matrix of the map T at the equilibrium $(\overline{x},\overline{y})$ are real and distinct and ${\lambda}_{1}<{\lambda}_{2}$. By (20), we have
From $tr{J}_{T}(\overline{x},\overline{y})={\lambda}_{1}+{\lambda}_{2}>2$ we get ${\lambda}_{2}>1$. The map T is competitive, which implies $c={g}_{x}^{\prime}(\overline{x},\overline{y})<0$. In view of Lemma 3, we get $a={f}_{x}^{\prime}(\overline{x},\overline{y})>1$, from which it follows that $sign({\tilde{x}}^{\prime}(\overline{y}))=sign(1{\lambda}_{1})$. □
Theorem 10 Assume that ${\mathrm{\Delta}}_{1}>0$ and ${\lambda}_{1}$ and ${\lambda}_{2}$ are eigenvalues of ${J}_{T}(\overline{x},\overline{y})$. Then there exists the unique equilibrium point $E=(\overline{x},\overline{y})$ and the following hold:

(a)
If ${a}_{2}<{A}_{1}{c}_{2}$, then E is a saddle point and $0<{\lambda}_{1}<1$, ${\lambda}_{2}>1$.

(b)
Assume that ${a}_{2}>{A}_{1}{c}_{2}$. Let
$$\mathrm{\Gamma}(\overline{y})=5{c}_{2}{\overline{y}}^{4}+{\overline{y}}^{2}(9{A}_{1}{c}_{2}{a}_{2})+3{a}_{2}{A}_{1}.$$
(b1)
If $\mathrm{\Gamma}(\overline{y})>0$, then E is a saddle point. Furthermore, the following hold: $1<{\lambda}_{1}<0$, ${\lambda}_{2}>1$.

(b2)
If $\mathrm{\Gamma}(\overline{y})<0$, then E is a repeller. Furthermore, the following hold: ${\lambda}_{1}<1$, ${\lambda}_{2}>1$; ${\lambda}_{1}<{\lambda}_{2}$.

(b3)
If $\mathrm{\Gamma}(\overline{y})=0$, then E is a nonhyperbolic equilibrium point. Furthermore, the following hold: ${\lambda}_{1}=1$, ${\lambda}_{2}>1$.

(b1)
Proof In view of (7) and Lemma 1, we have that the function
has one zero $\overline{y}$ of multiplicity one. In view of Lemma 1, the map T has a unique equilibrium point. Since $\tilde{f}(0)={a}_{2}{b}_{1}^{2}<0$ and ${lim}_{y\to +\mathrm{\infty}}\tilde{f}(y)=+\mathrm{\infty}$, we have $\tilde{f}(y)<0$ for $y<\overline{y}$ and $\tilde{f}(y)>0$ for $y>\overline{y}$. By Lemmas 6 and 7 from [27], the equilibrium curves ${\mathcal{C}}_{f}$ and ${\mathcal{C}}_{g}$ intersect transversally at $(\overline{x},\overline{y})$, i.e., ${\tilde{x}}^{\prime}(\overline{y})\ne 0$. In view of Lemma 3 and by the continuity of function $\tilde{x}(y)$, there exists a neighborhood ${U}_{\overline{y}}$ of $\overline{y}$ such that ${\tilde{x}}^{\prime}(y)>0$ for $y\in {U}_{\overline{y}}$, which implies
From (21) and Lemma 4 we obtain ${\lambda}_{1}<1$ and ${\lambda}_{2}>1$.
If ${a}_{2}<{A}_{1}{c}_{2}$, then $det{J}_{T}(\overline{x},\overline{y})={\lambda}_{1}{\lambda}_{2}>0$, which implies that ${\lambda}_{1}\in (0,1)$.
Now, assume that ${a}_{2}>{A}_{1}{c}_{2}$. By using
one can see that
and
where $p(\lambda )=(\lambda {\lambda}_{1})(\lambda {\lambda}_{2})$. In view of (18) and $p(1)=({\lambda}_{1}+1)({\lambda}_{2}+1)$, we obtain statement (b) of the theorem. □
Lemma 5 Suppose that all the assumptions of Theorem 10 are satisfied. Let
Then the following statements are true.

(a)
$\mathrm{\Gamma}(\overline{y})>0$ if and only if one of the following inequalities holds:
$$\begin{array}{c}9{A}_{1}{c}_{2}{a}_{2}\ge 0,\hfill \\ 9{A}_{1}{c}_{2}{a}_{2}<0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}78{a}_{2}{A}_{1}{c}_{2}+{a}_{2}^{2}+81{A}_{1}^{2}{c}_{2}^{2}<0,\hfill \\ 9{A}_{1}{c}_{2}{a}_{2}<0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}78{a}_{2}{A}_{1}{c}_{2}+{a}_{2}^{2}+81{A}_{1}^{2}{c}_{2}^{2}\ge 0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\hfill \\ (\tilde{f}({y}_{})>0\mathit{\text{or}}\tilde{f}({y}_{+})0);\hfill \end{array}$$ 
(b)
$\mathrm{\Gamma}(\overline{y})<0$ if and only if the following hold:
$$9{A}_{1}{c}_{2}{a}_{2}<0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}78{a}_{2}{A}_{1}{c}_{2}+{a}_{2}^{2}+81{A}_{1}^{2}{c}_{2}^{2}\ge 0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}(\tilde{f}({y}_{})<0\mathit{\text{and}}\tilde{f}({y}_{+})0);$$ 
(c)
$\mathrm{\Gamma}(\overline{y})=0$ if and only if
$$9{A}_{1}{c}_{2}{a}_{2}<0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}78{a}_{2}{A}_{1}{c}_{2}+{a}_{2}^{2}+81{A}_{1}^{2}{c}_{2}^{2}\ge 0\phantom{\rule{1em}{0ex}}\mathit{\text{and}}\phantom{\rule{1em}{0ex}}(\tilde{f}({y}_{})=0\mathit{\text{or}}\tilde{f}({y}_{+})=0).$$
Proof The function $\tilde{f}(y)$ has one simple zero $\overline{y}$, which implies $\tilde{f}(y)<0$ for $0\le y<\overline{y}$ and $\tilde{f}(y)>0$ for $y>\overline{y}$. Then
while
for some $\alpha ,\beta \in [0,\mathrm{\infty})$. Now the proof follows from the fact that $\tilde{f}(y)=0$ has real roots
if and only if
□
Theorem 11 Assume that ${\mathrm{\Delta}}_{1}<0$. Then there exist three distinct equilibrium points in the positive quadrant: ${E}_{1}=({\overline{x}}_{1},{\overline{y}}_{1})$, ${E}_{2}=({\overline{x}}_{2},{\overline{y}}_{2})$ and ${E}_{3}=({\overline{x}}_{3},{\overline{y}}_{3})$ such that ${E}_{1}{\ll}_{ne}{E}_{2}{\ll}_{ne}{E}_{3}$ and the following hold:

(a)
${E}_{1}$ and ${E}_{3}$ are saddle points. If ${\lambda}_{1}^{(i)}$ and ${\lambda}_{2}^{(i)}$ are the eigenvalues of ${J}_{T}({E}_{i})$, $i=1,3$, then $0<{\lambda}_{1}^{(i)}<1$, ${\lambda}_{2}^{(i)}>1$.

(b)
The equilibrium point ${E}_{2}$ is a repeller. If ${\lambda}_{1}^{(2)}$ and ${\lambda}_{2}^{(2)}$ are the eigenvalues of ${J}_{T}({E}_{2})$, then $1<{\lambda}_{1}^{(2)}<{\lambda}_{2}^{(2)}$.
Proof In view of Lemma 1, equation (7) has three positive roots of multiplicity one. Since
then by (a) of Lemma 1 we obtain that the map T has three equilibrium points that we denote by ${E}_{1}$, ${E}_{2}$ and ${E}_{3}$. Given points lie on the increasing curve
which implies that the points are in the northeast ordering. Descartes’ rule of signs and (8) imply that $det{J}_{T}(x,y)>0$ when ${a}_{2}<{A}_{1}{c}_{2}$. In view of (7) and Lemma 1, we have that the polynomial
has three zeros ${\overline{y}}_{i}$, $i=1,2,3$, of multiplicity one. Since $\tilde{f}(0)={a}_{2}{b}_{1}^{2}<0$ and ${lim}_{y\to +\mathrm{\infty}}\tilde{f}(y)=+\mathrm{\infty}$, we have $\tilde{f}(y)<0$ for $y\in (0,{\overline{y}}_{1})\cup ({\overline{y}}_{2},{\overline{y}}_{3})$ and $\tilde{f}(y)>0$ for $y\in ({\overline{y}}_{1},{\overline{y}}_{2})\cup ({\overline{y}}_{3},+\mathrm{\infty})$.
By Lemmas 6 and 7 from [27], the equilibrium curves ${\mathcal{C}}_{f}$ and ${\mathcal{C}}_{g}$ intersect transversally at ${E}_{1}$, ${E}_{2}$ and ${E}_{3}$, i.e., ${\tilde{x}}^{\prime}({\overline{y}}_{i})\ne 0$, $i=1,2,3$. By this and Lemma 3 and by the continuity of function $\tilde{x}(y)$, there exists a neighborhood ${U}_{{\overline{y}}_{i}}^{(i)}$ of ${\overline{y}}_{i}$ such that ${\tilde{x}}^{\prime}(y)>0$ for $y\in {U}_{{\overline{y}}_{i}}^{(i)}$ for $i=1,3$ and ${\tilde{x}}^{\prime}(y)<0$ for $y\in {U}_{{\overline{y}}_{i}}^{(2)}$. Using this we get
Let
In view of (18), we have $det{J}_{T}({E}_{i})={\lambda}_{1}^{(i)}{\lambda}_{2}^{(i)}>0$, $i=1,2,3$. By Lemma 4 we obtain $0<{\lambda}_{1}^{(i)}<1$ and ${\lambda}_{2}^{(i)}>1$ for $i=1,3$. Since ${\tilde{x}}^{\prime}({\overline{y}}_{2})<0$, by Lemma 4 we have $1<{\lambda}_{1}^{(2)}<{\lambda}_{2}^{(2)}$. This completes the proof. □
Theorem 12 Assume that ${\mathrm{\Delta}}_{1}=0$ and ${\mathrm{\Delta}}_{2}\ne 0$ Then there exist two distinct equilibrium points in the positive quadrant ${E}_{1}=({\overline{x}}_{1},{\overline{y}}_{1})$ and ${E}_{3}=({\overline{x}}_{3},{\overline{y}}_{3})$ such that ${E}_{1}{\ll}_{ne}{E}_{3}$. Let ${\lambda}_{1}^{(i)}$ and ${\lambda}_{2}^{(i)}$ be the eigenvalues of ${J}_{T}({E}_{i})$, $i=1,3$. Then the following hold:

(a)
Exactly one of the roots ${\overline{y}}_{1}$ or ${\overline{y}}_{3}$ of (7) has multiplicity two.

(b)
If ${\overline{y}}_{1}$ is a root of (7) of multiplicity two, then the equilibrium point ${E}_{1}$ is nonhyperbolic and ${E}_{3}$ is a saddle point. Furthermore, ${\lambda}_{1}^{(1)}=1$, ${\lambda}_{2}^{(1)}>1$ and $0<{\lambda}_{1}^{(3)}<1$, ${\lambda}_{2}^{(3)}>1$.

(c)
If ${\overline{y}}_{3}$ is a root of (7) of multiplicity two, then the equilibrium point ${E}_{3}$ is nonhyperbolic and ${E}_{1}$ is a saddle point. Furthermore, ${\lambda}_{1}^{(3)}=1$, ${\lambda}_{2}^{(3)}>1$ and $0<{\lambda}_{1}^{(1)}<1$, ${\lambda}_{2}^{(1)}>1$.
Proof In view of Lemma 1, equation (7) has two positive zeros, one of multiplicity one and another one of multiplicity two, which implies statement (a). Since $\overline{x}=({A}_{1}+{\overline{y}}^{2})/{b}_{1}>0$, we obtain that the map T has two equilibrium points that we denote by ${E}_{1}$ and ${E}_{3}$. Descartes’ rule of signs and (8) imply that ${a}_{2}<{A}_{1}{c}_{2}\Rightarrow det{J}_{T}(x,y)>0$. Now, we prove statement (b). Similarly as in the proof of Theorem 11, one can see that ${E}_{3}$ is a saddle point. In view of Lemmas 6 and 7, from [27] we have that ${\tilde{x}}^{\prime}({\overline{y}}_{1})=0$, since ${\overline{y}}_{1}$ is the root of (7) of multiplicity two. By Lemma 4 we obtain ${\lambda}_{1}^{(1)}=1$, ${\lambda}_{2}^{(1)}>1$. The proof of statement (c) is similar and we will skip it. □
Theorem 13 Assume that ${\mathrm{\Delta}}_{1}=0$ and ${\mathrm{\Delta}}_{2}=0$. Then there exists one equilibrium point in the positive quadrant ${E}_{1}=({\overline{x}}_{1},{\overline{y}}_{1})$ which is nonhyperbolic. If ${\lambda}_{1}^{(1)}$ and ${\lambda}_{2}^{(1)}$ are eigenvalues of ${J}_{T}({E}_{1})$, then ${\lambda}_{1}^{(1)}=1$, ${\lambda}_{2}^{(1)}>1$.
Proof In view of Lemma 1, ${\overline{y}}_{1}$ is zero of (7) of multiplicity three. In view of Lemmas 6 and 7, from [27] we have that ${\tilde{x}}^{\prime}({\overline{y}}_{1})=0$. The rest of the proof is similar to that in Theorem 12 and we skip it. □
4.1 Periodtwo solution
Let
where
and
Periodtwo solution $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}$ satisfies the system
which is equivalent to
The Jacobian matrix of the map ${T}^{2}$ at $(x,y)$ has the form
The determinant of (23) at $(x,y)$ is given by
The trace of (23) at $(x,y)$ is given by
Lemma 6 Let ${\mathcal{C}}_{F}:=\{(x,y):F(x,y)=x\}$ and ${\mathcal{C}}_{G}:=\{(x,y):G(x,y)=y\}$ be the periodtwo curves, that is, the curves the intersection of which is a periodtwo solution. Then, for all $y>0$, there exist exactly one ${x}_{F}(y)>0$ and exactly one ${x}_{G}(y)>0$ such that $F({x}_{F}(y),y)=x$ and $G({x}_{G}(y),y)=y$. Furthermore, ${x}_{F}(y)$ and ${x}_{G}(y)$ are continuous functions and ${x}_{F}^{\prime}(y)>0$.
Proof Since $F(x,y)=x$ and $G(x,y)=y$ if and only if
respectively, in view of Descartes’ rule of signs, we have that for all $y>0$ there exist exactly one ${x}_{F}(y)>0$ and exactly one ${x}_{G}(y)>0$ such that $F({x}_{F}(y),y)=x$ and $G({x}_{G}(y),y)=y$. Taking derivatives of $F(x,y)=x$ with respect to y, we get
From $F(x,y)=x$ we have that ${({A}_{1}+{y}^{2})}^{2}=\frac{{b}_{1}^{3}{x}^{7}}{{({a}_{2}+{c}_{2}{y}^{2})}^{2}+{A}_{1}{x}^{4}}$, which implies
Since ${F}_{y}^{\prime}(x,y)<0$, we get ${x}_{F}^{\prime}(y)>0$. □
Theorem 14 If ${a}_{2}\le {A}_{1}{c}_{2}$, then T has no minimal periodtwo solution. If ${a}_{2}>{A}_{1}{c}_{2}$ and T has a minimal periodtwo solution $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}$, then $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}$ is unstable. If ${\mu}_{1}$ and ${\mu}_{2}$ (${\mu}_{1}<{\mu}_{2}$) are the eigenvalues of ${J}_{{T}^{2}}(\mathrm{\Phi},\mathrm{\Psi})$, then ${\mu}_{1}>0$ and ${\mu}_{2}>1$. All periodtwo solutions are ordered with respect to the northeast ordering.
Proof If ${a}_{2}\le {A}_{1}{c}_{2}$, the statement follows from Lemma 2. If ${a}_{2}>{A}_{1}{c}_{2}$, then from the first equation of (22) we have that
By substituting this into (24) we obtain
The rest of the proof follows from the fact that $tr{J}_{{T}^{2}}(\mathrm{\Phi},\mathrm{\Psi})={\mu}_{1}+{\mu}_{2}>4$, $det{J}_{{T}^{2}}(\mathrm{\Phi},\mathrm{\Psi})={\mu}_{1}{\mu}_{2}>0$ and Lemma 6. □
Theorem 15 If the map T has a minimal periodtwo solution $\{({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),T({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})\}$, which is nonhyperbolic, then $D(p)=0$, where $D(p)$ is the discriminant of the polynomial
where the coefficients ${p}_{i}$, $i=0,\dots ,22$, are in the Appendix. If $\{({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),T({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})\}$ and $\{({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2}),T({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\}$ are two minimal periodtwo solutions such that T has no other minimal periodtwo solutions in $\u301a({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\u301b=\{(x,y):({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}){\u2aaf}_{ne}(x,y){\u2aaf}_{ne}({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\}$ and $D(p)\ne 0$, then one of them is a saddle point and the other is a repeller.
Proof Periodtwo solution curves ${\mathcal{C}}_{F}=\{(x,y)\in \mathcal{R}:\tilde{F}(x,y)=0\}$ and ${\mathcal{C}}_{G}=\{(x,y)\in \mathcal{R}:\tilde{G}(x,y)=0\}$, where
are algebraic curves. By using software Mathematica, one can see that the resultant of the polynomials $\tilde{F}(x,y)$ and $\tilde{G}(x,y)$ in variable x is given by
Suppose that $\{({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),T({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})\}$ is a nonhyperbolic minimal periodtwo solution. This implies that $\tilde{F}({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})=0$, $\tilde{G}({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})=0$. By Theorem 9.3 [28], $\tilde{F}$ and $\tilde{G}$ have a common nonconstant factor if and only $R(\tilde{F},\tilde{G})=0$, which implies that system $\tilde{F}(x,y)=0$, $\tilde{G}(x,y)=0$ has a solution if and only if $R(\tilde{F},\tilde{G})=0$. Since $\tilde{f}({\mathrm{\Psi}}_{1})\ne 0$, it must be $p({\mathrm{\Psi}}_{1})=0$. Similarly as in Lemma 4, one can see that
where ${p}_{1}(\mu )$ is the characteristic equation of the matrix
From Theorem 14 we have that $0<{\mu}_{1}<{\mu}_{2}$ and ${\mu}_{2}>1$. Since $\{({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),T({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})\}$ is nonhyperbolic, we obtain that ${\mu}_{1}=1$, from which it follows that ${x}_{F}^{\prime}({\mathrm{\Psi}}_{1}){x}_{G}^{\prime}({\mathrm{\Psi}}_{1})=0$. Since $R(\tilde{F},\tilde{G})\not\equiv 0$, we have that ${\mathcal{C}}_{F}$ and ${\mathcal{C}}_{G}$ have no common component. By Lemmas 6 and 7, from [27], the curves ${\mathcal{C}}_{F}$ and ${\mathcal{C}}_{G}$ intersect transversally at $({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})$ (i.e., ${y}_{\tilde{F}}^{\prime}({\mathrm{\Psi}}_{1}){y}_{\tilde{G}}^{\prime}({\mathrm{\Psi}}_{1})\ne 0$) if and only if ${\mathrm{\Psi}}_{1}$ is zero of $p(y)$ of multiplicity one. By Theorem 9.4 [28], $p(y)$ has zeros of multiplicity greater than one if and only if the discriminant $D(p)$ of the polynomial $p(y)$ is equal to zero, which proves the first statement of the lemma.
Assume that $\{({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),T({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1})\}$ and $\{({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2}),T({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\}$ are two minimal periodtwo solutions such that T has no other minimal periodtwo solutions in $\u301a({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}),({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\u301b=\{(x,y):({\mathrm{\Phi}}_{1},{\mathrm{\Psi}}_{1}){\u2aaf}_{ne}(x,y){\u2aaf}_{ne}({\mathrm{\Phi}}_{2},{\mathrm{\Psi}}_{2})\}$ and $D(p)\ne 0$. From the previous discussion we have ${x}_{F}^{\prime}({\mathrm{\Psi}}_{i}){x}_{G}^{\prime}({\mathrm{\Psi}}_{i})\ne 0$, $i=1,2$. Since ${x}_{F}({\mathrm{\Psi}}_{i}){x}_{G}({\mathrm{\Psi}}_{i})=0$, $i=1,2$, it follows that $({x}_{F}^{\prime}({\mathrm{\Psi}}_{1}){x}_{G}^{\prime}({\mathrm{\Psi}}_{1}))({x}_{F}^{\prime}({\mathrm{\Psi}}_{2}){x}_{G}^{\prime}({\mathrm{\Psi}}_{2}))<0$. Indeed assume, for example, that ${x}_{F}^{\prime}({\mathrm{\Psi}}_{1}){x}_{G}^{\prime}({\mathrm{\Psi}}_{1})<0$ and ${x}_{F}^{\prime}({\mathrm{\Psi}}_{2}){x}_{G}^{\prime}({\mathrm{\Psi}}_{2})<0$. Then there exists $\u03f5>0$ such that ${x}_{F}(y){x}_{G}(y)<0$ for $y\in ({\mathrm{\Psi}}_{1},{\mathrm{\Psi}}_{1}+\u03f5)$ and ${x}_{F}(y){x}_{G}(y)>0$ for $y\in ({\mathrm{\Psi}}_{2}\u03f5,{\mathrm{\Psi}}_{2})$. Since ${x}_{F}(y){x}_{G}(y)$ is a continuous function, this implies that there exists $\mathrm{\Psi}\in ({\mathrm{\Psi}}_{1},{\mathrm{\Psi}}_{2})$ such that ${x}_{F}(\mathrm{\Psi}){x}_{G}(\mathrm{\Psi})=0$, which is a contradiction. The rest of the proof follows from the fact that ${e}_{i}>4$ and ${g}_{i}<0$, $i=1,2$. □
Notice that
where $R(p,{p}^{\prime})=detSyl(p,{p}^{\prime})$, the determinant of the Sylvester matrix $Syl(p,{p}^{\prime})$, see [28, 29], and
Theorem 16 If ${a}_{2}>{A}_{1}{c}_{2}$ and $\mathrm{\Gamma}(\overline{y})<0$, then T has one equilibrium point $E(\overline{x},\overline{y})$, which is a repeller, and there exists at least one minimal periodtwo solution $\{(\psi ,\varphi ),T(\psi ,\varphi )\}$ which is nonhyperbolic or a saddle point. If T has no minimal periodtwo solutions which are nonhyperbolic, then $(\psi ,\varphi ){\ll}_{ne}E{\ll}_{ne}T(\psi ,\varphi )$.
Proof By Theorem 10 we have that T has one equilibrium point $E(\overline{x},\overline{y})$, which is a repeller. This and Lemma 2 imply that ${T}^{n}({x}_{0},{y}_{0})$ is asymptotic to either $(0,\mathrm{\infty})$ or $(\mathrm{\infty},0)$, or a minimal periodtwo solution, for all $({x}_{0},{y}_{0})\in \mathcal{R}$. Let $\mathcal{B}(0,\mathrm{\infty})$ be the basin of attraction of $(0,\mathrm{\infty})$, and let $\mathcal{B}(\mathrm{\infty},0)$ be the basin of attraction of $(\mathrm{\infty},0)$. By using Theorem 9 one can prove that $int({Q}_{2}(E))\subset \mathcal{B}(0,\mathrm{\infty})$ and $int({Q}_{4}(E))\subset \mathcal{B}(\mathrm{\infty},0)$. Let ${\mathcal{S}}_{1}$ denote the boundary of $\mathcal{B}(\mathrm{\infty},0)$ considered as a subset of ${Q}_{1}(E)$, and let ${\mathcal{S}}_{2}$ denote the boundary of $\mathcal{B}(\mathrm{\infty},0)$ considered as a subset of ${Q}_{3}(E)$. It is easy to see that $E\in {\mathcal{S}}_{1}$, $E\in {\mathcal{S}}_{2}$ and $T(\mathcal{R})\subset int(\mathcal{R})$. Now we prove the following claim.
Claim 1 Let ${\mathcal{S}}_{1}$ and ${\mathcal{S}}_{2}$ be the sets defined as above. Then

(a)
If $({x}_{0},{y}_{0})\in \mathcal{B}(\mathrm{\infty},0)$, then $({x}_{1},{y}_{1})\in \mathcal{B}(\mathrm{\infty},0)$ for all $({x}_{0},{y}_{0}){\u2aaf}_{se}({x}_{1},{y}_{1})$.

(b)
If $({x}_{0},{y}_{0})\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$, then $({x}_{1},{y}_{1})\in int(\mathcal{B})(\mathrm{\infty},0)$ for all $({x}_{0},{y}_{0}){\ll}_{se}({x}_{1},{y}_{1})$.

(c)
${\mathcal{S}}_{1}\cap int({Q}_{1}(E))\ne \mathrm{\varnothing}$ and ${\mathcal{S}}_{2}\cap int({Q}_{3}(E))\ne \mathrm{\varnothing}$.

(d)
$T({\mathcal{S}}_{1}\cup {\mathcal{S}}_{2})\subseteq {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$.

(e)
$({x}_{0},{y}_{0}),({x}_{1},{y}_{1})\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}\Rightarrow ({x}_{0},{y}_{0}){\ll}_{ne}({x}_{1},{y}_{1})$ or $({x}_{1},{y}_{1}){\ll}_{ne}({x}_{0},{y}_{0})$.
Proof

(a)
The statement follows from ${T}^{n}({x}_{0},{y}_{0}){\u2aaf}_{se}{T}^{n}({x}_{1},{y}_{1}){\u2aaf}_{se}(\mathrm{\infty},0)$ and ${T}^{n}({x}_{0},{y}_{0})\to (\mathrm{\infty},0)$ as $n\to \mathrm{\infty}$.

(b)
The claim (b) follows from the observation that there exists a ball centered at $({x}_{0},{y}_{0})$ with the property that all its points $(x,y)$ satisfy $(x,y){\ll}_{se}({x}_{1},{y}_{1})$. But one of these points necessarily lies in $\mathcal{B}(\mathrm{\infty},0)$, so by (a) there exists $({x}_{1},{y}_{1})\in \mathcal{B}(\mathrm{\infty},0)$. Furthermore, there exists a ball centered at $({x}_{1},{y}_{1})$ with the property that all its points $(x,y)$ satisfy $(x,y)\in \mathcal{B}(\mathrm{\infty},0)$, which implies $({x}_{1},{y}_{1})\in int(\mathcal{B})(\mathrm{\infty},0)$.

(c)
Take ${y}^{\prime}>\overline{y}$ arbitrary (but fixed). Since T is strongly competitive, we have $T(\overline{x},{y}^{\prime}){\ll}_{se}T(\overline{x},\overline{y})$, which implies $T(\overline{x},{y}^{\prime})\in int({Q}_{2}(E))$. This implies that there exists a ball ${B}_{\epsilon}(T(\overline{x},{y}^{\prime}))$ with the property ${B}_{\epsilon}(T(\overline{x},{y}^{\prime}))\subset int({Q}_{2}(E))$. Since T is a continuous map on a set ${\mathbb{R}}_{+}^{2}\setminus \{(0,y):y\ge 0\}$, then there exists a ball ${B}_{{\delta}_{1}}(\overline{x},{y}^{\prime})$ such that $T({B}_{{\delta}_{1}}(\overline{x},{y}^{\prime}))\subset {B}_{\epsilon}(T(\overline{x},{y}^{\prime}))\subset int({Q}_{2}(E))$, which implies ${T}^{n}(x,y)\to (0,\mathrm{\infty})$ as $n\to \mathrm{\infty}$ for all $(x,y)\in {B}_{{\delta}_{1}}(\overline{x},{y}^{\prime})$. Similarly, one can prove that then there exists a ball ${B}_{{\delta}_{2}}(\overline{x}+{\delta}_{1}/2,\overline{y})$ such that ${T}^{n}(x,y)\to (\mathrm{\infty},0)$ as $n\to \mathrm{\infty}$ for all $(x,y)\in {B}_{{\delta}_{2}}(\overline{x}+{\delta}_{1}/2,\overline{y})$. Let ${y}^{\u2033}=sup\{y:{lim}_{n\to \mathrm{\infty}}{T}^{n}(\overline{x}+{\delta}_{1}/2,y)=(\mathrm{\infty},0)\}$. It is easy to see that $(\overline{x}+{\delta}_{1}/2,{y}^{\u2033})\in {\mathcal{S}}_{1}\cap int({Q}_{1}(E))$. The assertion concerning ${\mathcal{S}}_{2}$ is proved in a similar fashion.

(d)
Take $(x,y)\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$. Assume that $T(x,y)\notin {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$. Since ${\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}=\partial \mathcal{B}(\mathrm{\infty},0)=\overline{\mathcal{B}(\mathrm{\infty},0)}\setminus int(\mathcal{B}(\mathrm{\infty},0))$, then either $T(x,y)\in int(\mathcal{B}(\mathrm{\infty},0))$ or $T(x,y)\notin \overline{\mathcal{B}(\mathrm{\infty},0)}$. Assume that $T(x,y)\in int(\mathcal{B}(\mathrm{\infty},0))$. This implies that there exists a ball ${B}_{\epsilon}(T(x,y))$ with the property ${B}_{\epsilon}(T(x,y))\subset int(\mathcal{B}(\mathrm{\infty},0))$. Since T is a continuous map on the set ${\mathbb{R}}_{+}^{2}\setminus \{(0,y):y\ge 0\}$, then there exists a ball ${B}_{\delta}(x,y)$, $\delta >0$ such that $T({B}_{\delta}(x,y))\subset {B}_{\epsilon}(T(x,y))$, which implies ${B}_{\delta}(x,y)\subset \mathcal{B}(\mathrm{\infty},0)$. This is in contradiction with $(x,y)\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}=\partial \mathcal{B}(\mathrm{\infty},0)$. Hence $T(x,y)\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$ in this case. Similarly, one can prove that $T(x,y)\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$ if $T(x,y)\notin \overline{\mathcal{B}(\mathrm{\infty},0)}$. This implies that $T({\mathcal{S}}_{1}\cup {\mathcal{S}}_{2})\subseteq ({\mathcal{S}}_{1}\cup {\mathcal{S}}_{2})$.

(e)
Assume that $({x}_{0},{y}_{0}),({x}_{1},{y}_{1})\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}\Rightarrow ({x}_{0},{y}_{0}){\u2aaf}_{se}({x}_{1},{y}_{1})$ and $({x}_{0},{y}_{0})\ne ({x}_{1},{y}_{1})$. Since T is strongly competitive, we get $T({x}_{0},{y}_{0}){\ll}_{ne}T({x}_{1},{y}_{1})$. This contradicts (e) and (b), which completes the proof. □
In view of Claim 1, we have that $({\mathcal{S}}_{1}\cup {\mathcal{S}}_{2},{\ll}_{ne})$ is a totally ordered set which is invariant under T. If $({x}_{0},{y}_{0})\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}$, then $\{{T}^{(2n)}({x}_{0},{y}_{0})\}$ is eventually componentwise monotone. Then there exists a minimal periodtwo solution $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}\in {\mathcal{S}}_{1}\cup {\mathcal{S}}_{2}\subset {Q}_{1}(E)\cup {Q}_{3}(E)$ such that ${T}^{(2n)}({x}_{0},{y}_{0})\to (\mathrm{\Phi},\mathrm{\Psi})$ as $n\to \mathrm{\infty}$. By Theorem 14, $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}$ is a nonhyperbolic or a saddle point. Assume that T has no minimal periodtwo solutions which are nonhyperbolic points and, for example, that $(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\in {\mathcal{S}}_{1}$ such that $E{\ll}_{ne}(\mathrm{\Phi},\mathrm{\Psi}){\ll}_{ne}T(\mathrm{\Phi},\mathrm{\Psi})$. Since $\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\}$ is a saddle point, in view of Theorems 6, 7 and 8, we have that the global stable manifolds ${\mathcal{W}}^{s}(\{(\mathrm{\Phi},\mathrm{\Psi}),T(\mathrm{\Phi},\mathrm{\Psi})\})$ are the union of two curves ${\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi})$ and ${\mathcal{W}}^{s}(T(\mathrm{\Phi},\mathrm{\Psi}))$ whose endpoints are repeller points such that $T({\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi}))={\mathcal{W}}^{s}(T(\mathrm{\Phi},\mathrm{\Psi}))$ and $E{\ll}_{ne}\overline{{\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi})}{\ll}_{ne}\overline{{\mathcal{W}}^{s}(T(\mathrm{\Phi},\mathrm{\Psi}))}$. If ${P}_{1}$ and ${P}_{2}$ (${P}_{1}{\u2aaf}_{ne}{P}_{2}$) are endpoints of ${\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi})$, then $T({P}_{1})$ and $T({P}_{2})$ are endpoints of $T({\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi}))$ and either $T({P}_{1}){\u2aaf}_{ne}T({P}_{2})$ or $T({P}_{2}){\u2aaf}_{ne}T({P}_{1})$. Assume, for example, that ${P}_{1}{\ll}_{ne}{\mathcal{W}}^{s}(\mathrm{\Phi},\mathrm{\Psi}){\ll}_{ne}{P}_{2}{\ll}_{ne}T({P}_{2}){\ll}_{ne}{\mathcal{W}}^{s}(T(\mathrm{\Phi},\mathrm{\Psi})){\ll}_{ne}T({P}_{1})$. By Theorem 15 between two repellers ${P}_{2}$ and $T({P}_{2})$, there exists a saddle point ${S}_{1}$ where its stable manifold is the union of two invariant curves ${\mathcal{W}}^{s}({S}_{1})$ and ${\mathcal{W}}^{s}(T({S}_{1}))$ whose endpoints are repellers such that ${P}_{2}{\ll}_{ne}\overline{{\mathcal{W}}^{s}({S}_{1})}{\ll}_{ne}T({P}_{2})$. Continuing in this way, we obtain that T has infinitely many minimal periodtwo solutions $\{{P}_{i},T({P}_{i})\}$, which is in contradiction with the fact that T has at most eleven minimal periodtwo solutions. Hence $(\mathrm{\Phi},\mathrm{\Psi}){\ll}_{ne}E{\ll}_{ne}T(\mathrm{\Phi},\mathrm{\Psi})$. □
Corollary 2 Assume that ${a}_{2}>{A}_{1}{c}_{2}$, $\mathrm{\Gamma}(\overline{y})<0$ and $D(p)\ne 0$. Then there exists one equilibrium point E which is a repeller. Further, the set $int({Q}_{1}(E))\cup int({Q}_{3}(E))$ contains an odd number of minimal periodtwo solutions $\{({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}),({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})\}$, $i=1,\dots ,2n+1$, such that $({\mathrm{\Phi}}_{i+1},{\mathrm{\Psi}}_{i+1}){\ll}_{ne}({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}){\ll}_{ne}E$ and $E{\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i}){\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i+1},{\tilde{\mathrm{\Psi}}}_{i+1})$, where $({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})=T({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i})$. Furthermore, odd indexed periodtwo points are saddles and even indexed periodtwo points are repellers.
Proof By Theorem 10 we have that T has one equilibrium point $E(\overline{x},\overline{y})$, which is a repeller. By Theorem 15 all minimal periodtwo solutions are hyperbolic and the number of minimal periodtwo solutions is finite. In view of Claim 1, T has at least one minimal periodtwo solution which is a saddle point. Let $\{{P}_{1},T({P}_{1})\}$ be a minimal periodtwo solution which is a saddle point such that ${P}_{1}{\ll}_{ne}E{\ll}_{ne}T({P}_{1})$ and T has no minimal periodtwo solutions in $\u301aE,T({P}_{1})\u301b$ and $\u301a{P}_{1},E\u301b$. Such a minimal periodtwo solution exists in view of Theorem 15. The map ${T}^{2}$ satisfies all conditions of Theorems 6, 7 and 8, which yields the existence of the global stable manifolds ${\mathcal{W}}^{s}(\{{P}_{1},{\tilde{P}}_{1}\})$ which are the union of two curves ${\mathcal{W}}^{s}({P}_{1})$ and ${\mathcal{W}}^{s}({\tilde{P}}_{1})$ that have a common endpoint E. If T has minimal periodtwo solutions in $int({Q}_{1}(T({P}_{1})))\cup int({Q}_{3}({P}_{1}))$, let $\{{P}_{2},{\tilde{P}}_{2}\}$ (${P}_{2}{\ll}_{ne}{\tilde{P}}_{2}$) denote minimal periodtwo solutions such that T has no other minimal periodtwo solutions in $\u301aT({P}_{1}),T({P}_{2})\u301b$ and $\u301a{P}_{2},{P}_{1}\u301b$. Then ${\mathcal{W}}^{s}({P}_{1})$ has the second endpoint at ${P}_{2}$ and ${\mathcal{W}}^{s}(T({P}_{1}))$ has the second endpoint at $T({P}_{2})$ and ${P}_{2}{\ll}_{ne}{P}_{1}{\ll}_{ne}E{\ll}_{ne}T({P}_{1}){\ll}_{ne}T({P}_{2})$. Furthermore, a minimal periodtwo solution $\{{P}_{2},T({P}_{2})\}$ is a repeller. Similarly as in Theorem 16, one can prove that $int({Q}_{1}(T({P}_{2})))\cup int({Q}_{3}({P}_{2}))$ contains at least one minimal periodtwo solution which is a saddle point. Since the number of minimal periodtwo solutions is finite, continuing in this way, we will end with a minimal periodtwo solution which is a saddle point, from which the proof follows. □
Corollary 3 If ${a}_{2}>{A}_{1}{c}_{2}$ and $\mathrm{\Gamma}(\overline{y})>0$, then there exists one equilibrium point E which is a saddle point. If $D(p)\ne 0$, then $int({Q}_{1}(E))\cup int({Q}_{3}(E))$ contains an even number of minimal periodtwo solutions $\{({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}),({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})\}$, $i=1,\dots ,2n$, such that $({\mathrm{\Phi}}_{i+1},{\mathrm{\Psi}}_{i+1}){\ll}_{ne}({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}){\ll}_{ne}E$ and $E{\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i}){\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i+1},{\tilde{\mathrm{\Psi}}}_{i+1})$ and $({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})=T({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i})$. Furthermore, even indexed periodtwo points are saddles and odd indexed periodtwo points are repellers.
Proof The proof is similar as the proof of Corollary 2 and it will be omitted. □
Based on a series of numerical simulations, we propose the following conjecture.
Conjecture 1 System (1) has at most one minimal periodtwo solution.
5 Global behavior
In this section we present global dynamics of system (1) in different parametric regions. We have five parametric regions with different dynamics which will be characterized by the following five theorems.
5.1 The case where the equilibrium points and periodtwo solutions are hyperbolic points (${\mathrm{\Delta}}_{1}\ne 0$ and $D(p)\ne 0$)
Theorem 17 Assume that ${\mathrm{\Delta}}_{1}<0$. Then system (1) has three equilibrium solutions ${E}_{1}{\ll}_{ne}{E}_{2}{\ll}_{ne}{E}_{3}$, where ${E}_{1}$ and ${E}_{3}$ are saddle points and ${E}_{2}$ is a repeller. In this case there exist four invariant continuous curves ${\mathcal{W}}^{s}({E}_{1})$, ${\mathcal{W}}^{s}({E}_{3})$, ${\mathcal{W}}^{u}({E}_{1})$, ${\mathcal{W}}^{u}({E}_{3})$, where ${\mathcal{W}}^{s}({E}_{1})$, ${\mathcal{W}}^{s}({E}_{3})$ have end points at ${E}_{2}$, and are graphs of increasing functions. The curves ${\mathcal{W}}^{u}({E}_{1})$, ${\mathcal{W}}^{u}({E}_{3})$ are the graphs of decreasing functions. Every solution $\{({x}_{n},{y}_{n})\}$ which starts below ${\mathcal{W}}^{s}({E}_{1})\cup {\mathcal{W}}^{s}({E}_{3})$ in the southeast ordering is asymptotic to $(0,\mathrm{\infty})$, and every solution $\{({x}_{n},{y}_{n})\}$ which starts above ${\mathcal{W}}^{s}({E}_{1})\cup {\mathcal{W}}^{s}({E}_{3})$ in the southeast ordering is asymptotic to $(\mathrm{\infty},0)$. The first quadrant of initial condition ${Q}_{1}=\{({x}_{0},{y}_{0}):{x}_{0}>0,{y}_{0}\ge 0\}$ is the union of five disjoint basins of attraction, i.e.,
where $\mathcal{B}({E}_{2})=\{{E}_{2}\}$, $\mathcal{B}({E}_{1})={\mathcal{W}}^{s}({E}_{1})$, $\mathcal{B}({E}_{3})={\mathcal{W}}^{s}({E}_{3})$ and
Proof Theorem 11 implies that there exist three equilibrium points, namely ${E}_{1}$, ${E}_{2}$ and ${E}_{3}$, such that ${E}_{1}{\ll}_{ne}{E}_{2}{\ll}_{ne}{E}_{3}$. In this case, ${E}_{1}$ and ${E}_{3}$ are saddle points and ${E}_{2}$ is a repeller. In view of (17), the map T is competitive on ℛ and strongly competitive on $int(\mathcal{R})$. It follows from the PerronFrobenius theorem and a change of variables [14] that at each point the Jacobian matrix of a strongly competitive map has two real and distinct eigenvalues, the larger one in absolute value being positive, and that corresponding eigenvectors may be chosen to point in the direction of the second and first quadrant, respectively. Also, one can show that if the map is strongly competitive, then no eigenvector is aligned with a coordinate axis.
Since ${\mathrm{\Delta}}_{1}<0$ implies ${a}_{2}<{A}_{1}{c}_{2}$, we have that $det{J}_{T}({E}_{i})>0$, $i=1,3$. Hence, all conditions of Theorems 6, 7 and 8 are satisfied, which yields the existence of the global stable manifolds ${\mathcal{W}}^{s}({E}_{1})$, ${\mathcal{W}}^{s}({E}_{3})$ and the global unstable manifolds ${\mathcal{W}}^{u}({E}_{1})$, ${\mathcal{W}}^{u}({E}_{3})$, where ${\mathcal{W}}^{s}({E}_{1})$, ${\mathcal{W}}^{s}({E}_{3})$ are passing through the point ${E}_{2}$ and are graphs of increasing functions. The curves ${\mathcal{W}}^{u}({E}_{1})$, ${\mathcal{W}}^{u}({E}_{3})$ are the graphs of decreasing functions. Let
Take $({x}_{0},{y}_{0})\in {\mathcal{W}}^{}\cap \mathcal{R}$. By Theorem 7 we have that there exists ${n}_{0}>0$ such that ${T}^{n}({x}_{0},{y}_{0})\in int(({Q}_{2}({E}_{1})\cup {Q}_{2}({E}_{3}))\cap \mathcal{R})$, $n>{n}_{0}$. In view of Theorem 9, since ${E}_{1}$ and ${E}_{3}$ are saddle points, we obtain that for all $({x}_{0},{y}_{0})\in int(({Q}_{2}({E}_{1})\cup {Q}_{2}({E}_{3}))\cap \mathcal{R})$, there exists a subsolution $({u}_{0},{v}_{0})$ such that $({x}_{0},{y}_{0}){\u2aaf}_{se}({u}_{0},{v}_{0})$. By monotonicity we have that $(0,\mathrm{\infty}){\u2aaf}_{se}{T}^{n}({x}_{0},{y}_{0}){\u2aaf}_{se}{T}^{n}({u}_{0},{v}_{0})\ll {E}_{3}\text{if}({x}_{0},{y}_{0})\in int({Q}_{2}({E}_{3})\cap \mathcal{R})$ and $(0,\mathrm{\infty}){\u2aaf}_{se}{T}^{n}({x}_{0},{y}_{0}){\u2aaf}_{se}{T}^{n}({u}_{0},{v}_{0})\ll {E}_{1}\text{if}({x}_{0},{y}_{0})\in int({Q}_{2}({E}_{1})\cap \mathcal{R})$. For the sequence $({u}_{n},{v}_{n})={T}^{n}({u}_{0},{v}_{0})$, we have $({u}_{n+1},{v}_{n+1}){\u2aaf}_{se}({u}_{n},{v}_{n}){\u2aaf}_{se}({u}_{0},{v}_{0})$. The sequence $\{{u}_{n}\}$ is monotone decreasing and bounded from below and $\{{v}_{n}\}$ is monotone increasing. Since T has no other equilibrium point except ${E}_{1}$, ${E}_{2}$ and ${E}_{3}$, this implies that the sequence $\{{u}_{n}\}$ has finite limit and ${v}_{n}\to +\mathrm{\infty}$ as $n\to \mathrm{\infty}$. From ${u}_{n+1}=({b}_{1}{u}_{n}^{2})/({A}_{1}+{v}_{n}^{2})$ we obtain that ${u}_{n}\to 0$ as $n\to \mathrm{\infty}$. This implies ${T}^{n}({x}_{0},{y}_{0})\to (0,\mathrm{\infty})$ as $n\to \mathrm{\infty}$. If $({x}_{0},{y}_{0})\in {\mathcal{W}}^{+}\cap \mathcal{R}$, the proof is similar and we skip it.
Another way of completing the proof is by using (3) of Lemma 2. □
Theorem 18 Assume that ${\mathrm{\Delta}}_{1}>0$. If
or
then system (1) has one equilibrium solution $E(\overline{x},\overline{y})$, which is a saddle point. There exists the global stable manifold ${\mathcal{W}}^{s}(E)$ which is the graph of a continuous increasing function and the global unstable manifold ${\mathcal{W}}^{u}(E)$, which is the graph of a continuous decreasing function. Every solution $\{({x}_{n},{y}_{n})\}$ which starts below ${\mathcal{W}}^{s}(E)$ in the southeast ordering is asymptotic to $(0,\mathrm{\infty})$, and every solution $\{({x}_{n},{y}_{n})\}$ which starts above ${\mathcal{W}}^{s}(E)$ in the southeast ordering is asymptotic to $(\mathrm{\infty},0)$. The first quadrant of initial condition ${Q}_{1}=\{({x}_{0},{y}_{0}):{x}_{0}>0,{y}_{0}\ge 0\}$ is the union of three disjoint basins of attraction, ${Q}_{1}=\mathcal{B}(0,\mathrm{\infty})\cup \mathcal{B}(\mathrm{\infty},0)\cup \mathcal{B}(E)$, where $\mathcal{B}(E)={\mathcal{W}}^{s}(E)$ and
Proof The conditions of this theorem and ${a}_{2}\ne {A}_{1}{c}_{2}$ imply all assumptions of Theorems 6, 7 and 8 for ℛ. The proof of this theorem in this case is similar to the proof of Theorem 17 and will be skipped.
Now, we assume that ${a}_{2}={A}_{1}{c}_{2}$. In view of Lemma 2, the set
is invariant and contains equilibrium E, and $T(x,y)=E$ for $(x,y)\in \mathcal{I}$. In view of the uniqueness of the global stable manifold, we conclude that ${\mathcal{W}}^{s}(E)=\mathcal{I}$. Let
Take any point $({x}_{0},{y}_{0})\in {\mathcal{W}}^{+}(E)$. Then there exists the point $({x}_{l},{y}_{l})\in \mathcal{I}$ such that $({x}_{l},{y}_{l}){\ll}_{se}({x}_{0},{y}_{0})$. Since the map T is strongly competitive, then $E=T({x}_{l},{y}_{l}){\ll}_{se}T({x}_{0},{y}_{0})$. This implies $T({x}_{0},{y}_{0})\in int({Q}_{4}(E)\cap \mathcal{R})$. If $({x}_{0},{y}_{0})\in {\mathcal{W}}^{}(E)$, then there exists the point $({x}_{r},{y}_{r})\in \mathcal{I}$ such that $({x}_{0},{y}_{0}){\ll}_{se}({x}_{r},{y}_{r})$. Since the map T is strongly competitive, then $T({x}_{0},{y}_{0}){\ll}_{se}E=T({x}_{r},{y}_{r})$. This implies $T({x}_{0},{y}_{0})\in int({Q}_{2}(E)\cap \mathcal{R})$. Similarly as in Theorem 17, one can prove that ${T}^{n}({x}_{0},{y}_{0})\to (0,\mathrm{\infty})$ as $n\to \mathrm{\infty}$ if $({x}_{0},{y}_{0})\in int({Q}_{2}(E)\cap \mathcal{R})$ and ${T}^{n}({x}_{0},{y}_{0})\to (\mathrm{\infty},0)$ as $n\to \mathrm{\infty}$ if $({x}_{0},{y}_{0})\in int({Q}_{4}(E)\cap \mathcal{R})$, from which the proof follows in this case. □
See Figure 1 for visual illustration of Theorems 17 and 18.
Theorem 19 If ${a}_{2}>{A}_{1}{c}_{2}$ and $\mathrm{\Gamma}(\overline{y})<0$, then there exists one equilibrium point E which is a repeller. If $D(p)\ne 0$, then $int({Q}_{1}(E))\cup int({Q}_{3}(E))$ contains an odd number of minimal periodtwo solutions $\{({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}),({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})\}$, $i=1,\dots ,2n+1$, such that $({\mathrm{\Phi}}_{i+1},{\mathrm{\Psi}}_{i+1}){\ll}_{ne}({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i}){\ll}_{ne}E$ and $E{\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i}){\ll}_{ne}({\tilde{\mathrm{\Phi}}}_{i+1},{\tilde{\mathrm{\Psi}}}_{i+1})$ and $({\tilde{\mathrm{\Phi}}}_{i},{\tilde{\mathrm{\Psi}}}_{i})=T({\mathrm{\Phi}}_{i},{\mathrm{\Psi}}_{i})$. Furthermore, the odd indexed periodtwo points are saddles and the even indexed periodtwo points are repellers. The global stable manifolds are given by
where ${\mathcal{W}}^{s}({\mathrm{\Phi}}_{}$