- Open Access
Periodicity and exponential stability of discrete-time neural networks with variable coefficients and delays
Advances in Difference Equations volume 2013, Article number: 226 (2013)
Discrete analogues of continuous-time neural models are of great importance in numerical simulations and practical implementations. In the current paper, a discrete model of continuous-time neural networks with variable coefficients and multiple delays is investigated. By Lyapunov functional, continuation theorem of topological degree, inequality technique and matrix analysis, sufficient conditions guaranteeing the existence and globally exponential convergence of periodic solutions are obtained, without assuming the boundedness and differentiability of activation functions. To show the effectiveness of our method, an illustrative example is presented along with numerical simulations.
MSC:34D23, 34K20, 39A12, 92B20.
Various neural networks have been so far proposed, and they attracted extensive interest of researchers from various fields, since they play important roles and have found successful applications in the fields such as pattern recognition, signal and image processing, nonlinear optimization problems, parallel computation, and other engineering areas; see, for example, [1–8]. The dynamical behaviors in neural networks, such as the existence and their asymptotic stability of equilibria, periodic solutions, bifurcations and chaos, have been the most active areas of research and have been extensively studied over the past years [9–21].
Time-delays in interactions between neurons are frequently unavoidable due to the finite transmission speed of signals among neurons, and they cause instability, divergence and oscillations in neural networks , so it is necessary to introduce time delays into the neural models. Numerous sufficient conditions ensuring the stability have been given for neural models with discrete, time-varying and distributed delays, respectively.
Meanwhile, in numerical simulations and practical implementations, discretization of continuous-time models is necessary and of great importance. Certainly, to faithfully reflect the dynamical behaviors of continuous systems, the discrete analogues should inherit the dynamical characteristics of continuous counterparts [13, 23]. The ways to derive the discrete-time analogues from continuous versions are diverse, but most of them cannot keep the original dynamics and display more complicated behaviors. To this end, an implicit scheme has been put forward  to derive the discrete analogues. For discrete models under this scheme, such as discrete Hopfield, bidirectional associate memory and cellular neural networks, several authors [24–29] have studied the existence and exponential stability of equilibria and periodic solutions. However, they are mainly concerned with the models of constant coefficients. Research on the discrete models with variable coefficients and delays is very rare. Since the neuron charging time, interconnection weights and external inputs often change during the course of time, neural models with temporal structure of neural activities are much closer to real systems, and hence studies on such systems and their discrete versions are of great practical and theoretical value.
Motivated by the above discussions, we present a discrete analogue of continuous-time neural networks with variable coefficients and delays. Here the activation functions are not assumed to be bounded, as opposed to those in [25–28]. To deal with this general model, a suitable and effective Lyapunov functional is constructed. Continuation theorem of topological degree , inequality technique and matrix analysis  are employed to obtain the sufficient conditions guaranteeing the existence and globally exponential stability of periodic solutions. As we see, these sufficient conditions are less conservative and easy to verify. Further, no restrictions of the differentiability and monotonicity are imposed on activation functions. Also, note that the discrete models admit the common dynamical behaviors with continuous versions. That implies they preserve the dynamics very well from continuous versions. To show the effectiveness of our results, an illustrative example along with numerical simulations is presented.
The paper is organized as follows. In Section 2, discrete-time neural networks with variable coefficients and delays are formulated. Some assumptions and mathematical preliminaries are also given. Section 3 is devoted to the existence of periodic solutions. In Section 4, the exponential convergence of this discrete model is discussed. To show the effectiveness of the method, an illustrative example is presented in Section 5. Some conclusions are drawn in Section 6.
2 Mathematical preliminaries
Continuous-time neural networks with time-varying coefficients and delays read as follows:
with initial values
where , , ; , is the state of the i th neuron at time t; the continuous function represents the neuron charging time, is the strength of the j th unit on the i unit at time ; denotes the activation function of the neuron, which satisfies the global Lipschitz condition; denotes the transmission delay along the axon of the j th unit; the continuous function is the external input on the i th neuron at time t; the initial value function is bounded and continuous on . Research into dynamics such as stability and periodic oscillations for this model has been extensively carried out . Now we will focus on the dynamics of its discrete analogue.
Set Z to be the set of integers and the set of nonnegative integers; let represent the set of integers between a and b with , , namely, .
Reformulate the continuous-time model (1) by equations with piecewise constant arguments of the form
where , , is the discretization step size and is the integer part of . Let
where , then one has
Integrate over the interval to get
let , then
The discrete model (3) is endowed with initial values
where is bounded on .
Note that and for small . It could be showed that the discrete-time analogue (3) converges to the continuous-time model (1) as .
To investigate the stability and periodic oscillations of system (3), we make further assumptions.
(H1) Suppose that , , and are all ω-periodic functions; moreover, , , with ω and k being positive integers, for .
(H2) Assume that a function satisfies the Lipschitz condition, i.e., there exists a constant such that
for any ; further, , .
(H3) Suppose that there exist constants such that
where , .
For any , a solution of system (3) and (4) is a vector-valued function satisfying system (3) and initial conditions (4) for . In this paper, it is always assumed that the neural model (3) and (4) admits a solution represented by or simply .
For later convenience, throughout this paper represents the mean value of a function over . Denote by , the maximum and the minimum of a function over , respectively, i.e.,
To analyze the existence and stability of periodic solutions for system (3), M-matrix theory is employed. Some notations and terminologies are given below. For more details, please refer to .
Definition 2.1 
Matrix is said to be a nonsingular M-matrix if (i) , (ii) for , (iii) , .
Lemma 2.1 
If Λ is a nonsingular M-matrix and , then
Lemma 2.2 
Let Λ be an matrix with nonpositive off-diagonal elements. Then Λ is a nonsingular M-matrix if and only if one of following statements holds true:
There exists a constant vector , with , , such that
The real parts of all eigenvalues of Λ are positive.
There exists a symmetric positive definite matrix W such that
is positive definite.
All of the principal minors of Λ are positive.
3 Periodic solutions
Next we investigate the existence and global exponential stability of periodic solutions of system (3). By the continuation theorem of topological degree, the Lyapunov functional and analytic techniques such as matrix analysis, the existence and its global exponential stability of ω-periodic solutions are established. Let us first introduce the continuation theorem due to Gaines and Mawhin .
Let X and Y be two real Banach spaces, let be a Fredholm operator of index zero, and let , be continuous projectors such that , , and , . Denote by the restriction of L on ; by the inverse of ; by the algebraic and topological isomorphism of ImQ onto KerL, due to the same dimensions of these two subspaces.
Lemma 3.1 (Continuation theorem )
Let be an open bounded set and let be a continuous operator which is L-compact on . Suppose that
for each , , ;
for each , ;
Then admits at least one solution in .
To achieve our goal, take
endowed with the norm
with which X, Y are Banach spaces, here is the Euclidean norm.
where I is the identity matrix. From periodicity and positiveness, it holds that , .
Lemma 3.2 Under hypothesis (H3), matrix Γ is a nonsingular M-matrix.
it follows from (H3) and Lemma 2.2 that Γ is a nonsingular M-matrix. □
Theorem 3.1 Suppose that hypotheses (H1)-(H3) hold. If
then system (3) admits at least one ω-periodic solution.
Proof Let be defined by
and let the i th component of the mapping be defined, respectively, by
With these notations, system (3) is rewritten into the form
It is not difficult to see that
hence ImL is closed in X, and , that is, L is a Fredholm operator of index zero.
Define the linear continuous projectors by
In this way, , , , and the isomorphism J from ImQ onto KerL is taken to be the identity map.
Clearly, the mapping is one-to-one and onto, so invertible. Its inverse is defined as
Consequently, by the Lebesgue convergence theorem, QN and are continuous, and via the Arzela-Ascoli theorem, and are compact for any open bounded set , namely, N is L-compact on .
For a certain , suppose that is a solution of
Now one could get the estimates as follows:
Therefore, we have
for . Set
then inequality (6) is equivalent to
From Lemma 2.2, we know that Γ is a nonsingular M-matrix, then from Lemma 2.1, it holds that
That means is bounded, i.e., there exist constants such that
Take such that
where , .
according to the above discussions, for , . So, condition (a) in Lemma 3.1 is satisfied. When , x is a constant vector in , with , then is expressed as , with
where . Since
in this way,
that is, condition (b) in Lemma 3.1 holds.
Let be defined by
where . When , it follows that
As a result, the homotopy invariance implies
Hence condition (c) in Lemma 3.1 is verified. By Lemma 3.1, we conclude that system (3) admits at least one ω-periodic solution. This completes the proof. □
4 Exponential stability of periodic solutions
By Theorem 3.1, system (3) has at least an ω-periodic solution . Clearly, if is exponentially stable, then the ω-periodic solution of system (3) is unique. Now we will investigate the exponential stability of periodic solutions. Set , then system (3) is equivalent to
Theorem 4.1 Suppose that all conditions in Theorem 3.1 hold except that (H3) is replaced by
(H4) Suppose that there exist constants such that
then the ω-periodic solution of system (3) is globally exponentially stable in the sense that there exist constants and such that for any solution of system (3) and (4) with initial condition ϕ, it holds that
for all .
Proof Define the function by
for . From (H4), we have
From the continuity of functions , there must be a number such that
for . From (9) and (H2), one has
Set , then it holds from (11) that
where . Define a Lyapunov functional as follows:
To investigate the exponential stability of an ω-periodic solution by the Lyapunov functional , it is necessary to calculate the difference along the solutions of (12). From (12), we have
Therefore, we have
In view of (10), it follows that for all , which means that for . Note that
so we have
where , and
The proof is complete. □
Theorem 4.2 Suppose that all conditions in Theorem 3.1 hold, then the ω-periodic solution of system (3) is globally exponentially stable in the sense that there exist constants and such that for any solution of system (3) and (4) with initial conditions ϕ, it holds that
for all .
Proof Define the function by
From (H3), we have
From the continuity of functions , there must be a number such that
From (9) and (H2), one has
Set , then it holds that
where . Let . It is clear that for , . We claim that
Otherwise, there should be an index r and a positive integer such that
That is, is the first time that inequality (16) is violated. Meanwhile, by (15) and (13), one has
That leads to a contradiction. Therefore, the assertion (16) is true. Consequently,
That implies that the ω-periodic solution of system (3) and (4) is globally exponentially stable. The proof is completed. □
Remark Frequently activation functions are assumed to be bounded and monotonic; see, for instance, [18, 28]. However, no restrictions of boundedness, monotonicity and differentiability are imposed in this paper. Moreover, the conditions ensuring exponential stability are less conservative and easy to verify. Also, it could be noted  that continuous model (1) admits the common behaviors with system (3).
From Theorems 3.1, 4.1, 4.2 and Lemma 2.2, some corollaries could be immediately derived.
Corollary 4.1 Assume that hypotheses (H1) and (H2) hold and ; further assume that one of the following conditions holds:
Then system (3) admits a unique ω-periodic solution, which is globally exponentially stable, that is, all other solutions converge to it exponentially as .
Corollary 4.2 Under conditions (H1), (H2) and , if the matrix is a nonsingular M-matrix, where , , then system (3) admits a unique ω-periodic solution, which is globally exponentially stable.
When , , , , then the matrices , and system (3) reduces to the model with constant coefficients and delays, that is,
Since the equilibrium could be viewed as the periodic solution of arbitrary period, as a consequence of Theorems 3.1, 4.1 and 4.2, we obtain the following corollary.
Corollary 4.3 If satisfies hypothesis (H2), , ,
and, further, the matrix is a nonsingular M-matrix, then system (17) has a unique equilibrium, which is globally exponentially stable.
5 Numerical results
Note that when neural networks are applied to practical problems such as image processing, pattern recognition, artificial intelligence, computer simulations, and so on, discrete versions of models are needed, since the information is discrete in nature and processing procedures occur in discrete steps.
Now some algorithms based on discrete-time neural models have been given. For example, Chen et al.  put forward an image processing method based on discrete neural models, which were implemented on circuits; Wang et al. [33, 34] proposed the discrete Hopfield neural models for Max-cut problems and cellular channel assignments; and Yashtini et al.  gave the discrete model for nonlinear convex programming. Moreover, the discrete-time cellular neural networks for associate memories with learning and forgetting capabilities [36, 37] also were established. So, discrete neural networks have extensive uses in real-life applications.
To show the effectiveness of the obtained theoretical results, an illustrative example is given. Consider the following discrete-time neural network:
The example is a discrete network of two neurons with self-connection, which is of Hopfield type. The model is often implemented in practical applications such as image processing, pattern recognition and artificial intelligence. Such networks are the prototypes to understand the dynamics of larger-scale networks.
The corresponding continuous model is
It is the Hopfield network with associate memory and data storage capability . To measure the information transmitted among neurons, i.e., inputs and outputs, discrete samplings are necessary on discrete time instances. Frequently, periodic samplings are adopted. So, under the proposed discretization scheme, the discrete version of neural model follows.
Activation functions , are chosen to be hyperbolic tangent and inverse tangent ones, respectively, which are of sigmoid symmetric type. They are frequently used in neural networks and work very well in applications. The coefficients , () are chosen to be periodic functions. Here we take trigonometric functions. It is not difficult to see that (H1) and (H2) are satisfied by this discrete-time neural network. The parameters are as follows:
When , are set to be 1, , respectively, (H3) is also true, so from Theorems 3.1 and 4.2, it admits a unique periodic solution, with all other solutions converging to it exponentially as (see Figures 1-3).
From Figures 1-3, note that all the solutions tend to the unique periodic solution. So, the unique periodic solution exists and it is exponentially stable. The trajectories of a continuous model are showed in Figure 4. Note that it admits a unique periodic solution. Also note that the discrete analogue model well preserves the dynamics of the corresponding continuous one.
In the current paper, a class of discrete-time neural networks has been studied. Using the coincidence degree, the Lyapunov functional and matrix analysis, the existence and its global exponential stability of a periodic solution have been established for this model, assuming no boundedness, monotonicity and differentiability of activation functions. The obtained results are less conservative and will be of practical use for applying neural models. Also note that the discrete-time analogue model well preserves the dynamical behaviors from the continuous version.
Hopfield J: Neurons with graded response have collective computational properties like those of two state neurons. Proc. Natl. Acad. Sci. USA 1984, 81: 3088-3092. 10.1073/pnas.81.10.3088
Chen M, Grossberg S: Absolute stability and global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Syst. Man Cybern. 1983, 13: 815-821.
Chua L, Yang L: Cellular neural networks: theory and applications. IEEE Trans. Circuits Syst. I 1988, 35: 1257-1290. 10.1109/31.7600
Kosko B: Neural Networks and Fuzzy Systems - A Dynamical System Approach Machine Intelligence. Prentice Hall, Englewood Cliffs; 1992.
Hjelmfelt A, Ross J: Pattern recognition, chaos and multiplicity in neural networks and excitable systems. Proc. Natl. Acad. Sci. USA 1994, 91: 63-67. 10.1073/pnas.91.1.63
Kennedy M, Chua L: Neural networks for nonlinear programming. IEEE Trans. Circuits Syst. 1998, 35: 554-562.
Cochocki A, Unbehauen R: Neural Networks for Optimization and Signal Processing. Wiley, Stuttgart; 1993.
Rolls E, Alessandro T: Neural Networks and Brain Function. Oxford University Press, Oxford; 1998.
Marcus CM, Westervelt RM: Stability of analog neural networks with delay. Phys. Rev. A 1989, 39(1):347-359. 10.1103/PhysRevA.39.347
Gopalsamy K, He X: Delay-independent stability in bidirectional associative memory networks. IEEE Trans. Neural Netw. 1994, 5: 998-1002. 10.1109/72.329700
Driessche PV, Zou XF: Global attractivity in delayed Hopfield neural networks. SIAM J. Appl. Math. 1998, 58: 1878-1890. 10.1137/S0036139997321219
Rao V, Phaneendra BR: Global dynamics of bidirectional associative memory neural networks involving transmission delays and dead zones. Neural Netw. 1999, 12(3):455-465. 10.1016/S0893-6080(98)00134-8
Mohamad S, Gopalsamy K: Dynamics of a class of discrete-time neural networks and their continuous-time counterparts. Math. Comput. Simul. 2000, 53: 1-39. 10.1016/S0378-4754(00)00168-3
Liu Y, Wang Z, Liu X: Asymptotic stability for neural networks with mixed time-delays: the discrete-time case. Neural Netw. 2009, 22(1):67-74. 10.1016/j.neunet.2008.10.001
Ensari T, Arik S: New results for robust stability of dynamical neural networks with discrete time delays. Expert Syst. Appl. 2010, 37(8):5925-5930. 10.1016/j.eswa.2010.02.013
Yu J, Zhang K, Fei S: Exponential stability criteria for discrete-time recurrent neural networks with time-varying delay. Nonlinear Anal., Real World Appl. 2010, 11(1):207-216. 10.1016/j.nonrwa.2008.10.053
Alanis AY, Sanchez EN, Loukianov AG, Hernandez EA: Discrete-time recurrent high order neural networks for nonlinear identification. J. Franklin Inst. 2010, 347(7):1253-1265. 10.1016/j.jfranklin.2010.05.018
Liu Z, Liao L: Existence and global exponential stability of periodic solution of cellular neural networks with time-varying delays. J. Math. Anal. Appl. 2004, 290: 247-262. 10.1016/j.jmaa.2003.09.052
Yucel E, Arik S: New exponential stability results for delayed neural networks with time varying delays. Physica D 2004, 191(3-4):314-322. 10.1016/j.physd.2003.11.010
Faydasicok O, Arik S: Robust stability analysis of a class of neural networks with discrete time delays. Neural Netw. 2012, 29-30: 52-59.
Cheng C, Lin K, Shih C: Multistability and convergence in delayed neural networks. Physica D 2007, 225: 61-74. 10.1016/j.physd.2006.10.003
Niculescu SI: Delay Effects on Stability: A Robust Approach. Springer, Berlin; 2001.
Stuart A, Humphries A: Dynamical Systems and Numerical Analysis. Cambridge University Press, Cambridge; 1996.
Sun C, Feng C: Discrete-time analogues of integrodifferential equations modelling neural networks. Phys. Lett. A 2005, 334: 180-191. 10.1016/j.physleta.2004.10.082
Mohamad S: Global exponential stability in continuous-time and discrete-time delayed bidirectional neural networks. Physica D 2001, 159: 233-251. 10.1016/S0167-2789(01)00344-X
Liang J, Cao JD, Ho D: Discrete-time bidirectional associative memory neural networks with variable delays. Phys. Lett. A 2005, 335: 226-234. 10.1016/j.physleta.2004.12.026
Liu X, Tang ML, Martin R, Liu BX: Discrete-time BAM neural networks with variable delays. Phys. Lett. A 2007, 367: 322-330. 10.1016/j.physleta.2007.03.037
Zhao H, Sun L, Wang G: Periodic oscillation of discrete-time bidirectional associative memory neural networks. Neurocomputing 2007, 70: 2924-2930. 10.1016/j.neucom.2006.11.010
Patan L: Local stability conditions for discrete-time cascade locally recurrent neural networks. Int. J. Appl. Math. Comput. Sci. 2010, 20(1):23-34.
Gaines RE, Mawhin JL: Coincidence Degree and Nonlinear Differential Equation. Springer, Berlin; 1977.
Berman A, Plemmons RJ: Nonnegative Matrices in the Mathematical Sciences. Academic Press, New York; 1979.
Chen HC, Hung YC, Chen CK, Liao TL, Chen CK: Image-processing algorithms realized by discrete-time neural networks and their circuit implementations. Chaos Solitons Fractals 2006, 29(5):1100-1108. 10.1016/j.chaos.2005.08.067
Wang JH: An improved discrete Hopfield neural network for Max-Cut problems. Neurocomputing 2006, 69(13-15):1665-1669. 10.1016/j.neucom.2006.02.001
Wang JH, Tang Z, Xu XS, Li Y: A discrete competitive Hopfield neural network for cellular channel assignment problems. Neurocomputing 2005, 67: 436-442.
Yashtini M, Malek A: A discrete-time neural network for solving nonlinear convex problems with hybrid constraints. Appl. Math. Comput. 2008, 195(2):576-584. 10.1016/j.amc.2007.05.034
Michele B, Leonarda C, Giuseppe G: Discrete-time cellular neural networks for associate memories with learning and forgetting capabilities. IEEE Trans. Circuits Syst. I 1995, 42(7):396-399. 10.1109/81.401156
Michel AN, Si J, Yen G: Analysis and synthesis of a class of discrete-time neural networks described on hypercubes. IEEE Trans. Neural Netw. 1991, 2(1):32-46. 10.1109/72.80289
McEliece RJ, Posner EC, Rodemich ER, Venkatsch SS: The capacity of the Hopfield associative memory. IEEE Trans. Inf. Theory 1998, 33: 461-482.
This study is supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20093401120001), the Natural Science Foundation of Anhui Province (No. 11040606M12) and the Natural Science Foundation of Anhui Education Bureau (No. KJ2010A035), the 211 project of Anhui University (No. KJJQ1102).
The authors declare that they have no competing interests.
RW directed the study, helped inspection, established the models, carried out the results of this article and drafted the manuscript. HX performed the numerical simulation. All the authors read and approved the final manuscript.