- Research
- Open Access
- Published:

# On the stability of the Cartesian product of a neural ring and an arbitrary neural network

*Advances in Difference Equations*
**volume 2014**, Article number: 176 (2014)

## Abstract

The stability of a system of neural networks connected to a ring has been studied extensively throughout recent years. Our main contribution within this work states that the stability region in the parameter space of a discrete-time model can be extended by breaking such a ring provided that there is a sufficiently large number of networks. Also, it has been shown that for a small ring, paradoxical values may appear within its parameter space for which such a ring is stable; meanwhile, corresponding linear configuration is unstable.

**MSC:**37B25.

## 1 Introduction

Many neural networks of artificial or natural origin, including the brain net, have a ring structure [1]. The stability of a ring neural network with delayed interactions has been studied in recent works such as [2–5]. In particular, [5] examined the breaking of a ring neural network into a linear neural network which gives an extended stability region in the parameter space provided that there is a sufficiently large number of neurons at the ring neural network. In this paper, we take such an approach to address the related question dealing with a discrete-model of ring consisting of identical (maybe complicated) networks. We characterize closely what happens with the stability of such rings after they are broken.

This paper is structured as follows. In Section 2, formal definitions of the Cartesian product of neural networks, ring and linear configuration of a network are stated. In Section 3, it is proven that by breaking a sufficiently large ring of neural networks, it does not lose its stability. Also an example of a small torus neural network, *i.e.* a ring consisting of small neural rings, is given. Hence, after two consecutive cut transformations, it yields a grid configuration. We show that there is a small region within the parameter space resulting in loss of stability in the breaking of the ring neural network. Such parameter values will be called paradoxical.

## 2 The Cartesian product of neural networks

Neural networks have been described either by nonlinear equations [6, 7] or by linear nonhomogeneous equations as it is done in [8]. Nonetheless, local stability analysis of steady states offers an interesting approach as we have chosen within this work. When linearized discrete-time neural networks models are considered, the state vector ${x}_{s}:{\mathbb{Z}}_{+}\to {\mathbb{R}}^{n}$ of a network at time *s* is governed by the following linear homogeneous equation (see [2, 6, 9]):

where *n* is the number of neurons in the network, $\gamma \in \mathbb{R}$ is a damping factor of neuron self-oscillations, $m\in {\mathbb{Z}}_{+}$ is a delay in the damping process of neuron self-oscillations, $k\in {\mathbb{Z}}_{+}$ is a delay in the neuron interactions ($k\u2a7em$). Entries of the matrix $A\in {\mathbb{R}}^{n\times n}$ represent interaction forces among *n* different neurons, thus that every entry at the principal diagonal of *A* will be zero. For every *j* ($1\u2a7dj\u2a7dn$), the *j* th component of ${x}_{s}$ is the state of the *j* th neuron at the moment *s*. The entry ${a}_{jv}$ of the matrix *A* is the force of action from the *v* th neuron to the *j* th neuron. We proceed to give formal definitions to neural networks and the Cartesian product of networks as follows.

**Definition 1** A neural network is an ordered 5-tuple $\mathcal{A}=(\gamma ,k,m,n,A)$, where $\gamma \in \mathbb{R}$, $k,m\in {\mathbb{Z}}_{+}$ ($k\u2a7em$), $A\in {\mathbb{R}}^{n\times n}$. We call (1) the defining equation of the network . We say that two neural networks are compatible if and only if they have the same *γ*, *k*, *m*. Given two compatibles networks ${\mathcal{A}}_{1}=(\gamma ,k,m,r,{A}_{1})$ and ${\mathcal{A}}_{2}=(\gamma ,k,m,n,{A}_{2})$, we define their Cartesian product as the neural network ${\mathcal{A}}_{1}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{A}}_{2}=(\gamma ,k,m,rn,{A}_{1}\oplus {A}_{2})$, where the Kronecker sum operation ⊕ is defined as follows: ${A}_{1}\oplus {A}_{2}={I}_{n}\otimes {A}_{1}+{A}_{2}\otimes {I}_{r}$, having ⊗ as the Kronecker product operation, and ${I}_{n}$, ${I}_{r}$ stand for the unit matrices of orders *n*, *r*, respectively.

These definitions do not contradict those given in [10, 11]. We also notice that the square block matrix ${A}_{1}\oplus {A}_{2}$ of order *rn* has the form

where ${a}_{jv}$ ($1\u2a7dj,v\u2a7dn$) are entries of ${A}_{2}$.

It is not hard to see that for any given neural network $\mathcal{A}=(\gamma ,k,m,n,A)$, its matrix *A* can be seen as a weighted directed graph $(V;E)$ with a set of vertices $V=\{1,2,\dots ,n\}$ and a set of directed edges *E* defined as follows: if an entry ${a}_{jv}$ of *A* is nonzero, then $(j,v)\in E$ and weighted by ${a}_{jv}$. Such a graph does not depend on *γ*, *k*, *m*.

For any given pair , of compatible networks, their Cartesian products $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}\mathcal{B}$ and $\mathcal{B}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}\mathcal{A}$ are isomorphic in the sense that one defining equation can be obtained from another by a straightforward permutation of ${x}_{s}$ components.

Now, let us consider the following example of ring and linear configurations of networks, both playing a crucial role in our main results.

**Example 1** Let ${C}_{n}(a,b)$ be an $n\times n$ circulant matrix for $n>3$, and let ${L}_{n}(a,b)$ be a tridiagonal matrix for $n>2$:

We define ${\mathcal{C}}_{n}(a,b)=(\gamma ,k,m,n,{C}_{n}(a,b))$ and ${\mathcal{L}}_{n}(a,b)=(\gamma ,k,m,n,{L}_{n}(a,b))$ as the ring and linear neural networks, respectively, where *b* is the strength of the connection between a neuron to its counterclockwise neighbor neuron, *a* is the strength of their opposite direction connection (Figure 1). We point out that ${\mathcal{C}}_{n}(a,b)$ has a connection between its first and last neuron, meanwhile ${\mathcal{L}}_{n}(a,b)$ has no connection between them.

It follows that ${\mathcal{C}}_{3}(a,b)\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{2}(c,d)$ has the defining equation (1) with

We state the following key property of the Kronecker sum.

*If* ${\lambda}_{j}$ ($1\u2a7dj\u2a7dr$) *is a full list of eigenvalues of an* $r\times r$ *matrix* ${A}_{1}$ *and* ${\mu}_{v}$ ($1\u2a7dv\u2a7dn$) *is the corresponding list for an* $n\times n$ *matrix* ${A}_{2}$, *then the eigenvalues of* ${A}_{1}\oplus {A}_{2}$ *are given by* ${\lambda}_{j}+{\mu}_{v}$ ($1\u2a7dj\u2a7dr$, $1\u2a7dv\u2a7dn$).

## 3 The stability of a ring of neural networks

Our main purpose is to study the stability of a ring and linear configuration of a neural network. Hence, we proceed to state straightforwardly stability definitions for the defining equation (1). We say that this equation is stable (asymptotically stable) if and only if every solution ${x}_{s}$ has a bounded norm (the sequence $|{x}_{s}|$ tends to zero as $s\to \mathrm{\infty}$). Quite often stability requirements of a system are adjusted [15, 16], we will state the following definitions along these lines. Given *ρ* is a positive real number, we say that equation (1) is *ρ*-stable (*ρ*-asymptotically stable) if and only if the sequence $|{x}_{s}|/{\rho}^{s}$ is bounded (the sequence $|{x}_{s}|/{\rho}^{s}$ tends to zero as $s\to \mathrm{\infty}$). Equations that are not *ρ*-stable (asymptotically *ρ*-stable) will be called *ρ*-unstable (asymptotically *ρ*-unstable). We should notice that when $\rho =1$, (asymptotic) *ρ*-stability is equivalent to the usual Lyapunov notion of (asymptotic) stability. Furthermore, stability cones [17, 18] for stability analysis of (1) will be extensively used in our analysis. Stability cones for stability analysis of differential delayed matrix equations were introduced in [19].

It is a plausible step to take the compatible network ${\mathcal{L}}_{n}(a,b)$ as the breaking of the network ${\mathcal{C}}_{n}(a,b)$. Now, let us consider to be an arbitrary neural network, then it follows that the network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$ is the compatible breaking of the ring $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ (Figure 2) in the sense that it is the resulting neural network after the breaking of all links between the first and the last copy of at the latter network. The stability of those neural networks involved along this process will be addressed in the following theorem.

**Theorem 2** *Let* $\mathcal{A}=(\gamma ,k,m,r,A)$, ${\mathcal{C}}_{n}(a,b)$ *and* ${\mathcal{L}}_{n}(a,b)$ *be compatible neural networks*, *obeying the condition* ${a}^{2}+{b}^{2}\ne 0$, *then for every* $\rho >0$, *there exists* ${n}_{0}$ *such that for all* $n>{n}_{0}$, *if* $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ *is* *ρ*-*stable*, *then* $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$ *is asymptotically* *ρ*-*stable*.

*Proof* Let $\mathcal{A}=(\gamma ,k,m,r,A)$ be a neural network and ${\lambda}_{1},\dots ,{\lambda}_{r}$ be the list of eigenvalues of *A*. We assume the condition ${a}^{2}+{b}^{2}\ne 0$ and that $k,m,\gamma ,\rho >0$ are fixed. It was shown in [12–14] that the set of values $(a+b)cos\frac{2\pi v}{n}+i(a-b)sin\frac{2\pi v}{n}$ are the eigenvalues of ${C}_{n}(a,b)$; in a similar fashion, the set of values $2\sqrt{ab}cos\frac{\pi v}{n+1}$, $1\u2a7dv\u2a7dn$, are the eigenvalues of ${L}_{n}$. By applying Theorem 1 and related stability analysis results from [18] over the neural networks $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ and $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$, we construct two sets of points as follows. Firstly, the set ${M}_{jv}=({u}_{1jv},{u}_{2jv},{u}_{3jv})$ ($1\u2a7dj\u2a7dr$, $1\u2a7dv\u2a7dn$) obeying

Secondly, the set of points ${P}_{jv}=({u}_{1jv}^{\prime},{u}_{2jv}^{\prime},{u}_{3jv}^{\prime})$ ($1\u2a7dj\u2a7dr$, $1\u2a7dv\u2a7dn$) obeying

We proceed with such a construction by an exhaustive case analysis over *a* and *b*.

CASE 1: $a\u2a7e0$, $b\u2a7e0$. Let us construct for every *j* ($1\u2a7dj\u2a7dr$) points ${M}_{j}^{1}=({u}_{1j}^{1},{u}_{2j}^{1},{u}_{3j}^{1})$ and ${M}_{j}^{2}=({u}_{1j}^{2},{u}_{2j}^{2},{u}_{3j}^{2})$ so that

CASE 1.1: There exists *j* ($1\u2a7dj\u2a7dr$) such that ${M}_{j}^{1}$ lies outside the *ρ*-stability cone for the given values of *k*, *m*. Then the point ${M}_{jn}$ (see (3)) lies outside the *ρ*-stability cone, therefore the network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ is *ρ*-unstable for every $n\u2a7e3$. So we can put ${n}_{0}=3$ in the conclusion of the theorem.

CASE 1.2: There exists *j* ($1\u2a7dj\u2a7dr$) such that ${M}_{j}^{2}$ lies outside the *ρ*-stability cone. Let us use the fact that $[n/2]/n$ approaches $1/2$ when $n\to \mathrm{\infty}$, $[z]$ being the integral part of *z*. We conclude from (3) that there exists an ${n}_{0}$ such that for every $n>{n}_{0}$, the point ${M}_{j[n/2]}$ lies outside the *ρ*-stability cone. Therefore the network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ is *ρ*-unstable for every $n>{n}_{0}$.

CASE 1.3: For all *j* ($1\u2a7dj\u2a7dr$), both ${M}_{j}^{1}$ and ${M}_{j}^{2}$ lie inside the *ρ*-stability cone or on its boundary. Since $2\sqrt{ab}<a+b$ (recall that ${a}^{2}+{b}^{2}\ne 0$), all the points ${P}_{jv}$ (see (4)) lie inside the line segment with the endpoints ${M}_{j}^{1}$, ${M}_{j}^{2}$ (see (5), (6)). But the section of the *ρ*-stability cone at the level ${u}_{3}=|\gamma |$ has the property of being convex, hence all the points ${P}_{jv}$ ($1\u2a7dj\u2a7dr$, $1\u2a7dv\u2a7dn$) lie inside the *ρ*-stability cone. Therefore the neural network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$ is asymptotically *ρ*-stable. This enables one to put ${n}_{0}=3$ in the conclusion of the theorem.

CASE 2: $a\u2a7d0$, $b\u2a7d0$. This case is similar to CASE 1.

CASE 3: $a>0$, $b<0$. For every *j* ($1\u2a7dj\u2a7dr$), let us construct points ${M}_{j}^{3}=({u}_{1j}^{3},{u}_{2j}^{3},{u}_{3j}^{3})$ and ${M}_{j}^{4}=({u}_{1j}^{4},{u}_{2j}^{4},{u}_{3j}^{4})$ such that

CASE 3.1: There exists *j* ($1\u2a7dj\u2a7dr$) such that ${M}_{j}^{3}$ lies outside the *ρ*-stability cone. If $n\to \mathrm{\infty}$, then $[n/4]/n\to 1/4$. Hence by (3) there exists ${n}_{0}$ such that for every $n>{n}_{0}$, the point ${M}_{j[n/4]}$ lies outside the *ρ*-stability cone. Therefore the network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ is *ρ*-unstable for every $n>{n}_{0}$.

CASE 3.2: There exists *j* ($1\u2a7dj\u2a7dr$) such that ${M}_{j}^{4}$ lies outside the *ρ*-stability cone. This case is similar to CASE 3.1, the only difference being in using $[3n/4]/n\to 3/4$ instead of $[n/4]/n\to 1/4$.

CASE 3.3: For all *j* ($1\u2a7dj\u2a7dr$), both ${M}_{j}^{3}$ and ${M}_{j}^{4}$ lie inside the *ρ*-stability cone or on its boundary. This case is similar to CASE 1.3, the only difference being in using $2\sqrt{|ab|}<a-b$ instead of $2\sqrt{ab}<a+b$.

CASE 4: $a<0$, $b>0$. This case is similar to CASE 3. Hence, our proof is completed. □

Considering the semigroup structure of all neural networks for *γ*, *k*, *m* fixed, it is not hard to see that the neural network $\mathcal{E}=(\gamma ,k,m,1,0)$ its identity element, the fact which entails that such a structure is really a commutative monoid. By replacing by ℰ in Theorem 2, we obtain an interesting consequence.

**Theorem 3** *Let* ${\mathcal{C}}_{n}(a,b)$ *and* ${\mathcal{L}}_{n}(a,b)$ *be compatible neural networks*, *obeying that* ${a}^{2}+{b}^{2}\ne 0$, *then for every* $\rho >0$, *there exists* ${n}_{0}$ *such that for all* $n>{n}_{0}$ *if* ${\mathcal{C}}_{n}(a,b)$ *is* *ρ*-*stable*, *then* ${\mathcal{L}}_{n}(a,b)$ *is asymptotically* *ρ*-*stable*.

A similar result to this corollary for a continuous-time neural network model was shown in [5]. We do remark that our main Theorem 2 states that in the case ${a}^{2}+{b}^{2}\ne 0$, the breaking of large ring neural networks extends the asymptotic stability domain in the parameter space providing a sufficiently large size. The latter is crucial to it, in fact it is no longer true when the number of networks in such a ring is not large enough. We will state adequate definitions and an example to support this issue.

**Definition 2** Let , ${\mathcal{C}}_{n}(a,b)$ and ${\mathcal{L}}_{n}(a,b)$ be pairwise compatible neural networks. Consider $(a,b)$ to be a point in the *ab*-plane; we call it paradoxical for both transformations $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)\to \mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$ and ${\mathcal{C}}_{n}(a,b)\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}\mathcal{A}\to {\mathcal{L}}_{n}(a,b)\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}\mathcal{A}$, if the network $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{n}(a,b)$ is asymptotically stable, and $\mathcal{A}\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{L}}_{n}(a,b)$ is unstable.

**Example 2** By considering ${\mathcal{C}}_{3}(a,b)\phantom{\rule{0.2em}{0ex}}\mathrm{\square}\phantom{\rule{0.2em}{0ex}}{\mathcal{C}}_{5}(a,b)$ be a toroidal neural network, significant changes in the stability domains can be shown after ${\mathcal{C}}_{3}(a,b)$ and ${\mathcal{C}}_{5}(a,b)$ are broken according to the following diagram

Now, by using the stability cones methods from [9, 17, 18], stability domains can be found for those networks involved in the previous diagram. It is not hard to see how in all four operations denoted by arrows in (9), the stability domains are significantly increased. Nevertheless, Figure 3 shows in detail how these operations create paradoxical points, for which the system loses stability after the ring has been broken.

## 4 Conclusion

In connection with the above investigations, some open problems arise. For example, in [5] a detailed analysis of appearance and disappearance of paradoxical points in a continuous-time model of neural ring networks was performed. Consequently, natural directions for future research are the analysis of these phenomena in our discrete-time model of neural networks. Moreover, in the future, we intend to examine relevant issues in neural networks with distributed delays.

## References

- 1.
Vishwanathan A, Bi GQ, Zeringue HC: Ring-shaped neuronal networks: a platform to study persistent activity.

*Lab Chip*2011, 11(6):1081–1088. 10.1039/c0lc00450b - 2.
Kaslik E: Dynamics of a discrete-time bidirectional ring of neurons with delay. In

*Proceedings of Int. Joint Conf. on Neural Networks*. IEEE Comput. Soc., Los Alamitos; 2009:1539–1546. Atlanta, Georgia, USA, 14–19 June 2009 - 3.
Kaslik E, Balint S: Complex and chaotic dynamics in a discrete-time delayed Hopfield neural network with ring architecture.

*Neural Netw.*2009, 22(10):1411–1418. 10.1016/j.neunet.2009.03.009 - 4.
Yuan Y, Campbell SA: Stability and synchronization of a ring of identical cells with delayed coupling.

*J. Dyn. Differ. Equ.*2004, 16: 709–744. 10.1007/s10884-004-6114-y - 5.
Khokhlova TN, Kipnis MM: The breaking of a delayed ring neural network contributes to stability: the rule and exceptions.

*Neural Netw.*2013, 48: 148–152. - 6.
Kaslik E, Balint S: Bifurcation analysis for a two-dimensional delayed discrete-time Hopfield neural network.

*Chaos Solitons Fractals*2007, 34: 1245–1253. 10.1016/j.chaos.2006.03.107 - 7.
Idels L, Kipnis M: Stability criteria for a nonlinear nonautonomous system with delays.

*Appl. Math. Model.*2009, 33(5):2293–2297. 10.1016/j.apm.2008.06.005 - 8.
Arbib M (Ed):

*The Handbook of Brain Theory and Neural Networks*. 2nd edition. MIT Press, Cambridge; 2002. - 9.
Ivanov SA, Kipnis MM: Stability analysis of discrete-time neural networks with delayed interactions: torus, ring, grid, line.

*Int. J. Pure Appl. Math.*2012, 78(5):691–709. - 10.
Imrich W, Klavžar S, Rall DF:

*Graphs and Their Cartesian Products*. AK Peters, Wellesley; 2008. - 11.
Brualdi RA, Cvetković DM:

*A Combinatorial Approach to Matrix Theory and Its Applications*. CRC Press, Boca Raton; 2008. - 12.
Horn RA, Johnson CR:

*Topics in Matrix Analysis*. Cambridge University Press, Cambridge; 1994. - 13.
Brouwer AE, Haemers WH:

*Spectra of Graphs*. Springer, Berlin; 2011. - 14.
Cvetković DM, Doob M, Sachs H:

*Spectra of Graphs - Theory and Applications*. 3rd edition. Wiley, New York; 1998. - 15.
Chestnov VN:Synthesis ${H}_{\mathrm{\infty}}$-controllers for multidimensional systems with given accuracy and degree of stability.

*Autom. Remote Control*2011, 72(10):2161–2175. 10.1134/S0005117911100134 - 16.
Gryazina EN, Polyak BT: Stability regions in the parameter space:

*D*-decomposition revisited.*Automatica*2006, 42(1):13–26. 10.1016/j.automatica.2005.08.010 - 17.
Kipnis MM, Malygina VV: The stability cone for a matrix delay difference equation.

*Int. J. Math. Math. Sci.*2011., 2011: Article ID 860326 - 18.
Ivanov SA, Kipnis MM, Malygina VV: The stability cone for a difference matrix equation with two delays.

*ISRN Appl. Math.*2011., 2011: Article ID 910936 - 19.
Khokhlova TN, Kipnis MM, Malygina VV: Stability cone for linear delay differential matrix equation.

*Appl. Math. Lett.*2011, 24: 742–745. 10.1016/j.aml.2010.12.020

## Acknowledgements

This work was supported by grants from the Ministry of Education of Russia, Chelyabinsk State Pedagogical University, Russia, and by the grant from Fondecyt No. 1130112, Chile.

## Author information

## Additional information

### Competing interests

The authors declare that they have no competing interests.

### Authors’ contributions

All authors contributed equally to the manuscript and typed, read, and approved the final manuscript.

## Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

## Rights and permissions

## About this article

#### Received

#### Accepted

#### Published

#### DOI

### Keywords

- stability
- discrete-time model
- ring of neural networks
- Cartesian product of networks
- the stability cone