Mean-square stability of Riemann–Liouville fractional Hopfield’s graded response neural networks with random impulses

In this paper a model of Hopfield’s graded response neural network is investigated. A network whose neurons are subject to a certain impulsive state displacement at random times is considered. The model is set up and studied. The presence of random moments of impulses in the model leads to a change of the solutions to stochastic processes. Also, we use the Riemann–Liouville fractional derivative to model adequately the long-term memory and the nonlocality in the neural networks. We set up in an appropriate way both the initial conditions and the impulsive conditions at random moments. The application of the Riemann–Liouville fractional derivative leads to a new definition of the equilibrium point. We define mean-square Mittag-Leffler stability in time of the equilibrium point of the model and study this type of stability. Some sufficient conditions for this type of stability are obtained. The general case with time varying self-regulating parameters of all units and time varying functions of the connection between two neurons is studied.


Introduction
The study of the dynamic behavior of neural networks has been investigated in many papers in the literature. Some researchers have found that in contrast to signal processing of integer-order models of neural networks, the application of fractional-order derivatives has significant advantages in modeling the problem. One of the most popular and influential types of neural networks is the classical first-order Hopfield neural network [12]. The long-term memory and the nonlocality of fractional derivatives allowed them to be incorporated into neural networks to describe better the behavior of the neurons connected with memory and heredity. Many researchers studied fractional neural networks and obtained many advantages over integer-order neural networks. For instance, Alofi et al. [6] studied the finite-time stability of Caputo fractional neural networks with distributed delay. Kaslik et al. [13] discussed the stability analysis of fractional-order neural networks of Hopfield type. It is worth mentioning that a novel conceptual framework of the fractional-order Hopfield neural network has been given in [18]. Several results concerning the implementation of the Caputo fractional derivative in the model have been obtained in the literature. We could mention, for example, the papers in [13,18,23,31] about Caputo fractional-order Hopfield type neural networks.
Note that the case of fractional-neural networks with Riemann-Liouville (RL) fractional derivatives is not well studied because of this type of derivative and the required initial condition (see, for example, [10,11,15,27,29]). At the same time, there are some inaccuracies when the RL fractional derivative is applied. For example, in papers [10,15,27] the RL fractional integral and the RL fractional derivative, respectively, are not well defined in the initial conditions associated with the RL fractional model with delay. In [30] the equilibrium point is defined by an equation which has only the zero solution (see Remarks 3 and 1).
Sometimes, neural networks are subject to some perturbations acting on a negligible small time. The models in this case are so-called impulsive models. In the case when the time of impulsive perturbations are initially given deterministic points, the impulsive models with ordinary derivatives for neural networks are studied in [9,19,22,25,26,33]. Note that impulsive fractional neural networks are studied for Caputo fractional derivative in several papers, such as [17,20,24,32], and for RL fractional derivative, see [28,29].
The presence of randomness in the neural networks could be incorporated in various ways. One of them is considering stochastic models (see, for example, the review paper [21] and the references cited therein). Another is considering impulsive perturbation in the neural networks occurring at random times (see, for example, [3,4]).
The main goal of this paper is to define and study for the first time the RL fractional generalization of a Hopfield neural network with impulses occurring at random times. Initially, a brief detailed explanation of the solutions being stochastic processes is provided. Then, the equilibrium, deeply connected with the RL fractional derivative, is defined (it is different than the equilibrium in impulsive Caputo fractional models). Note that the presence of the RL fractional derivative and the type of the initial condition require a new type of stability which excludes an appropriate neighborhood of the initial time point (it is different than in impulsive Caputo fractional models). This stability, called the meansquare Mittag-Leffler stability in time, is defined and studied in the paper. Some examples are provided to illustrate the introduced equilibrium, the defined fractional Dini derivative of Lyapunov functions as well as the practical application of the obtained sufficient conditions.
The main contributions in this paper could be summarized as follows: -the model of Hopfield's neural networks with RL fractional derivative and impulses at random times is defined; -the model with all variables in time coefficients is investigated; -both the initial conditions and the impulsive conditions are set up in an appropriate way; -the equilibrium point, deeply connected with the properties of RL fractional derivative, is defined; -appropriate types of stability, called mean-square Mittag-Leffler stability in time and eventual mean-square Mittag-Leffler stability in time, are defined and studied.

Preliminary notes on fractional derivatives and equations
Let t 0 ≥ 0 be a given number. In this paper we use the following definitions for fractional derivatives and integrals: -Riemann-Liouville (RL) fractional integral of order q ∈ (0, 1) (see [7,8,16]) where is the gamma function.
(a) If there exists a.e. a limit lim t→t 0 + [(tt 0 ) 1-q m(t)] = C, then there also exists a limit (b) If there exists a.e. a limit t 0 I 1-q t m(t)| t=t 0 = B and if there exists the limit Remark 1 Note that if m(t) ≡ C = const, then Let 0 ≤ a < b ≤ ∞ and consider the scalar RL fractional differential equation Remark 2 Note that according to [14] the initial conditions to (1) could be one of the following forms: weighted Cauchy type problem (see (3.1.7) [14]) satisfies the Volterra integral equation and vise verse.

Description of the RL fractional model with random impulses
We consider the model proposed by Hopfield [12] and known as Hopfield's graded response neural network. We generalize it in two ways: -the rate of change of the state variables of neurons will be modeled by an RL fractional derivative; -the neurons are subject to certain impulsive state displacements at random moments.

RL fractional model with fixed points of impulses
Initially, we consider the case of fixed initially given points of impulses {T k } ∞ k=1 with 0 < T k < T k+1 , k = 1, 2, . . . , and lim k→∞ T k = ∞, T 0 = 0. Consider the general model of RL fractional Hopfield's graded response neural networks with impulses occurring at fixed initially given times (INN): where n represents the number of neurons in the network, x i (t) is the pseudostate variable denoting the average membrane potential of the ith neuron at time t, x(t) = (x 1 (t), x 2 (t), . . . , x n (t)) ∈ R n , c i (t) > 0, i = 1, 2, . . . , n, is the self-regulating parameter of the ith unit, a ij (t), i, j = 1, 2, . . . , n, correspond to the synaptic connection strength of the ith neuron to the jth neuron at time t, f j (x j (t)) denotes the activation functions of the neurons at time t and represent the response of the jth neuron to its membrane potential, f (x) = (f 1 (x 1 ), f 2 (x 2 ), . . . , f n (x n )) is the activation function, and I(t) = (I 1 (t), I 2 (t), . . . , I n (t)) is the external bias vector.
The RL fractional derivative leads to a specific type of the initial condition of the model (see Remark 2 and Lemma 2.1): Let the average membrane potential of each neuron be subject to some instantaneous perturbations at times T k , k = 1, 2, . . . . Then model (4) will have impulses at times T k , k = 1, 2, . . . . Let us recall the meaning of impulses in differential equations. At the time point the state variable has a jump and then this state variable continues to behave according to the previously given dynamic equation. So, the value after the jump/impulse has a meaning of the initial value. In the case of ordinary derivatives and Caputo fractional derivatives, because of the type of the initial condition, the value of the impulse condition could be given in the form equaling to values of the state variable after the jump and before the jump, respectively, or similar one. In the case of the RL fractional derivative, some authors use this type of impulsive conditions (see, for example, [28]). But because of the type of the initial condition, we think that the above given type of the impulsive condition is not appropriate. Now, following the interpretation of impulses, Remark 2, and Lemma 2.1, we will define the impulsive conditions in the form: where the functions ψ i,k (u), k = 1, 2, . . . , are the impulsive functions giving the impulsive perturbation of the ith neuron at time T k , k = 1, 2, . . . . In our study we apply some results for the initial value problem for the scalar linear RL fractional differential equations with fixed points of impulses where u 0 ∈ R, a < 0.

RL fractional model with random impulses
Note that the fractional differential equations with impulses occurring at random times are studied in [1,3,5]. Similar to these papers, we will define now the RL fractional model with impulses at random times. Let the probability space ( , F, P) be given. Let a sequence of independent exponentially distributed random variables {τ k } ∞ k=1 with the same parameter λ > 0 defined on the sample space be given.
Define the sequence of random variables {ξ k } ∞ k=0 by The random variable τ k measures the waiting time of the kth impulse after the (k -1)th impulse occurs, and the random variable ξ k denotes the length of time until k impulses occur for t ≥ 0.
Let the points t k be arbitrary values of the corresponding random variables τ k , k = 1, 2, . . . . Define the increasing sequence of points T k = k i=1 t i , k = 1, 2, 3, . . . , that are values of the random variables ξ k .
Consider the initial value problem for INN with fixed points of impulses (4)- (6). The solution of the INN with fixed moments of impulses (4) depends not only on the initial value x 0 = (x 0 1 , x 0 2 , . . . , x 0 n ) but also on the moments of impulses T k , k = 1, 2, . . . , i.e., the solution depends on the chosen arbitrary values t k of the random variables τ k , k = 1, 2, . . . . We denote the solution of initial value problem (4) by The set of all solutions x(t; x 0 , {T k }) of INN (4)-(6) for any values t k of the random variables τ k , k = 1, 2, . . . , generates a stochastic process with state space R n . We denote it by x(t; x 0 , {τ k }), and we say that it is a solution of the general model of Hopfield's graded response neural networks with impulses occurring at random times (RINN), formally denoted by Now, similar to the papers [1, 3, 5], we will define the solution of the RL fractional model with impulses at random times.
Definition 1 Suppose that t k is a value of the random variable τ k , k = 1, 2, 3, . . . , and is called a sample path solution of the IVP for RINN (9) (here, T 0 = 0). Definition 2 A stochastic process x(t; x 0 , {τ k }) with an uncountable state space R n is said to be a solution of the IVP for RINN (9) if, for any values t k of the random variable τ k , k = 1, 2, 3, . . . , and T k = k i=1 t i , k = 1, 2, . . . , the corresponding function x(t; x 0 , {T k }) is a sample path solution of the IVP for INN (4)- (6).
According to Definition 2 any solution of IVP (7) is a sample path solution of the following scalar linear RL fractional differential equation with random moments of impulses: where u 0 ∈ R, a < 0.
where E(·) is the expected value of the stochastic process |u(t; u 0 , {τ k })| and it depends on the time t.

Equilibrium of RL fractional model with random impulses
The main problem in the definition for the equilibrium point of RL fractional model is based on the properties RL a D q Remark 1). These properties lead to a totally different definition of equilibrium of RL model (9), defined as follows. and hold. (11) is the main part of the definition for the equality of an equilibrium of any type of a model with RL fractional derivative. If the term 1 t q (1-q) x * i is missing in the definition for equilibrium, then it is correct only for zero equilibrium (see, for example, where the definition does not contain the mentioned term). In [27] the equilibrium point is defined as a function depending on time.

Remark 3 Equality
Remark 4 In the case of zero external bias vector, if zero is a fixed point of the activation function and the impulsive functions, then for any c i (t) the point 0 is an equilibrium point of RINN (9).
Remark 5 In the special case of the zero external bias vector and a i,j (t) = c i (t) + 1 t q (q) , i, j = 1, 2, . . . , n, any fixed points of the activation functions could form an equilibrium point, i.e., if x * i = f i (x * i ), then x * = (x * 1 , x * 2 , . . . , x * n ) could be an equilibrium of RINN (9). The equilibrium point depends not only on the RL fractional equations but also on the impulsive conditions. We will illustrate it in the following example.
Example 1 Consider the model of a neural network with three neurons with impulses at random times: and impulsive conditions Note that the point x * = (π, 2π, 3π) satisfies the system of RL fractional differential equations (13).
We assume the following: Assumption A1 Let RINN (9) have an equilibrium point x * ∈ R n .
If Assumption A1 is satisfied, then we can shift the equilibrium point x * of system (9) to the origin. The transformation y(t) = x(t)x * is used to put system (9) in the following form: for ξ k < t < ξ k+1 , k = 0, 1, . . . , i = 1, 2, . . . , n, where Remark 8 RINN (15) has a zero external bias vector, zero is a fixed point of both the activation functions F j (u), j = 1, 2, . . . , n, and the impulsive functions k,i (u), i = 1, 2, . . . , n, k = 1, 2, . . . , and according to Remark 5 the zero vector is an equilibrium of RINN (15). Therefore, if we know the equilibrium point x * ∈ R n of (9), then we can construct a model with zero equilibrium point. But oppositely, if the point y * = 0 is an equilibrium of RINN (15), then we are not able to obtain the equilibrium point x * ∈ R n of (9).

p-moment Mittag-Leffler stability in time for RL fractional model with random impulses
We define the mean-square Mittag-Leffler stability in time of the equilibrium point of RINN (9). This type of stability is deeply connected with the application of Mittag-Leffler functions with one parameter. Also, the presence of the RL fractional derivative and its singularity at the initial time leads to excluding this point from the interval of stability. This definition is similar to the definition of p-moment Mittag-Leffler stability in time, introduced for RL fractional delay differential equations by Agarwal et al. [5].

Definition 4
The equilibrium point x * of RINN (9) is said to be mean-square Mittag-Leffler stable in time if, for any > 0 and any initial value x 0 ∈ R n , there exists a constant α > 0 such that

Definition 5
The equilibrium point x * of RINN (9) is said to be eventually mean-square Mittag-Leffler stable if there exists a number T > 0 such that, for any > 0 and any initial value x 0 ∈ R n , there exists a constant α > 0 such that Remark 9 The above defined types of stability are mainly characterized by the corresponding inequalities, giving bounds that the solutions are satisfied out of an enough small neighborhood of the initial time. Note that it differs from the case of the Caputo fractional derivative and ordinary derivatives. This is because of the type of the initial condition which is deeply connected with the RL fractional derivative.
In the study of the stability properties of RINN(9) we use Lyapunov functions V (t, x) : [0, ∞) × R n → R + from the class ([0, ∞), R n ), i.e., they are continuous on [0, ∞) × R n and Lipschitzian with respect to their second vector argument.
We use the Dini fractional derivative of a Lyapunov function V (t, x), and we define it similarly to [2].
The fractional Dini derivatives along trajectories of solutions of RINN (9) are defined as follows: where x = (x 1 , x 2 , . . . , x n ). (16) to obtain the fractional Dini derivative of V along the trajectories of solutions of RINN (15): (-1) r q C r m(trh) Note that the fractional Dini derivative depends significantly not only on the order q of the fractional differential equation but also on the initial time (0 in our case).
We will introduce the following assumptions:

Assumption A2
The fractional order q ∈ (0, 1) of the RL fractional derivative is such that, for any > 0, the equation 1 holds.

Figure 2
Remark 11 The number q = 0.2, for example, satisfies condition A2 (see Fig. 1 and Fig. 2 for different values of ), but the number q = 0.8 does not satisfy Assumption A2 (see Fig. 3). Proof According to Remark 8, we study the behavior of the zero equilibrium of RINN (15).
Choose a positive number C such that According to Example 2, Remark 10, and Assumption A6, we have, for t ≥ 0, where 1, n = {1, 2, . . . , n}. Let x 0 ∈ R n be an arbitrary initial value and the stochastic process x(t; x 0 , {τ k }) be a solution of the initial value problem for RINN (9). Then the stochastic process y(t; x 0x * , {τ k }) is a solution of the initial value problem for RINN (15).
In the case inequality (17) is not satisfied for all t ≥ 0 but inequality (18) is satisfied for enough large t,we could prove the eventually mean-square Mittag-Leffler stability in time of the equilibrium. Proof According to Assumption A7, there exist a function m(t) and a number T such that m(t) > 0 for t > T. Choose a constant C > 0 such that Similar to the proof of Theorem 4.1, we consider the Lyapunov function V (t, x) = m(t) n i=1 x 2 i , x = (x 1 , x 2 , . . . , x n ) and prove inequality (19) for t > T. Let x 0 ∈ R n be an arbitrary initial value and the stochastic process x(t; x 0 , {τ k }) be a solution of the initial value problem for RINN (9) and y(t; x 0x * , {τ k }) be a solution of (15).

Conclusions
In this paper the RL fractional generalization of the first-order Hopfield neural network is studied in the case when some impulses occur at random times. We study the case when the waiting time between two consecutive times of impulses is exponentially distributed. In connection with the application of the RL fractional derivative in the model, we define in an appropriate way both the initial condition and the impulsive conditions. We define mean-square stability in time of the model and obtain some sufficient conditions.
In further work we hope to consider a number of directions: (i) Considering both fractional models, i.e., with the Caputo fractional derivative as well as the RL fractional derivative, and generalizing the waiting time between two consecutive impulses is Erlang distributed, is Log-normal distributed, etc. (ii) Generalizing the RL fractional model to various other types of delays.