The LS-SVM algorithms for boundary value problems of high-order ordinary differential equations

Abstract

This paper introduces the improved LS-SVM algorithms for solving two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations. To demonstrate the reliability and powerfulness of the improved LS-SVM algorithms, some numerical experiments for third-order, fourth-order linear and nonlinear ordinary differential equations with two-point and multi-point boundary conditions are performed. The idea can be extended to other complicated ordinary differential equations.

1 Introduction

High-order boundary value problems for ordinary differential equations are used to model different problems in some fields such as biology, economics, and engineering. Due to the importance of high-order ordinary differential equations, a considerable size of research work has been carried out about this problem. Among others, finite difference method [1] was proposed to solve two-point boundary value problems for high-order linear and nonlinear ordinary differential equations. Homotopy perturbation method [2, 3] was used for the solution of fourth-order and sixth-order boundary value problems. Ali [4] proposed the optimal homotopy asymptotic method to solve multi-point boundary value problems. Adomian decomposition method [5,6,7,8,9,10] was presented for solving two-point and multi-point boundary value problems of high-order ordinary differential equations. Haar wavelets method [11] and Shannon wavelet method [12] were proposed to solve boundary value problems of high-order ordinary differential equations. Doha [13] proposed spectral Galerkin algorithms based on Jacobi polynomials for solving two-point boundary value problems of third-order and fifth-order ordinary differential equations. Doha [14] proposed spectral Galerkin algorithms by using Chebyshev polynomials of the third and fourth kinds for solving high even-order differential equations. Shifted Jacobi collocation method [15] was proposed for solving nonlinear high-order multi-point boundary value problems. Saadatmandi and Dehghan [16] discussed sinc-collocation method for solving multi-point boundary value problems. Variational iteration method [17,18,19] was applied to solving two-point boundary value problems of high-order linear and nonlinear ordinary differential equations. Although these numerical methods provide good approximations to the solution, the approximate solution derivatives are discontinuous and can seriously affect the stability of the solution.

Neural network, which is one of machine intelligence techniques, has universal function approximation capabilities [20,21,22], and the solution obtained from the neural network is differentiable and in closed analytic form. Neural network has been widely used for solving ordinary differential equations [23, 24], partial differential equations [25,26,27], fractional differential equations [28,29,30], and integro-differential equations [31, 32]. Chakraverty and Mall [33] analyzed a regression-based neural network algorithm to solve two-point boundary value problems of fourth-order linear ordinary differential equations. Malek [34] proposed a novel hybrid method based on optimization techniques and feed forward artificial neural networks methods for two-point boundary value problems of fourth-order ordinary differential equations. Mai-Duy [35] discussed radial basis function networks for boundary value problems of high-order ordinary differential equations directly. However, artificial neural network has several drawbacks, such as the need for a large number of controlling parameters and the difficult choice of the number of hidden units. Furthermore, its training procedure is time-consuming and can be trapped in local minima.

SVM algorithms [36] were introduced by Vapnik in the framework of statistical learning theory. SVM algorithms map the input data into a high-dimensional feature space using a feature map. SVM algorithms can achieve a global optimum by solving a convex quadratic programming problem. Meanwhile, SVM algorithms adopt the structural risk minimization principle, which has a better generalization performance. LS-SVM algorithms [37] are a modification of SVM algorithms. LS-SVM algorithms change inequality constraints to equality constraints and regard the sum of squared errors loss function as experience loss of the training set. LS-SVM algorithms will deal with a set of linear equations instead of a quadratic optimization problem, which reduces the computation time of model learning significantly and improves higher solution accuracy. Therefore, LS-SVM algorithms have various applications in the area of pattern recognition [38], fault diagnosis [39], and time-series prediction [40, 41]. In addition, LS-SVM algorithms have been successfully applied for solving differential equations [42, 43], differential algebraic equations [44, 45], and integral equations [46].

LS-SVM algorithms are only used to solve two-point boundary value problems of second-order linear ordinary differential equations [42]. To the best of our knowledge, there are not too many results on LS-SVM algorithms for solving two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations. The main goal of the present thesis is to develop improved LS-SVM algorithms to solve two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations.

The remainder of this paper is organized as follows. First, Sect. 2 introduces least squares support vector machines. A brief overview of LS-SVM algorithms for solving ordinary differential equations is provided, and some definitions are given in Sect. 3. Following, in Sect. 4, the proposed LS-SVM algorithms for solving two-point boundary value problems of high-order linear and nonlinear ordinary differential equations and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations are discussed. In Sect. 5, we present five numerical examples to exhibit the accuracy and the efficiency of our proposed LS-SVM algorithms. Finally, concluding remarks are presented in Sect. 6.

2 Least squares support vector machines

Consider a given training data set $\{(x_{i}, y_{i})|x_{i}\in R^{n}, y _{i}\in R\}_{i=1}^{N}$ (in this paper $n=1$), where $\{x_{i}\}_{i=1} ^{N}$ are input data points and $\{y_{i}\}_{i=1}^{N}$ are the corresponding output data points. One assumes that the underlying function describing the relation between input points and output points has the following form:

$$ y(x)=\boldsymbol{\omega }^{T}\boldsymbol{\phi }({x)}+b, $$

(1)

where ω and b are parameters of the model that have to be determined and $\boldsymbol{\phi }({x)}$ is the nonlinear feature map which maps an input space into a higher dimensional feature space. Then, the optimal solution is sought in that space by minimizing the residual between the model outputs and the measurements [47]. To this end, the LS-SVM model in the primal is formulated as the following optimization problem [37, 48]:

$$ \min _{\boldsymbol{\omega },b, e _{i}} J(\boldsymbol{\omega },\boldsymbol{e})= \frac{1}{2} \boldsymbol{\omega }^{T}\boldsymbol{\omega }+ \frac{1}{2}\gamma \boldsymbol{e}^{T}\boldsymbol{e} $$

(2)

subject to

$$ y_{i}={\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{i})+b}+e_{i},\quad i=1,2,\ldots,N, $$

where γ is a positive regularization parameter and $e_{i}$ is the error of the ith input data. The first term is a regularization term, while the second one minimizes the training errors.

The optimization problem with equality constraints (2) can be solved by using the Lagrange multipliers method

$$ L(\boldsymbol{\omega }, b, \alpha _{i},e_{i} )= \frac{1}{2} \boldsymbol{\omega }^{T}\boldsymbol{\omega }+ \frac{1}{2}\gamma \boldsymbol{e}^{T}\boldsymbol{e}-\sum _{i=1}^{N}\alpha _{i}\bigl[{\boldsymbol{ \omega } ^{T} \boldsymbol{\phi }(x_{i})+b}+e_{i}-y_{i} \bigr], $$

(3)

where $\alpha _{i}$ ($i=1,2,\ldots,N$) are Lagrange multipliers that can be positive or negative in the LS-SVM formulation.

According to the KKT conditions, we will obtain

$$ \textstyle\begin{cases} \frac{\partial L}{\partial \boldsymbol{\omega }}= \boldsymbol{\omega }-\sum_{i=1}^{N}\alpha _{i} \boldsymbol{\phi }(x _{i})=0;\\ \frac{\partial L}{\partial b}=\sum_{i=1}^{N}\alpha _{i}=0; \\ \frac{\partial L}{\partial e_{i}}=\alpha _{i} -\gamma e_{i}=0;\\ \frac{\partial L}{ \partial \alpha _{i}}={\boldsymbol{\omega }^{T} \boldsymbol{\phi }(x _{i})+b}-y_{i}+e_{i}=0. \end{cases} $$

(4)

When ω and $e_{i}$ are eliminated from a system of Eq. (4), we obtain the following linear system:

(5)

where $\varTheta _{ij}=K(x_{i},x_{j})=\boldsymbol{\phi }(x_{i})^{T} \boldsymbol{\phi }(x_{j})$ ($i,j=1,2,\ldots,N$) is the ijth entry of the positive definite kernel matrix; $\boldsymbol{y}=[y_{1},y_{1},\ldots,y _{N}]^{T};\boldsymbol{\alpha }=[\alpha _{1},\alpha _{1},\ldots,\alpha _{N}]^{T}$ and $I_{N-1}=[1,1,\ldots,1]$.

Finally, the LS-SVM model in the dual form can be described as

$$ y(x)=\sum_{i=1}^{N}\alpha _{i}K(x_{i},x)+b. $$

(6)

3 Brief overview of LS-SVM model for solving ODEs and some definitions

In this section, a brief overview of LS-SVM algorithms for solving ordinary differential equations is provided and some definitions are given.

With regard to the initial value problem of the first-order linear ordinary differential equation in the following form [42]:

$$ \textstyle\begin{cases} \frac{dy}{dx}=a(x)y(x)+r(x),& x\in [a,c],\\ y(a)=A, \end{cases} $$

(7)

the authors in [42] assume that a general approximate solution is $y=\boldsymbol{\omega }^{T}\boldsymbol{\phi }({x)}+b$, where ω and b are the parameters to be solved. Then the interval $[a, c]$ is discretized into a series of collocation points by using collocation methods [49], and the optimal values of the parameters ω and b are obtained by solving the optimization problem with constraints, see [42]. According to the Lagrange multipliers method [50], the optimization problem with constraints is transformed into the Lagrangian function which is composed of the LS-SVM cost function and constraints that the approximate solution $y=\boldsymbol{\omega }^{T}\boldsymbol{\phi }({x)}+b$ satisfies the given first-order linear ordinary differential equation and the initial condition at the collocation points. The described methodology is applicable for solving other types of differential equations including second-order boundary value problems, partial differential equations, and descriptor systems [42,43,44].

The feature map ϕ is not explicitly known in general, so the kernel function will be introduced. By utilizing Mercer’s theorem [36], the derivative of the kernel function is defined as [42, 44]

$$ \nabla _{n,m}\bigl(K(x_{i},x_{j})\bigr)= \frac{\partial ^{n+m}(K(u,v))}{\partial u ^{n}\partial v^{m}}\bigg|_{u=x_{i},v=x_{j}}=\boldsymbol{\phi }^{(n)}(x_{i})^{T} \boldsymbol{\phi }^{(m)}(x_{j})=[\varTheta _{n,m}]_{i,j}. $$

(8)

In this paper, the RBF kernel $K(u,v)=\exp (-(u-v)^{2}/\sigma ^{2})$ is considered as a kernel function, then we can obtain

$$\begin{aligned}& \begin{aligned} \nabla _{1,3} \bigl(K(x_{i},x_{j})\bigr) &=\frac{\partial ^{4}(K(u,v))}{\partial u \partial v^{3}}\bigg|_{u=x_{i},v=x_{j}}= \boldsymbol{\phi }^{(1)}(x_{i})^{T} \boldsymbol{\phi }^{(3)}(x_{j})=[\varTheta _{1,3}]_{i,j} \\ &=- \biggl[\frac{12}{ \sigma ^{4}}-\frac{12}{\sigma ^{2}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{2}+\biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{4} \biggr]K(x _{i},x_{j}); \end{aligned} \\& \begin{aligned} \nabla _{2,3}\bigl(K(x_{i},x_{j})\bigr)&= \frac{\partial ^{5}(K(u,v))}{ \partial u^{2}\partial v^{3}}\bigg|_{u=x_{i},v=x_{j}}=\boldsymbol{\phi } ^{(2)}(x_{i})^{T} \boldsymbol{\phi }^{(3)}(x_{j})=[\varTheta _{2,3}]_{i,j} \\ &= \biggl[\frac{60}{\sigma ^{4}}-\frac{20}{\sigma ^{2}} \biggl[\frac{2(x _{i}-x_{j})}{\sigma ^{2}} \biggr]^{2}+\biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{4} \biggr] \frac{2(x_{i}-x_{j})}{\sigma ^{2}}K(x_{i},x_{j}); \end{aligned} \\& \begin{aligned} \nabla _{3,3}\bigl(K(x_{i},x_{j})\bigr)={}& \frac{\partial ^{6}(K(u,v))}{\partial u ^{3}\partial v^{3}}\bigg|_{u=x_{i},v=x_{j}}=\boldsymbol{\phi }^{(3)}(x_{i})^{T} \boldsymbol{\phi }^{(3)}(x_{j})=[\varTheta _{3,3}]_{i,j}\\ ={}& \biggl[\frac{120}{ \sigma ^{6}} -\frac{180}{\sigma ^{4}} \biggl[\frac{2(x_{i}-x_{j})}{ \sigma ^{2}} \biggr]^{2}+ \frac{30}{\sigma ^{2}} \biggl[\frac{2(x_{i}-x_{j})}{ \sigma ^{2}} \biggr]^{4}-\biggl[ \frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{6} \biggr]\\ &{}\times K(x_{i},x_{j}); \end{aligned} \\& \begin{aligned} \nabla _{3,4}\bigl(K(x_{i},x_{j})\bigr)={}& \frac{\partial ^{7}(K(u,v))}{\partial u^{3}\partial v^{4}}\bigg|_{u=x_{i},v=x_{j}}= \boldsymbol{\phi }^{(3)}(x_{i})^{T} \boldsymbol{\phi }^{(4)}(x_{j})=[ \varTheta _{3,4}]_{i,j} \\ ={}& \biggl[\frac{840}{\sigma ^{6}}-\frac{420}{\sigma ^{4}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{2}+\frac{42}{\sigma ^{2}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{4}- \biggl[\frac{2(x _{i}-x_{j})}{\sigma ^{2}} \biggr]^{6} \biggr] \\ &{}\times\frac{2(x_{i}-x_{j})}{\sigma ^{2}}K(x_{i},x_{j}); \end{aligned} \\& \begin{aligned} \nabla _{4,4}\bigl(K(x_{i},x_{j})\bigr)={}& \frac{\partial ^{8}(K(u,v))}{\partial u^{4}\partial v^{4}}\bigg|_{u=x_{i},v=x_{j}}= \boldsymbol{\phi }^{(4)}(x_{i})^{T} \boldsymbol{\phi }^{(4)}(x_{j})=[ \varTheta _{4,4}]_{i,j}\\ ={} &\biggl[\frac{1680}{\sigma ^{8}} -\frac{3360}{ \sigma ^{6}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{2}+ \frac{840}{ \sigma ^{4}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{4}- \frac{56}{ \sigma ^{2}} \biggl[\frac{2(x_{i}-x_{j})}{\sigma ^{2}} \biggr]^{6} \\ &{}+\biggl[\frac{2(x _{i}-x_{j})}{\sigma ^{2}} \biggr]^{8} \biggr]K(x_{i},x_{j}). \end{aligned} \end{aligned}$$

4 Boundary value problems of high-order ordinary differential equations

In this section, we formulate the improved LS-SVM algorithms to the solution of two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations.

4.1 Two-point boundary value problems of high-order ordinary differential equations

The improved LS-SVM algorithms to the solution of two-point boundary value problems of high-order linear and nonlinear ordinary differential equations are described.

4.1.1 Nonlinear ordinary differential equations for two-point boundary value problems

Two-point boundary value problems of Mth-order nonlinear ordinary differential equations to be solved can be stated as follows:

$$ \frac{d^{M}y}{dx^{M}}+a_{M-1}(x)\,\frac{d^{M-1}y}{dx^{M-1}}+\cdots+a _{1}(x)\,\frac{dy}{dx}=f(x,y), \quad x \in [a,c], $$

(9)

subject to boundary conditions $y^{(s)}(a)=p_{s}$, $y^{(r)}(c)=q_{r}$, $0\leq s\leq S$, $0\leq r\leq R$, $R=M-2-S$.

The interval $[a, c]$ is discretized into a series of collocation points $\varOmega = \{a = x_{1} < x_{2} < \cdots < x_{N} = c\}$. Assume that a general approximate solution to (9) is $y=\boldsymbol{\omega }^{T} \boldsymbol{\phi }({x)}+b$. The optimal values of the parameters ω and b are obtained by the following optimization problem:

$$ \min _{\boldsymbol{\omega }, b, \boldsymbol{e}, \boldsymbol{\xi }, y _{i}} J(\boldsymbol{\omega },\boldsymbol{e}, \boldsymbol{\xi })= \frac{1}{2}\boldsymbol{\omega }^{T} \boldsymbol{\omega }+\frac{1}{2} \gamma \boldsymbol{e}^{T} \boldsymbol{e}+\frac{1}{2}\gamma \boldsymbol{\xi }^{T}\boldsymbol{ \xi } $$

(10)

subject to

$$\begin{aligned} &\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=1} ^{M-1} \boldsymbol{\omega }^{T}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i}) =f(x_{i},y_{i})+e_{i}, \quad i=2,\ldots,N-1; \\ &y_{i}={\boldsymbol{\omega } ^{T}\boldsymbol{\phi }(x_{i})+b}+\xi _{i},\quad i=2,\ldots,N-1; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{1})+b=p_{0}; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{N})+b=q_{0}; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(s)}(x_{1})=p_{s}, \quad s=1,2,\ldots,S; \\ &\boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(r)}(x_{N})=q_{t}, \quad r=1,2,\ldots,R. \end{aligned}$$

Theorem 1

Given a positive definite kernel function $K: R\times R\rightarrow R$ and a regularization parameter $\gamma \in R^{+}$, the solution to (10) is given by the following dual problem:

(11)

where $[\widehat{\varTheta }_{l,l'}]_{N-2}=[ \widetilde{\varTheta }_{M,M}]_{N-2}+ \overline{D}_{a_{l}}[ \overline{\varTheta }_{l,M}]_{N-2}+[\overline{ \varTheta }_{M,l'}]_{N-2}\overline{D}_{a_{l'}}^{T} + \overline{D}_{a_{l}}[\overline{ \varTheta }_{l,l'}]_{N-2} \overline{D}^{T}_{a_{l'}} +\gamma ^{-1}E$; $[\overline{ \varTheta }_{M,l'}]_{N-2}= [[\widetilde{\varTheta }_{M,1}]_{N-2},[ \widetilde{ \varTheta }_{M,2}]_{N-2},\ldots,[\widetilde{\varTheta }_{M,M-1}]_{N-2} ]$; $[\overline{\varTheta }_{l,M}]_{N-2}= [[\widetilde{ \varTheta } _{1,M}]_{N-2};[\widetilde{\varTheta }_{2,M}]_{N-2}; \ldots;[ \widetilde{\varTheta }_{M-1,M}]_{N-2} ]$; $\overline{D}_{a_{l}}=[D_{a _{1}}, D_{a_{2}}, \ldots,D_{a_{M-1}}]$; $[\overline{\varTheta }_{l,l'}]_{N-2}=[ \widetilde{\varTheta }_{1:M-1,1:M-1}]_{N-2};\overline{D}_{a_{l'}}=[D_{a _{1}}, D_{a_{2}}, \ldots,D_{a_{M-1}}]$; $[\widetilde{\varTheta }_{0,0}]_{N-2}=[ \varTheta _{0,0}]_{2:N-1,2:N-1}+\gamma ^{-1}E$; $l,l'=1,2,\ldots,M-1$; $\boldsymbol{\alpha }=[\alpha _{2},\alpha _{3}, \ldots, \alpha _{N-1}]^{T}$; $D _{a_{l'}}=\operatorname{diag}(a_{l'}(t_{2}),a_{l'}(t_{3}), \ldots,a_{l'}(t_{N}))$; $D_{a _{l}}= \operatorname{diag}(a_{l}(t_{2}),a_{l}(t_{3}), \ldots,a_{l}(t_{N}))$; $[ \widehat{\varTheta }_{0,l'}]_{N-2}=[\widetilde{\varTheta }_{0,M}]_{N-2}+[\overline{ \varTheta }_{0,l'}]_{N-2} \overline{D}_{a_{l'}}^{T}$; $\boldsymbol{I}_{1,N-2}=[1,1, \ldots,1]$; $\boldsymbol{\beta }=[\beta _{0},\beta _{1}, \ldots,\beta _{S}]^{T}$; $[\overline{ \varTheta }_{0,l'}]_{N-2}= [[\widetilde{ \varTheta }_{0,1}]_{N-2},[ \widetilde{\varTheta }_{0,2}]_{N-2},\ldots,[\widetilde{\varTheta }_{0,M-1}]_{N-2} ]$; $\boldsymbol{\eta }=[\eta _{2},\eta _{3},\ldots,\eta _{N-1}]^{T}$; $[\widehat{\varTheta }_{s,l'}^{1} ]_{N-2}=[\widetilde{\varTheta }_{0:S,M} ^{1} ]_{N-2}+[\overline{\varTheta }_{0:S,l'}^{1} ]_{N-2}\overline{D}_{a _{l'}}^{T}$; $[\widehat{ \varTheta }_{r,l'}^{N}]_{N-2}=[\widetilde{ \varTheta } _{0:R,M}^{N}]_{N-2}+[\overline{ \varTheta }_{0:R,l'}^{N}]_{N-2} \overline{D}_{a_{l'}}^{T}$; $[\overline{\varTheta }_{0:S,l'}^{1}]_{N-2}= [[\widetilde{\varTheta }_{0:S,1}^{1} ]_{N-2}, [\widetilde{\varTheta }_{0:S,2} ^{1} ]_{N-2},\ldots,[\widetilde{\varTheta }_{0:S,M-1}^{1} ]_{N-2} ]$; $\boldsymbol{q}=[q_{0},q_{1},q_{2}, \ldots,q_{R}]^{T}$; $[\overline{ \varTheta }_{0:R,l'}^{N}]_{N-2}= [[\widetilde{\varTheta }_{0:R,1}^{N} ]_{N-2},[ \widetilde{\varTheta }_{0:R,2}^{N} ]_{N-2}, \ldots,[\widetilde{\varTheta }_{0:R,M-1} ^{N}]_{N-2} ]$; $\boldsymbol{\lambda }=[\lambda _{0},\lambda _{1}, \lambda _{2},\ldots,\lambda _{R}]^{T}$; $[\overline{\varTheta }_{s,0}^{1}]_{N-2}= [ \widetilde{\varTheta }_{0:S,0}^{1}]_{N-2}$; $[\overline{\varTheta }_{r,0}^{N}]_{N-2}= [ \widetilde{\varTheta }_{0:R,0}^{N}]_{N-2}$; $\boldsymbol{y}=[y_{2},y_{3},\ldots,y _{N-1}]$; $0_{s,N-2}=0_{S+1,N-2}$; $[\widetilde{\varTheta }_{s,s'}]_{1,1}=[ \varTheta _{0:S,0:S}]_{1,1}$; $[\widetilde{\varTheta }_{r,s'}]_{N,1}=[ \varTheta _{0:R,0:S}]_{N,1}$; $[\widetilde{\varTheta }_{r,r'}]_{N,N}=[ \varTheta _{0:R,0:R}]_{N,N}$; $f_{N-2}(\boldsymbol{x},\boldsymbol{y})=[f(x _{2},y_{2}),f(x_{3},y_{3}),\ldots,f(x_{N-1},y_{N-1}) ]^{T}$; $\boldsymbol{p}=[p _{0},p_{1},p_{2}, \ldots,p_{S}]^{T}$; $\frac{\partial f(\boldsymbol{x}, \boldsymbol{y})}{\partial y}= [\frac{\partial f(x,y)}{\partial y} |_{x=x_{2},y=y_{2}}, \frac{\partial f(x,y)}{\partial y}|_{x=x _{3},y=y_{3}},\ldots,\frac{\partial f(x,y))}{\partial y}|_{x=x_{N-1},y=y _{N-1}} ]$; $D_{N-2}(\boldsymbol{y})= \operatorname{diag}(\frac{\partial f( \boldsymbol{x},\boldsymbol{y})}{\partial y} )$; $\boldsymbol{1}_{r}=[1;0;\ldots;0]_{R+1,1}$; $\boldsymbol{1}_{s}=[1;0;\ldots;0]_{S+1,1}$; $0_{r,N-2}=0_{R+1,N-2}$; where $[\widetilde{\varTheta }_{m,n}]_{N-2}=[\varTheta _{m,n}]_{2:N-1,2:N-1}$, $[\widetilde{\varTheta }_{0:S,n}^{1}]_{N-2}= [[\varTheta _{0:S,n}]_{1,2},[ \varTheta _{0:S,n}]_{1,3},\ldots,[\varTheta _{0:S,n}]_{1,N-1} ]$ and $[\widetilde{\varTheta }_{0:R,n}^{N}]_{N-2}= [[\varTheta _{0:R,n}]_{N,2},[ \varTheta _{0:R,n}]_{N,3},\ldots,[\varTheta _{0:R,n}]_{N,N-1} ]$, $m,n=0,1,\ldots,M$.

Proof

Consider the Lagrangian function of the optimization problem (10):

$$\begin{aligned} &L(\boldsymbol{\omega }, y_{i},\alpha _{i},\eta _{i},\beta _{0},\beta _{s}, \lambda _{0},\lambda _{r},b,e_{i},\xi _{i}) \\ &\quad =\frac{1}{2} \boldsymbol{\omega }^{T}\boldsymbol{\omega }+\frac{1}{2}\gamma \boldsymbol{e}^{T}\boldsymbol{e}+\frac{1}{2}\gamma \boldsymbol{\xi } ^{T}\boldsymbol{\xi } \\ &\qquad {}-\sum_{i=2}^{N-1} \alpha _{i} \Biggl[ \boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=1} ^{M-1}\boldsymbol{\omega }^{T}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i}) -f(x_{i},y_{i})-e_{i} \Biggr] \\ &\qquad {}-{\sum_{i=2}^{N-1}\eta _{i}\bigl( \boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{i})+b}+\xi _{i} -y _{i}\bigr)-\beta _{0}\bigl(\boldsymbol{\omega ^{T}}\boldsymbol{\phi }(x_{1})+b-p _{0}\bigr)- \sum_{s=1}^{S}\beta _{s}\bigl( \boldsymbol{\omega ^{T}} \boldsymbol{\phi }^{(s)}(x_{1})-p_{s} \bigr) \\ &\qquad {}-\lambda _{0}\bigl( \boldsymbol{\omega ^{T}} \boldsymbol{\phi }(x_{N})+b -q_{0}\bigr)-\sum_{r=1}^{M-2-S} \lambda _{r}\bigl(\boldsymbol{\omega ^{T}}\boldsymbol{\phi } ^{(r)}(x_{N})-q_{r}\bigr). \end{aligned}$$

(12)

Then the KKT optimality conditions are given by

$$\begin{aligned}& \begin{gathered} \begin{aligned}\frac{\partial L}{\partial \boldsymbol{\omega }}={}& \boldsymbol{\omega }-\sum_{i=2}^{N-1}\alpha _{i} \Biggl[ \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=1}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr]-\sum _{i=2}^{N-1}\eta _{i} \boldsymbol{\phi }(x_{i})-\beta _{0}\boldsymbol{\phi }(x_{1}) \\ & {}-\sum_{s=1}^{S}\beta _{s}\boldsymbol{\phi }^{(s)}(x_{1})-\lambda _{0} \boldsymbol{\phi }(x_{N})-\sum _{r=1}^{M-2-S}\lambda _{r} \boldsymbol{\phi }^{(r)}(x_{N})=0; \end{aligned} \\ \frac{\partial L}{\partial \alpha _{i}}=\boldsymbol{\omega }^{T} \Biggl[\boldsymbol{\phi }^{(M)}(x _{i})+\sum_{l=1}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr]-f(x_{i},y_{i})-e_{i}=0,\quad i=2,3, \ldots,N-1; \\ \frac{\partial L}{ \partial \eta _{i}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{i})+b+ \xi _{i}-y_{i}=0,\quad i=2,3,\ldots,N-1; \\ \frac{\partial L}{\partial \beta _{0}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{1})+b-p_{0}=0; \\ \frac{ \partial L}{\partial \beta _{s}}=\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(s)}(x_{1})-p_{s}=0,\quad s=1,2,\ldots,S; \\ \frac{\partial L}{\partial \lambda _{0}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x _{N})+b-q_{0}=0; \\ \frac{\partial L}{\partial \lambda _{r}}=\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(r)}(x_{N})-q_{r}=0,\quad r=1,2, \ldots,M-2-S; \\ \frac{ \partial L}{\partial b}=-\sum_{i=2}^{N-1}\eta _{i}-\beta _{0}-\lambda _{0}=0; \\ \frac{\partial L}{\partial y_{i}}=\eta _{i}+\alpha _{i} \frac{ \partial f(x_{i},y_{i})}{\partial y_{i}}=0,\quad i=2,3,\ldots,N-1; \\ \frac{ \partial L}{\partial e_{i}}=\alpha _{i}+\gamma e_{i}=0,\quad i=2,3, \ldots,N-1; \\ \frac{\partial L}{\partial \xi _{i}}=-\eta _{i}+\gamma \xi _{i}=0,\quad i=2,3, \ldots,N-1. \end{gathered} \end{aligned}$$

(13)

Finally, rewriting the above system in matrix form will result in (11). □

System (11) is solved by Newton’s method. Therefore, the LS-SVM model in the dual form becomes

$$ \begin{aligned}[b] \hat{y}(x) ={}&b+\sum _{i=2}^{N-1}\alpha _{i} \Biggl[\nabla _{M,0}\bigl(K(x_{i},x)\bigr)+ \sum _{l=1}^{M-1}a_{l}(x_{i})\nabla _{l,0}\bigl(K(x_{i},x)\bigr) \Biggr]\\ &{}+\sum _{i=2} ^{N-1}\eta _{i} \nabla _{0,0}\bigl(K(x_{i},x)\bigr) +\beta _{0} \nabla _{0,0}\bigl(K(x _{1},x)\bigr)+\sum _{s=1}^{S}\beta _{s}\nabla _{s,0}\bigl(K(x_{1},x)\bigr)\\ &{}+\lambda _{0} \nabla _{0,0}\bigl(K(x_{N},x)\bigr)+\sum _{r=1}^{M-2-S}\lambda _{r}\nabla _{r,0}\bigl(K(x _{N},x)\bigr). \end{aligned} $$

(14)

4.1.2 Linear ordinary differential equations for two-point boundary value problems

Two-point boundary value problems of Mth-order linear ordinary differential equations to be solved can be stated as follows:

$$ \frac{d^{M}y}{dx^{M}}+a_{M-1}(x)\,\frac{d^{M-1}y}{dx^{M-1}}+\cdots+a _{1}(x)\,\frac{dy}{dx}+a_{0}(x)y=r(x), \quad x \in [a,c], $$

(15)

subject to boundary conditions $y^{(s)}(a)=p_{s}$, $y^{(r)}(c)=q_{r}$, $0\leq s\leq S$, $0\leq r\leq R$, $R=M-2-S$.

Assume that a general approximate solution to (15) is $y= \boldsymbol{\omega }^{T}\boldsymbol{\phi }({x)}+b$. To obtain the optimal values of the parameters ω and b, collocation methods which discretize the interval $[a, c]$ into a series of collocation points $\varOmega = \{a = x_{1} < x_{2} < \cdots < x_{N} = c \}$ can be used. Therefore, these parameters are obtained by solving the following optimization problem:

$$ \min _{\boldsymbol{\omega }, b, e_{i}} J( \boldsymbol{\omega },\boldsymbol{e})= \frac{1}{2}\boldsymbol{\omega } ^{T}\boldsymbol{\omega }+ \frac{1}{2}\gamma \boldsymbol{e}^{T} \boldsymbol{e} $$

(16)

subject to

$$ \begin{aligned} &\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=0} ^{M-1} \boldsymbol{\omega }^{T}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i})+a_{0}(x_{i})b=r(x_{i})+e_{i},\quad i=2,\ldots,N-1; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{1})+b=p_{0}; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{N})+b=q_{0}; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(s)}(x_{1})=p_{s},\quad s=1,2, \ldots,S; \\ &\boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(r)}(x_{N})=q_{t},\quad r=1,2, \ldots,R. \end{aligned} $$

Theorem 2

Given a positive definite kernel function $K: R\times R\rightarrow R$ and a regularization parameter $\gamma \in R^{+}$, the solution to (16) is obtained by the following dual problem:

(17)

where $[\widehat{\varTheta }_{l,l'}]_{N-2}=[ \widetilde{\varTheta }_{M,M}]_{N-2}+ \overline{D}_{a_{l}}[ \overline{\varTheta }_{l,M}]_{N-2}+[\overline{ \varTheta }_{M,l'}]_{N-2}\overline{D}_{a_{l'}}^{T} + \overline{D}_{a_{l}}[\overline{ \varTheta }_{l,l'}]_{N-2}\overline{D}^{T}_{a_{l'}}+\gamma ^{-1}E$; $[\overline{ \varTheta }_{M,l'}]_{N-2}= [[\widetilde{\varTheta }_{M,0}]_{N-2},[ \widetilde{\varTheta }_{M,1}]_{N-2}, \ldots,[\widetilde{\varTheta }_{M,M-1}]_{N-2} ]$; $\overline{D}_{a_{l'}}=[D_{a_{0}},D_{a_{1}}, \ldots,D_{a_{M-1}}]$; $[\overline{ \varTheta }_{l,M}]_{N-2}= [[\widetilde{\varTheta }_{0,M}]_{N-2}; [ \widetilde{ \varTheta }_{1,M}]_{N-2};\ldots;[\widetilde{\varTheta }_{M-1,M}]_{N-2} ]$; $\overline{D}_{a_{l}}=[D_{a_{0}},D_{a_{1}}, \ldots,D_{a_{M-1}}]$; $[\overline{ \varTheta }_{l,l'}]_{N-2}=[ \widetilde{\varTheta }_{0:M-1,0:M-1}]_{N-2}$; $D _{a_{l'}}=\operatorname{diag}(a_{l'}(t_{2}),a_{l'}(t_{3}), \ldots,a_{l'}(t_{N-1}))$; $l,l'=0,1,\ldots, M-1$; $D_{a_{l}}=\operatorname{diag}(a_{l}(t_{2}),a_{l}(t_{3}), \ldots,a_{l}(t_{N-1}))$; $\boldsymbol{p}=[p_{0},p_{1},p_{2}, \ldots,p_{S}]^{T}$; $\boldsymbol{q}=[q _{0},q_{1},q_{2}, \ldots,q_{R}]^{T}$; $[\widehat{\varTheta }_{s,l'}^{1}]_{N-2}= [ \widetilde{\varTheta }_{0:S,M}^{1}]_{N-2}+ [\overline{\varTheta }_{0:S,l'} ^{1}]_{N-2} \overline{D}_{a_{l'}}^{T}$; $[\widehat{\varTheta }_{r,l'}^{N}]_{N-2}=[ \widetilde{\varTheta }_{0:R,M}^{N}]_{N-2}+[\overline{\varTheta }_{0:R,l'} ^{N}]_{N-2}\overline{D}_{a_{l'}}^{T}$; $[\overline{\varTheta }_{0:S,l'} ^{1} ]_{N-2}= [[\widetilde{\varTheta }_{0:S,0}^{1} ]_{N-2},[ \widetilde{\varTheta }_{0:S,1}^{1} ]_{N-2},\ldots,[\widetilde{\varTheta }_{0:S,M-1} ^{1}]_{N-2} ]$; $\boldsymbol{1}_{s}=[1;0; \ldots;0]_{S+1,1}$; $[\overline{ \varTheta }_{0:R,l'}^{N}]_{N-2}= [[\widetilde{\varTheta }_{0:R,0}^{N} ]_{N-2},[ \widetilde{\varTheta }_{0:R,1}^{N} ]_{N-2},\ldots,[\widetilde{\varTheta }_{0:R,M-1} ^{N}]_{N-2} ]$; $\boldsymbol{\alpha }=[\alpha _{2},\alpha _{3},\ldots, \alpha _{N-1}]^{T}$; $[\widetilde{\varTheta }_{s,s'}]_{1,1}=[\varTheta _{0:S,0:S}]_{1,1}$; $[ \widetilde{\varTheta }_{r,s'}]_{N,1}=[ \varTheta _{0:R,0:S}]_{N,1}$; $[ \widetilde{\varTheta }_{r,r'}]_{N,N}=[\varTheta _{0:R,0:R}]_{N,N}$; $\boldsymbol{\beta }=[\beta _{0},\beta _{1},\beta _{2},\ldots,\beta _{S}]^{T}$; $\boldsymbol{\lambda }=[\lambda _{0},\lambda _{1},\lambda _{2}, \ldots, \lambda _{R}]^{T}$; $\boldsymbol{1}_{r}=[1;0; \ldots;0]_{R+1,1}$; $A=[a_{0}(x_{2}),a _{0}(x_{3}), \ldots,a_{0}(x_{N-1})]$; $r(\boldsymbol{x})=[r(x_{2}),r(x_{3}),\ldots, r(x_{N-1})]^{T}$.

Proof

We construct the Lagrangian function of the optimization problem (16):

$$ \begin{aligned}[b] &L(\boldsymbol{\omega }, \alpha _{i},\beta _{0},\beta _{s},\lambda _{0}, \lambda _{r},b,e_{i})\\ &\quad =\frac{1}{2} \boldsymbol{\omega }^{T} \boldsymbol{\omega }+\frac{1}{2}\gamma \boldsymbol{e}^{T} \boldsymbol{e}\\ &\qquad {}-\sum_{i=2}^{N-1} \alpha _{i} \Biggl[\boldsymbol{\omega } ^{T}\boldsymbol{\phi }^{(M)}(x_{i})+\sum_{l=0}^{M-1} \boldsymbol{\omega }^{T} a_{l}(x_{i})\boldsymbol{\phi }^{(l)}(x _{i})+a_{0}(x_{i})b -r(x_{i})-e_{i} \Biggr]\\ &\qquad {}-\beta _{0}\bigl( \boldsymbol{\omega ^{T}} \boldsymbol{\phi }(x_{1})+b-p_{0}\bigr)-\sum _{s=1} ^{S}\beta _{s}\bigl(\boldsymbol{ \omega ^{T}} \boldsymbol{\phi }^{(s)}(x _{1})-p_{s}\bigr)- \lambda _{0}\bigl(\boldsymbol{\omega ^{T}}\boldsymbol{\phi }(x _{N})+b-q_{0}\bigr)\\ &\qquad {}-\sum_{r=1}^{M-2-S} \lambda _{r}\bigl(\boldsymbol{\omega ^{T}} \boldsymbol{\phi }^{(r)}(x_{N})-q_{r}\bigr). \end{aligned} $$

(18)

The conditions for optimality are as follows:

$$ \begin{aligned} &\begin{aligned}\frac{\partial L}{\partial \boldsymbol{\omega }}={}& \boldsymbol{\omega }-\sum_{i=2}^{N-1}\alpha _{i} \Biggl[ \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=0}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr]-\beta _{0} \boldsymbol{\phi }(x _{1})-\sum_{s=1}^{S} \beta _{s}\boldsymbol{\phi }^{(s)}(x_{1}) \\ & {} -\lambda _{0}\boldsymbol{\phi }(x_{N})-\sum _{r=1}^{M-2-S}\lambda _{r} \boldsymbol{\phi }^{(r)}(x_{N})=0; \end{aligned} \\ &\frac{\partial L}{\partial \alpha _{i}}=\boldsymbol{\omega }^{T} \Biggl[\boldsymbol{\phi }^{(M)}(x _{i})+\sum_{l=0}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr]+a_{0}(x_{i})b -r(x_{i})-e_{i}=0,\quad i=2,3,\ldots,N-1; \\ &\frac{\partial L}{\partial \beta _{0}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x _{1})+b-p_{0}=0; \\ &\frac{\partial L}{\partial \beta _{s}}= \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(s)}(x_{1})-p_{s}=0,\quad s=1,2,\ldots,S; \\ &\frac{\partial L}{\partial \lambda _{0}}=\boldsymbol{\omega }^{T} \boldsymbol{\phi }(x_{N})+b-q_{0}=0; \\ &\frac{\partial L}{\partial \lambda _{r}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(r)}(x_{N})-q _{r}=0,\quad r=1,2,\ldots,M-2-S; \\ &\frac{\partial L}{\partial b}=-\sum_{i=2} ^{N-1}a_{0}(x_{i}) \alpha _{i}-\beta _{0}-\lambda _{0}=0; \\ &\frac{\partial L}{\partial e_{i}}=\alpha _{i}+\gamma e_{i}=0,\quad i=2,3,\ldots,N-1. \end{aligned} $$

(19)

Finally, rewriting the above system in matrix form will result in (17). □

The linear system (17), which consists of unknowns $(\alpha , \beta , \lambda , b)$, is solved. The LS-SVM model in the dual form becomes

$$ \begin{aligned}[b] \hat{y}(x) ={}&\sum _{i=2}^{N-1}\alpha _{i} \Biggl[\nabla _{M,0}\bigl(K(x_{i},x)\bigr)+ \sum _{l=0}^{M-1}a_{l}(x_{i})\nabla _{l,0}\bigl(K(x_{i},x)\bigr) \Biggr]\\ &{}+\beta _{0} \nabla _{0,0}\bigl(K(x_{1},x)\bigr) +\sum_{s=1}^{S}\beta _{s} \nabla _{s,0}\bigl(K(x _{1},x)\bigr)\\ &{}+\lambda _{0} \nabla _{0,0}\bigl(K(x_{N},x)\bigr)+\sum _{r=1}^{R}\lambda _{r}\nabla _{r,0}\bigl(K(x_{N},x)\bigr)+b. \end{aligned} $$

(20)

4.2 Multi-point boundary value problems of high-order ordinary differential equations

The improved LS-SVM algorithms to the solution of multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations are described.

4.2.1 Nonlinear ordinary differential equations for multi-point boundary value problems

Consider the following Mth-order nonlinear ordinary differential equations for multi-point boundary value problems [15]:

$$ \frac{d^{M}y}{dx^{M}}+a_{M-1}(x)\,\frac{d^{M-1}y}{dx^{M-1}}+\cdots+a _{1}(x)\,\frac{dy}{dx}=f(x,y), \quad x \in [a,c], $$

(21)

subject to $y^{(q_{0})}(a)=s_{0}$, $y^{(q_{j})}(x_{p_{j}})=s_{j}$, $y^{(q _{M-1})}(c)=s_{M-1}$, $x_{p_{j}}\in [a,c]$, $p_{j}\in Z$, $j=1,2,\ldots, M-2$, $0 \leq q_{0},q_{1},\ldots, q_{M-1}\leq M-1$.

The interval $[a, c]$ is discretized into a series of collocation points $\varOmega = \{a = x_{p_{0}}=x_{1} < x_{2} < \cdots<x_{p_{1}}<\cdots<x_{p_{2}}<\cdots<x _{p_{M-2}}<\cdots< x_{p_{M-1}}= x_{N} =c\}$. Assume that the approximate solution to (21) is $y=\boldsymbol{\omega }^{T}\boldsymbol{\phi }( {x)}+b$, the primal optimization problem is described as follows:

$$ \min _{\boldsymbol{\omega }, b, e_{i}, \boldsymbol{\xi }, y_{i}} J( \boldsymbol{\omega },\boldsymbol{e}, \boldsymbol{\xi })=\frac{1}{2} \boldsymbol{\omega }^{T} \boldsymbol{\omega }+\frac{1}{2}\gamma \boldsymbol{e}^{T} \boldsymbol{e}+\frac{1}{2}\gamma \boldsymbol{\xi } ^{T} \boldsymbol{\xi } $$

(22)

subject to

$$ \begin{aligned} &\boldsymbol{\omega }^{T} \Biggl[\boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=1}^{M-1} a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr]=f(x_{i},y _{i})+e_{i},\quad i=1,2,\ldots,N-M; \\ &y_{i}={\boldsymbol{\omega }^{T} \boldsymbol{\phi }(x_{i})+b}+\xi _{i},\quad i=1,2,\ldots,N-M; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(q_{0})}(x_{1})+b^{(q_{0})}=s _{0}; \\ &\boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(q_{j})}(x_{p _{j}})+b^{(q_{j})}=s_{j},\quad j=1,2, \ldots, M-2; \\ &\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(q_{M-1})}(x_{N})+b^{(q_{M-1})}=s_{M-1}. \end{aligned} $$

Theorem 3

Given a positive definite kernel function $K: R\times R\rightarrow R$ and a regularization parameter $\gamma \in R^{+}$, the solution to (22) is obtained by the following dual problem:

(23)

where $[\widehat{\varTheta }_{l,l'}]_{N-M}=[ \widetilde{\varTheta }_{M,M}]_{N-M}+ \overline{D}_{a_{l}}[ \overline{\varTheta }_{l,M}]_{N-M}+[\overline{ \varTheta }_{M,l'}]_{N-M}\overline{D}_{a_{l'}}^{T} + \overline{D}_{a_{l}} [\overline{\varTheta }_{l,l'}]_{N-M}\overline{D}^{T}_{a_{l'}}+ \gamma ^{-1}E$; $[\overline{\varTheta }_{M,l'}]_{N-M}= [[\widetilde{\varTheta } _{M,1}]_{N-M}, [\widetilde{ \varTheta }_{M,2}]_{N-M},\ldots,[ \widetilde{\varTheta }_{M,M-1}]_{N-M} ]$; $[\overline{\varTheta }_{l,M}]_{N-M}= [[\widetilde{ \varTheta }_{1,M}]_{N-M}; [\widetilde{\varTheta }_{2,M}]_{N-M};\ldots;[ \widetilde{\varTheta }_{M-1,M}]_{N-M} ]$; $\overline{D}_{a_{l'}}=[D _{a_{1}},D_{a_{2}}, \ldots,D_{a_{M-1}}]$; $\overline{D}_{a_{l}}=[D_{a_{1}},D _{a_{2}},\ldots,D_{a_{M-1}}]$; $[\overline{\varTheta }_{l,l'}]_{N-M}=[ \widetilde{\varTheta }_{1:M-1,1:M-1}]_{N-M}$; $\boldsymbol{\alpha }=[\alpha _{2}, \ldots,\alpha _{p_{1}-1},\alpha _{p_{1}+1},\ldots,\alpha _{N-1}]^{T}$; $[ \widetilde{\varTheta }_{0,0}]_{N-M}= [\varTheta _{0,0}]_{1:N-M,1:N-M}+\gamma ^{-1}E$; $l,l'=1,2, \ldots,M-1$; $D_{a_{l}}=\operatorname{diag}(a_{l}(x_{2}), \ldots,a_{l}(x _{p_{1}-1}), a_{l}(x_{p_{1}+1}), \ldots, a_{l}(x_{N-1}))$; $B=[\chi _{b_{0}}, \chi _{b_{1}},\ldots,\chi _{b_{M-1}}]$; $D_{a_{l'}}=\operatorname{diag}(a_{l'}(x_{2}), \ldots,a _{l'}(x_{p_{1}-1}), a_{l'}(x_{p_{1}+1}), \ldots,a_{l'}(x_{N-1}))$; $[\widehat{\varTheta }_{0,l'}]_{N-M}=[ \widetilde{\varTheta }_{0,M}]_{N-M}+[\overline{ \varTheta }_{0,l'}]_{N-M}\overline{D}_{a_{l'}}^{T}$; $\boldsymbol{s}=[s_{0},s _{1},s_{2}, \ldots,s_{M-1}]^{T}$; $[\overline{\varTheta }_{0,l'}]_{N-M}= [[\widetilde{ \varTheta }_{0,1}]_{N-M}, [\widetilde{\varTheta }_{0,2}]_{N-M},\ldots,[ \widetilde{\varTheta }_{0,M-1}]_{N-M} ]$; $\boldsymbol{y}=[y_{2}, \ldots,y _{P_{1}-1},y_{P_{1}+1},\ldots,y_{N-1}]^{T}$; $[\widehat{\varTheta }_{q_{j},l'}]_{M,N-M}= [ \widetilde{\varTheta }_{q_{0}:q_{M-1},M}]_{M,N-M}+[\overline{\varTheta } _{q_{0}:q_{M-1},l'}]_{M,N-M} \overline{D}_{a_{l'}}^{T}$; $\boldsymbol{\beta }=[\beta _{0},\beta _{1},\beta _{2},\ldots,\beta_{M-1}]^{T}$; $[\overline{\varTheta }_{q_{0}:q_{M-1},l'}]_{M,N-M}= [[ \widetilde{ \varTheta }_{q_{0}:q_{M-1},1}]_{M,N-M},[\widetilde{\varTheta } _{q_{0}:q_{M-1},2}]_{M,N-M},\ldots,[\widetilde{\varTheta }_{q_{0}:q_{M-1},M-1}]_{M,N-M} ]$; $[\widetilde{\varTheta }_{q_{j},0}]_{M,N-M}=[ \varTheta _{q_{0}:q_{M-1},0}]_{M,N-M}$; $\boldsymbol{\eta }=[\eta _{2}, \ldots, \eta _{p_{1}-1},\eta _{p_{1}+1},\ldots,\eta _{N-1}]^{T}$; $[ \widetilde{\varTheta }_{q_{j},q_{j'}}]_{p_{j},p_{j'}}= [ \varTheta _{q_{0}:q_{M-1},q_{0}:q_{M-1}}]_{p_{0}:p_{M-1},p_{0}:p_{M-1}}$; $D _{N-M}(\boldsymbol{y})= \operatorname{diag}(\frac{\partial f(\boldsymbol{x}, \boldsymbol{y})}{\partial y})$; $f_{N-M}(\boldsymbol{x}, \boldsymbol{y})=[f(x_{2},y_{2}), \ldots, f(x_{p_{1}-1},y_{p_{1}-1}),f(x _{p_{1}+1},y_{p_{1}+1}),\ldots,f(x_{N-1},y_{N-1}) ]^{T}$; $\frac{\partial f(\boldsymbol{x},\boldsymbol{y})}{\partial y}= [\frac{\partial f(x,y)}{ \partial y}|_{\substack{x=x_{2}\\y=y_{2}}},\ldots, \frac{\partial f(x,y)}{ \partial y}|_{\substack{x=x_{p_{1}-1}\\y=y_{p_{1}-1}}}, \frac{ \partial f(x,y)}{\partial y}|_{\substack{x=x_{p_{1}+1}\\y=y_{p _{1}+1}}},\ldots, \frac{\partial f(x,y))}{\partial y}|_{\substack{x=x _{N-1}\\y=y_{N-1}}} ]$, where $[\widetilde{\varTheta }_{m,n}]_{N-M}=[\varTheta _{m,n}]_{1:N-M,1:N-M};[ \widetilde{\varTheta }_{q_{0}:q_{M-1},m}]_{M,N-M}=[ \varTheta _{q_{0}:q_{M-1},m}]_{p _{0}:p_{M-1},1:N-M}$; $m,n=0,1,\ldots,M-1$.

Proof

The Lagrangian function of the constrained optimization problem (22) is introduced as follows:

$$ \begin{aligned}[b] &L(\boldsymbol{\omega }, y_{i},\alpha _{i},\eta _{i},\beta _{j},b,e_{i}, \xi _{i})\\ &\quad =\frac{1}{2} \boldsymbol{\omega }^{T}\boldsymbol{\omega }+ \frac{1}{2}\gamma \boldsymbol{e}^{T}\boldsymbol{e}+\frac{1}{2}\gamma \boldsymbol{\xi }^{T}\boldsymbol{\xi }\\ &\qquad {}-\sum_{i=1}^{N-M} \alpha _{i} \Biggl[\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(M)}(x_{i})+ \sum _{l=1}^{M-1}\boldsymbol{\omega }^{T}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i})-f(x_{i},y_{i})-e_{i} \Biggr]\\ &\qquad {}-\sum_{i=1} ^{N-M}\eta _{i}\bigl(\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x_{i})+b +\xi _{i}-y_{i}\bigr)-\sum_{j=0}^{M-1} \beta _{j} \bigl( \boldsymbol{\omega ^{T}}\boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})+b^{(q _{j})}-s_{j} \bigr). \end{aligned} $$

(24)

The conditions for optimality

$$\begin{aligned}& \begin{gathered} \frac{\partial L}{\partial \boldsymbol{\omega }}= \boldsymbol{\omega }-\sum_{i=1}^{N-M}\alpha _{i} \Biggl[ \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=1}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i})) \Biggr]-\sum _{i=1}^{N-M}\eta _{i} \boldsymbol{\phi }(x_{i})-\sum_{j=0}^{M-1}\beta _{j} \boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})=0; \\ \frac{\partial L}{ \partial \alpha _{i}}=\boldsymbol{\omega }^{T} \Biggl(\boldsymbol{\phi } ^{(M)}(x_{i})+\sum_{l=1}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i}) \Biggr) -f(x_{i},y_{i})-e_{i}=0,\quad i=1,2,\ldots,N-M; \\ \frac{\partial L}{\partial \eta _{i}}=\boldsymbol{\omega }^{T}\boldsymbol{\phi }(x _{i})+b+\xi _{i}-y_{i}=0,\quad i=1,2,\ldots,N-M; \\ \frac{\partial L}{\partial \xi _{i}}=-\eta _{i}+\gamma \xi _{i}=0,\quad i=1,2, \ldots,N-M; \\ \frac{\partial L}{\partial e_{i}}=\alpha _{i}+\gamma e_{i}=0,\quad i=1,2, \ldots,N-M; \\ \frac{ \partial L}{\partial \beta _{j}}=\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})+b^{(q_{j})}-s_{j}=0,\quad j=0,1, \ldots, M-1; \\ \frac{\partial L}{\partial b}=-\sum_{i=1}^{N-M}\eta _{i}-\sum_{j=0}^{M-1}\beta _{j}\chi _{b_{j}}=0,\quad \chi _{b_{j}}= \textstyle\begin{cases} 1,& q_{j}=0;\\ 0,& q_{j}=1,2,\ldots,M-1; \end{cases}\displaystyle \\ \frac{\partial L}{\partial y_{i}}=\eta _{i}+\alpha _{i} \frac{ \partial f(x_{i},y_{i})}{\partial y_{i}}=0 \end{gathered} \end{aligned}$$

(25)

can be written as a system in matrix form (23), after eliminating parameters ω and $e_{i}$. □

System (23), which consists of $3N-2M+1$ equations with unknowns $(\alpha ,\eta ,\beta ,b, y)$, is solved by Newton’s method. The LS-SVM model in the dual form becomes

$$ \begin{aligned}[b] \hat{y}(x) ={}&\sum _{i=1}^{N-M}\alpha _{i} \Biggl[\nabla _{M,0}\bigl(K(x_{i},x)\bigr)+ \sum _{l=1}^{M-1}a_{l}(x_{i})\nabla _{l,0}\bigl(K(x_{i},x)\bigr) \Biggr]\\ &{}+\sum _{i=1} ^{N-M}\eta _{i} \nabla _{0,0}\bigl(K(x_{i},x)\bigr) +\sum _{j=0}^{M-1}\beta _{j}\nabla _{q_{j},0}\bigl(K(x_{p_{j}},x)\bigr)+b. \end{aligned} $$

(26)

4.2.2 Linear ordinary differential equations for multi-point boundary value problems

Consider the following Mth-order linear ordinary differential equations for multi-point boundary value problems:

$$ \frac{d^{M}y}{dx^{M}}+a_{M-1}(x)\frac{d^{M-1}y}{dx^{M-1}}+\cdots+a _{1}(x)\frac{dy}{dx}+a_{0}(x)y=r(x), \quad x \in [a,c], $$

(27)

subject to $y^{(q_{0})}(a)=s_{0}$, $y^{(q_{j})}(x_{p_{j}})=s_{j}$, $y^{(q _{M-1})}(c)=s_{M-1}$, $x_{p_{j}}\in [a,c]$, $p_{j}\in Z$, $j=1,2,\ldots, M-2$, $0 \leq q_{0},q_{1},\ldots, q_{M-1}\leq M-1$.

The interval $[a, c]$ is discretized into a series of collocation points $\varOmega = \{a = x_{p_{0}}=x_{1} < x_{2} < \cdots<x_{p_{1}}<\cdots<x_{p_{2}}<\cdots<x _{p_{M-2}}<\cdots< x_{p_{M-1}}=x_{N} = c\}$. Suppose that the approximate solution to (27) is $y=\boldsymbol{\omega }^{T}\boldsymbol{\phi }( {x)}+b$, the original optimal problem is described as follows:

$$ \operatorname*{min}\limits _{\boldsymbol{\omega }, b, e_{i}} J( \boldsymbol{\omega },\boldsymbol{e})= \frac{1}{2}\boldsymbol{\omega } ^{T}\boldsymbol{\omega }+ \frac{1}{2}\gamma \boldsymbol{e}^{T} \boldsymbol{e} $$

(28)

subject to

$$ \begin{aligned} &\boldsymbol{\omega }^{T} \Biggl[\boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=0}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i})+a_{0}(x_{i})b \Biggr]=r(x_{i})+e_{i}, \\ & \quad i=2,3,\ldots,p_{1}-1, p_{1}+1,\ldots, N-1; \\ &\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(q_{0})}(x_{1})+b^{(q_{0})}=s_{0}; \\ & \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})+b^{(q _{j})}=s_{j},\quad j=1,2, \ldots, M-2; \\ &\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(q_{M-1})}(x_{N})+b^{(q_{M-1})}=s_{M-1}. \end{aligned} $$

Theorem 4

Given a positive definite kernel function $K: R\times R\rightarrow R$ and a regularization parameter $\gamma \in R^{+}$, the solution to (28) is obtained by the following dual problem:

(29)

where $[\widehat{\varTheta }_{l,l'}]_{N-M}=[ \widetilde{\varTheta }_{M,M}]_{N-M}+ \overline{D}_{a_{l}}[ \overline{\varTheta }_{l,M}]_{N-M}+[\overline{ \varTheta }_{M,l'}]_{N-M}\overline{D}_{a_{l'}}^{T} + \overline{D}_{a_{l}} [\overline{\varTheta }_{l,l'}]_{N-M}\overline{D}^{T}_{a_{l'}}+ \gamma ^{-1}E$; $[\overline{\varTheta }_{M,l'}]_{N-M}= [[\widetilde{\varTheta } _{M,0}]_{N-M},[\widetilde{ \varTheta }_{M,1}]_{N-M},\ldots,[ \widetilde{\varTheta }_{M,M-1}]_{N-M} ]$; $\overline{D}_{a_{l'}}=[D _{a_{0}},D_{a_{1}}, \ldots,D_{a_{M-1}}]$; $\overline{D}_{a_{l}}=[D_{a_{0}},D _{a_{1}},\ldots,D_{a_{M-1}}]$; $[\overline{\varTheta }_{l,l'}]_{N-M}=[ \widetilde{\varTheta }_{0:M-1,0:M-1}]_{N-M}$; $[\overline{\varTheta }_{l,M}]_{N-M}= [[\widetilde{ \varTheta }_{0,M}]_{N-M}; [\widetilde{\varTheta }_{1,M}]_{N-M};\ldots;[ \widetilde{\varTheta }_{M-1,M}]_{N-M} ]$; $D_{a_{l'}}=\operatorname{diag}(a_{l'}(x _{2}), \ldots,a_{l'}(x_{p_{1}-1}),a_{l'}(x_{p_{1}+1}), \ldots,a_{l'}(x_{N-1}))$; $\boldsymbol{s}=[s_{0},s_{1},s_{2}, \ldots,s_{M-1}]^{T}$; $D_{a_{l}}=\operatorname{diag}(a _{l}(x_{2}), \ldots,a_{l}(x_{p_{1}-1}),a_{l}(x_{p_{1}+1}), \ldots,a_{l}(x_{N-1}))$; $l,l'=0,1, \ldots,M-1$; $[\widehat{\varTheta }_{q_{j},l'}]_{M,N-M}=[\widetilde{\varTheta}_{q _{j},M}]_{M,N-M}+[\overline{\varTheta }_{q_{j},l'}]_{M,N-M} \overline{D} _{a_{l'}}^{T}$; $\boldsymbol{\beta }=[\beta _{0},\beta _{1},\beta _{2},\ldots, \beta _{M-1}]^{T}$; $[\overline{\varTheta }_{q_{j},l'}]_{M,N-M}= [[ \widetilde{ \varTheta }_{q_{j},0}]_{M,N-M},[\widetilde{\varTheta }_{q_{j},1}]_{M,N-M},\ldots,[ \widetilde{\varTheta }_{q_{j},M-1}]_{M,N-M} ]$; $[ \widetilde{\varTheta }_{q_{j},l'}]_{M,N-M}= [\varTheta _{q_{j},l'}]_{p_{0}:p_{M-1},1:N-M}$; $[\widetilde{\varTheta }_{q_{j},q_{j'}}]_{p_{j},p_{j'}}=[ \varTheta _{q_{0}:q_{M-1},q_{0}:q_{M-1}}]_{p_{0}:p_{M-1},p_{0}:p_{M-1}}$; $A=[a_{l}(x_{2}),\ldots,a_{l}(x_{p_{1}-1}), a_{l}(x_{p_{1}+1}), \ldots,a _{l}(x_{N-1})]$; $\boldsymbol{\alpha }=[\alpha _{2},\ldots,\alpha _{x_{p_{1}-1}}, \alpha _{x_{p_{1}+1}},\ldots, \alpha _{N-1}]^{T}$; $B=[\chi _{b_{0}},\chi _{b_{1}},\ldots,\chi _{b_{M-1}}]$; $r( \boldsymbol{x})=[r(x_{2}),\ldots,r(x_{p _{1}-1}),r(x_{p_{1}+1}), \ldots,r(x_{N-1}) ]^{T}$.

Proof

The Lagrangian function of the optimization problem (28) becomes

$$ \begin{aligned}[b] &L(\boldsymbol{\omega }, \alpha _{i},\beta _{j},b,e_{i})\\ &\quad =\frac{1}{2} \boldsymbol{\omega }^{T}\boldsymbol{\omega }+\frac{1}{2}\gamma \boldsymbol{e}^{T}\boldsymbol{e}\\ &\qquad {}-\sum_{i=1}^{N-M} \alpha _{i} \Biggl[ \boldsymbol{\omega }^{T}\boldsymbol{\phi }^{(M)}(x_{i})+\sum_{l=0} ^{M-1}\boldsymbol{\omega }^{T}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i})+a_{0}(x_{i})b -r(x_{i})-e_{i} \Biggr]\\ &\qquad {}-\sum_{j=0}^{M-1} \beta _{j} \bigl(\boldsymbol{\omega ^{T}}\boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})+b ^{(q_{j})}-s_{j} \bigr). \end{aligned} $$

(30)

Setting the partial derivatives of the Lagrangian function to zero, we will obtain

$$\begin{aligned} &\frac{\partial L}{\partial \boldsymbol{\omega }}= \boldsymbol{\omega }-\sum_{i=1}^{N-M}\alpha _{i} \Biggl( \boldsymbol{\phi }^{(M)}(x_{i})+\sum _{l=0}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x_{i}) \Biggr)-\sum _{j=0}^{M-1}\beta _{j} \boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})=0; \\ &\frac{\partial L}{ \partial \alpha _{i}}=\boldsymbol{\omega }^{T} \Biggl(\boldsymbol{\phi } ^{(M)}(x_{i})+\sum_{l=0}^{M-1}a_{l}(x_{i}) \boldsymbol{\phi }^{(l)}(x _{i}) \Biggr)+a_{0}(x_{i})b -r(x_{i})-e_{i}=0,\quad i=1,2,\ldots,N-M; \\ &\frac{ \partial L}{\partial e_{i}}=\alpha _{i}+\gamma e_{i}=0,\quad i=1,2, \ldots,N-M; \\ &\frac{\partial L}{\partial \beta _{j}}=\boldsymbol{\omega }^{T} \boldsymbol{\phi }^{(q_{j})}(x_{p_{j}})+b^{(q_{j})}-s_{j}=0,\quad j=0,1, \ldots, M-1; \\ &\frac{\partial L}{\partial b}=-\sum_{i=1}^{N-M}a_{0}(x_{i}) \alpha _{i}-\sum_{j=0}^{M-1}\beta _{j}\chi _{b_{j}}=0,\quad \chi _{b_{j}}= \textstyle\begin{cases} 1,& q_{j}=0;\\ 0, &q_{j}=1,2,\ldots,M-1. \end{cases}\displaystyle \end{aligned}$$

(31)

Finally, rewriting system (31) in matrix form will result in (29). □

System (29) with unknowns $(\alpha ,\beta , b)$ is solved. The LS-SVM model in the dual form becomes

$$ \begin{aligned}[b] \hat{y}(x) ={}&\sum _{i=1}^{N-M}\alpha _{i} \Biggl[\nabla _{M,0}\bigl(K(x_{i},x)\bigr)+ \sum _{l=0}^{M-1}a_{l}(x_{i})\nabla _{l,0}\bigl(K(x_{i},x)\bigr) \Biggr]\\ &{}+\sum _{j=0} ^{M-1}\beta _{j}\nabla _{q_{j},0}\bigl(K(x_{p_{j}},x)\bigr)+b. \end{aligned} $$

(32)

5 Numerical experiments

In this section, some numerical experiments are performed in order to demonstrate the reliability and powerfulness of the improved LS-SVM algorithms. The algorithms are applied to third-order, fourth-order linear and nonlinear ordinary differential equations with two-point boundary conditions and to third-order, fourth-order linear and nonlinear ordinary differential equations with multi-point boundary conditions.

In our experiments, the performance of the proposed LS-SVM algorithms is directly related to the choice of the regularization parameter γ and the kernel parameter σ. The larger the regularization parameter γ is, the smaller the error $e_{i}$ is, but when γ is a quite large value, the system of equations will be ill-conditioning. Therefore, the chosen value for γ was 10¹⁰. The validation set is obtained to be the set of midpoints $Z=\{z_{i}|z_{i}=(x_{i}+x_{i+1})/2,i =1,\ldots,N-1\}$, where $\{x_{i}\}_{i=1}^{N}$ are training points [42]. The optimal parameter σ that results in minimum root mean squared error (RMSE) on the validation set is selected and used for evaluating the LS-SVM model on the test set. The RMSE is defined as follows:

$$ \mathrm{RMSE}=\sqrt{\frac{1}{M}\sum _{i=1}^{M}\bigl[y(z_{i})- \hat{y}(z_{i})\bigr]^{2}}. $$

(33)

5.1 Example 1

Consider the fourth-order nonlinear ordinary differential equation [51]:

$$ \frac{d^{4}y}{dx^{4}}=-\frac{x^{2}}{1+y^{2}}-72\bigl(1-5x+5x^{2} \bigr)+\frac{x ^{2}}{1+(x-x^{2})^{6}}, \quad x \in [0, 1], $$

(34)

subject to two-point boundary conditions $y(0)=0$, $y'(0)=0$, $y(1)=0$, $y'(1)=0$. The analytic solution is $y=x^{3}(1-x)^{3}$.

We train the proposed LS-SVM algorithm for 11 equidistant points in the given interval $[0,1]$. The exact solution and the approximate solution via our proposed LS-SVM algorithm are shown in Fig. 1(a). Furthermore, the error between the analytic solution and the approximate solution is plotted in Fig. 1(b). In spite of using fewer points, we can see that the proposed LS-SVM algorithm could have a much better performance in terms of accuracy. The mean squared error is $6.5732\times 10^{-15}$ and the maximum absolute error is approximately $1.1063\times 10^{-7}$.

Figure 1

Two-point BVP of fourth-order nonlinear ODE (Example 1)

Table 1 lists the results of the exact solution and the approximate solution via our proposed LS-SVM algorithm for 11 testing points at unequal intervals in the domain $[0, 1]$. The absolute errors are shown in Table 1, in which we can see that the maximum absolute error is approximately $1.1293\times 10^{-7}$.

Table 1 Comparison between the exact solution and the LS-SVM solution (Example 1)

Figure 2 shows the logarithmic relation between the kernel bandwidth and the RMSE in Example 1. The red circle indicates the location of selected kernel bandwidth.

Figure 2

The logarithmic relation between σ and RMSE (Example 1)

5.2 Example 2

Let us consider the fourth-order linear ordinary differential equation [34]:

$$ \frac{d^{4}y}{dx^{4}}=120x, \quad x \in [-1, 1], $$

(35)

subject to two-point boundary conditions $y(-1)=1$, $y'(-1)=5$, $y(1)=3$, $y'(1)=5$. The analytic solution is $y=x^{5}+2$.

The proposed LS-SVM model has been trained with 11 equidistant points in the given interval $[-1,1]$. Figure 3(a) shows comparison between the exact solution and the approximate solution via our proposed LS-SVM algorithm, and Fig. 3(b) depicts the error plot between the analytic solution and the approximate solution. From the obtained results, we can see that the mean squared error is $6.5835\times 10^{-12}$ and the maximum absolute error is approximately $3.7390\times 10^{-6}$. The error obtained by the proposed LS-SVM algorithm remains low for the training points.

Figure 3

Two-point BVP of fourth-order linear ODE (Example 2)

Finally, the test results of the exact solution and the approximate solution via our proposed LS-SVM algorithm for 11 equidistant points in the domain $[-1, 1]$ are listed in Table 2. The absolute errors are shown in Table 2, in which we can see that the maximum absolute error is approximately $3.7071\times 10^{-6}$. It is clear that the proposed LS-SVM algorithm has a better performance in terms of accuracy.

Table 2 Comparison between the exact solution and the LS-SVM solution (Example 2)

5.3 Example 3

Consider the fourth-order linear ordinary differential equation [52]:

$$ \frac{d^{4}y}{dx^{4}} + y(x)= \biggl(\biggl(\frac{\pi }{2} \biggr)^{4}+1 \biggr)\cos \biggl(\frac{\pi }{2}x \biggr) , \quad x \in [-1, 1], $$

(36)

subject to two-point boundary conditions $y(-1)=0$, $y'(-1)=\pi /2$, $y(1)=0$, $y'(1)=-\pi /2$. The analytic solution is $y=\cos (\pi x/2 )$.

The proposed LS-SVM algorithm for two-point boundary value problems of high-order linear ordinary differential equation has been trained with 11 equidistant points in the given interval $[-1,1]$. Comparison between the exact solution and the approximate solution via our proposed LS-SVM algorithm is depicted in Fig. 4(a). Plot of the error function is cited in Fig. 4(b), from which we can see that the mean squared error is $2.6426\times 10^{-18}$ and the maximum absolute error is approximately $2.5670\times 10^{-9}$. The accuracy of the error obtained by the proposed LS-SVM algorithm is $O(10^{-9})$. The results reveal that the proposed LS-SVM algorithm has higher accuracy, although we only choose 11 equidistant points for training process.

Figure 4

Two-point BVP of fourth-order linear ODE (Example 3)

Finally, Table 3 incorporates results of the exact solution and the approximate solution via our proposed LS-SVM algorithm for 11 testing points at unequal intervals in the domain $[-1, 1]$. The absolute errors are shown in Table 3, in which we can see that the maximum absolute error is approximately $2.5543\times 10^{-9}$.

Table 3 Comparison between the exact solution and the LS-SVM solution (Example 3)

5.4 Example 4

Consider the third-order nonlinear ordinary differential equation [15]:

$$ \frac{d^{3}y}{dx^{3}}=-y^{2}-\cos (x)+\sin ^{2}(x), \quad x \in [0, 1], $$

(37)

subject to multi-point boundary conditions $y'(0)=1$, $y(\frac{1}{2})=\sin ( \frac{1}{2})$, $y'(1)=\cos (1)$. The analytic solution is $y=\sin (x)$.

When 11 equidistant points in the interval $[0, 1]$ are used for training, the results are depicted in Fig. 5(a). Figure 5(b) shows the errors between the exact solution and the approximate solution obtained by the proposed LS-SVM algorithm. From the obtained results, although training was performed just for 11 equidistant points in the domain $[0, 1]$, the mean squared error is approximately $4.3564\times 10^{-7}$. The proposed LS-SVM algorithm obtains a satisfactory result for multi-point boundary value problems of third-order nonlinear ordinary differential equation.

Figure 5

Multi-point BVP of third-order nonlinear ODE (Example 4)

Finally, Table 4 tabulates results of the exact solution and the approximate solution via our proposed LS-SVM algorithm for 11 testing points at unequal intervals in the domain $[0, 1]$. The absolute errors are shown in Table 4, in which we can see that the mean squared error is approximately $4.9717\times 10^{-7}$.

Table 4 Comparison between the exact solution and the LS-SVM solution (Example 4)

5.5 Example 5

Consider the fourth-order linear ordinary differential equation:

$$ \frac{d^{4}y}{dx^{4}}+\frac{dy}{dx}=4x^{3}+24, \quad x \in [0, 1], $$

(38)

subject to multi-point boundary conditions $y(0)=0$, $y'''(0.25)=6$, $y''(0.5)=3$, $y(1)=1$. The analytic solution is $y=x^{4}$.

When 21 equidistant points in the interval $[0, 1]$ are used for training, the approximate solution obtained by the proposed LS-SVM algorithm is compared with the exact solution in Fig. 6(a), and the error is plotted in Fig. 6(b). From which, the mean squared error is approximately $2.2915\times 10^{-10}$. The proposed LS-SVM algorithm can obtain the desired accuracy, although the training was performed using just a small part points in the domain $[0, 1]$.

Figure 6

Multi-point BVP of third-order linear ODE (Example 5)

The test results of the exact solution and the approximate solution via our proposed LS-SVM algorithm for 20 equidistant points in the domain $[0, 1]$ are listed in Table 5, and the absolute error is also calculated in Table 5. We can see that the mean squared error is approximately $2.3557\times 10^{-10}$ and the maximum absolute error is approximately $2.7702\times 10^{-5}$. The improved LS-SVM algorithm has a good performance for solving multi-point boundary value problems of fourth-order linear ordinary differential equations.

Table 5 Comparison between exact solution and LS-SVM solution (Example 5)

6 Conclusion

In this paper, the improved LS-SVM algorithms have been developed for solving two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations. Accuracy of the improved LS-SVM algorithms has been checked by solving a fourth-order nonlinear ordinary differential equation with two-point boundary conditions, two fourth-order linear ordinary differential equations with two-point boundary conditions, a three-order nonlinear ordinary differential equation with multi-point boundary conditions, and a fourth-order linear ordinary differential equation with multi-point boundary conditions. The results obtained by the improved LS-SVM algorithms are compared with the exact solution. It has been noted that our proposed LS-SVM algorithms can solve two-point and multi-point boundary value problems of high-order linear and nonlinear ordinary differential equations with higher accuracy in the tables and graphs. So the improved LS-SVM algorithms in the use of the two-point and multi-point boundary value problems are found to be efficient and straightforward.