# Infinite Horizon Discrete Time Control Problems for Bounded Processes

- Joël Blot
^{1}and - Naïla Hayek
^{2, 1}Email author

**2008**:654267

**DOI: **10.1155/2008/654267

© J. Blot and N. Hayek. 2008

**Received: **15 October 2008

**Accepted: **27 December 2008

**Published: **9 February 2009

## Abstract

We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal control problems for bounded processes, for systems governed by difference equations. Results due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary conditions with weakened hypotheses of concavity and without invertibility, and we provide new results on the adjoint variable. We show links between bounded problems and nonbounded ones. We also give sufficient conditions of optimality.

## 1. Introduction

The first works on infinite horizon optimal control problems are due to Pontryagin and his school [1]. They were followed by few others [2–6]. We consider in this paper an infinite horizon Optimal Control problem in the discrete time framework. Such problems are fundamental in the macroeconomics growth theory [7–10] and see references of [11]. Even in the finite horizon case, the discrete time framework presents significant differences from the continuous time one. Boltianski [12] shows that in the discrete time case, a convexity condition is needed to guarantee a strong Pontryagin Principle while this last one can be obtained without such condition in the continuous time setting. We study our problem in the space of bounded sequences which allows us to use Analysis in Banach spaces instead of using reductions to finite horizon problems as in [5, 6]. According to Chichlinisky [13, 14], the space of bounded sequences was first used in economics by Debreu [15]. It can also be found in [7, 8, 16]. We obtain Pontryagin Maximum Principles in the strong form using weaker convexity hypotheses than the traditional ones and without invertibility [5]. When we study the problem in a general sequence space it turns out that the infinite series will not always converge. Therefore we present other notions of optimality that are currently used, notably in the economic literature, see [3, 4, 9] and we show how our problem can be related to these other problems. We end the paper by establishing sufficient conditions of optimality.

Now we briefly describe the contents of the paper. In Section 2 we introduce the notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary conditions of optimality namely the existence of the adjoint variable in the space satisfying the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove these theorems through some lemmas and using results due to Ioffe-Tihomirov [17]. In Section 4 we introduce some other notions of optimality for problems in the nonbounded case and we show links with our problem. For example, we show that when the objective function is positive then a bounded solution is a solution among the unbounded processes. Finally we give sufficient conditions of optimality for problems in the bounded and unbounded cases adapting for each case the approprate transversality condition.

## 2. Pontryagin maximum principles for bounded processes

We first precise our notations. Let be a nonempty open convex subset of and a nonempty compact subset of Let and, for all We set

Define

For every define as the closure of the set of terms of the sequence If is compact. We set such that is thus the set of the bounded sequences which are in the interior of Note that is a convex open subset of since is open and convex. We set Define ; it is the set of admissible processes with respect to the considered dynamical system.

which can be written as follows.

(P1) Maximize when

Theorem 2.1.

Let be a solution of (P1). Assume the following.

- (i)
For all the mapping is of class on and for all the mapping is Fréchet-differentiable on

- (ii)For all for all for all for all there exists such that(2.1)

- (iii)
For any compact set there exists a constant such that for all for all for all and

- (iv)There exists such that and for all(2.2)

Then there exists such that

- (a)
- (b)
- (c)

Comments

For continuous time problems, one does not need conditions to obtain a strong Pontryagin maximum principle, both in the finite horizon case (see, e.g., [18]) and in the infinite horizon case (see, e.g., [5]). But for discrete time problems, strong Pontryagin principles cannot hold without an additional assumption namely a convexity condition, as Boltyanski shows in [12] for the finite horizon framework. Condition (ii) comes from the Ioffe and Tihomirov book [17]. It generalizes the usual convexity condition used to garantee a strong Pontryagin maximum principle. The usual condition is: convex subset, concave with respect to and for every affine with respect to It implies condition (ii). In (iii) the condition is satisfied when is continuous (since is compact) and the condition is satisfied when exists and is continuous.

Conclusion (a) is the adjoint equation, conclusion (b) is the strong Pontryagin maximum principle and conclusion (c) is a transversality condition at infinity. In our case (c) is immediately obtained since is in but in general (nonbounded cases) it is very delicate to obtain such a conclusion. [9]

In the next theorem we consider the autonomous case. Thus the hypotheses are simpler and easier to manipulate.

Theorem 2.2.

Let for all Let be a solution of (P1). Assume that the following conditions are fulfilled.

- (i)
For all the mappings and are of class on

- (ii)For all for all for all for all there exists such that(2.3)

- (iii)

Then there exists such that the assertions (a), (b), and (c) of Theorem 2.1 are satisfied.

## 3. Proofs of Theorems 2.1 and 2.2

Proof of Theorem 2.1

First Part

The first part of the proof goes through several lemmas.

Lemma 3.1.

is well-defined and under hypothesis (i) of Theorem 2.1, for all the mapping is of class and one has, for all

For the proof see [19].

We set for all

Lemma 3.2.

Assume that hypothesis (iii) of Theorem 2.1 holds. Then for one has Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold, then for all the mapping is of class on the ball in and for all one has

Proof.

Let Let be the constant of (iii) with So for all So one has

Assume now that hypotheses (i) and (iv) of Theorem 2.1 hold. Let us show that is of class on Take Let be given. Let be such that Then, for all under (iv), which implies that

Let us now show that is Fréchet-differentiable on Take Let be given. Let be such that Then, for all under (iii). But this implies that Thus is Fréchet-differentiable at and

To show the continuity of at let be the constant of hypothesis (iii) corresponding to Let be given and let be such that So is of class

Lemma 3.3.

Proof.

Set so and satisfies the required relations.

Lemma 3.4.

Under hypotheses (i) and (iv) of Theorem 2.1,

Proof.

Since the problem is a problem of bounded solutions of first-order linear difference equations.

where

Recall that the norm of is defined by and that the norm of a linear operator between normed spaces is defined by

So So Since and is invertible so it is surjective.

Set Then under (iv) one has So is surjective that is

Recall that where consists of all singular functionals, see Aliprantis and Border [20]. In fact it consists (up to scalar multiples) of all extensions of the "limit functional" to

If then there exists such that for all ( being the space of convergent sequences having a limit in )

Lemma 3.5 ( ).

If then where for every So there exists such that for all

Second Part

Set where and

So one has if hence

Thus, it holds that

Now

So for all

using as is of finite support.

Lemma 3.6.

( ).

Proof.

If then since

Hence We can set it equal to one.

From Lemma 3.6 and the previous results, conclusions (a) and (b) are satisfied.

Conclusion (c) is a straightforward consequence of the belonging of to

Lemma 3.7.

( ).

Proof.

Indeed we obtained for all Using one has for all Thus

Proof of Theorem 2.2.

Define on such that Under hypothesis (i) of Theorem 2.2, for all the mappings and are of class on The proof can be found in [19].

We consider the proof of Lemma 3.4 and we set Then the proof goes like that of Theorem 2.1.

## 4. Results for unbounded problems

We study now problems of maximization over admissible processes which are not necessarily bounded when the optimal solution is bounded. So consider the following problems.

(P2) Maximize on

(P3) Find such that, for all

(P4) Find such that, for all

(P5) Find such that, for all

The optimality notion of (P3) is called "the strong optimality," that of (P4) is called "the overtaking optimality" and that of (P5) the "weak overtaking optimality" in [3] (in the continuous-time framework). Many existence results of overtaking optimal solutions and weakly overtaking optimal solutions are obtained in [3, 4]. In [4] there are also results in the discrete-time framework.

Remark 4.1.

Notice that is an optimal solution of (P3) implies is an optimal solution of (P4) which implies is an optimal solution of (P5).

Moreover if is a bounded optimal solution of (P4) then (P3) and (P4) reduce to the same problem.

Lemma 4.2.

The two following assertions hold.

- (a)
If is an optimal solution of problem (P2), (P3), (P4) or (P5) and then is an optimal solution of problem (P1). Therefore Theorem 2.1 applies.

- (b)
Assume on If is an optimal solution of problem (P3) or (P4) and then

Proof. (a) Since a bounded optimal solution of (P2) or (P3) is an optimal solution of (P1). Suppose now that is a bounded optimal solution of (P4) that is for all Since this can be written for all and so in particular for all

In that case The proof is analogous for (P5).

(b) If is an optimal solution of problem (P3) and one has for all Since the sequence is increasing and since it is also upper bounded it converges in

So and

Theorem 4.3.

Let for all One assumes the following conditions fulfilled:

- (i)
on

- (ii)
For all there exists

Then one has

- (a)
- (b)
If is an optimal solution of problem (P1), then it is an optimal solution of problems (P3), (P4), and (P5) which all reduce to the same problem.

Remark 4.4. (b) shows that under a nonnegativity assumption, solving the problem in the space of bounded processes provides solutions for problems in spaces of admissible processes which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny [21] where problems are studied in the continuous-time case.

where is such that

and are bounded and

(b) Since for all the sequence is nonnegative and nondecreasing so it converges in

Let and Construct and as in (a). Thus

Hence we obtain so which contradicts the hypothesis so is an optimal solution of problem (P3).

Following Michel, [22], for all and for all we define as the set of the for which there exists satisfying We also define as the set of the for which there exists satisfying for all

Theorem 4.5.

Let for all Let be an optimal solution of problem (P1). One assumes the following conditions fulfilled.

- (i)
on

- (ii)
For all there exists

- (iii)
For all the mappings and are Fréchet-differentiable at

- (iv)
For all for all co where co denotes the convex hull.

- (v)
For all is invertible.

Then there exists such that and

- (a)
- (b)
for all for all

Remark 4.6.

Notice that condition (iv) is a convexity condition and that condition (ii) of Theorem 2.2 implies this condition (iv). Condition (ii) of Theorem 2.2 is equivalent to the following condition: for all the set is convex.

Proof.

Use Theorem 4.3 of this paper and apply Theorem 3 in Blot-Chebbi [5].

## 5. Sufficient conditions of optimality

Let for all

Theorem 5.1.

Let where is convex. One assumes that there exists and that the following conditions are fulfilled.

- (i)
The mappings and for all are of class on

- (ii)
- (iii)
- (iv)
The mapping is concave with respect to for all

Then is an optimal solution of (P1).

Proof.

Notice that from (ii), for all

Since for all is concave with respect to and one has Using hypothesis (iii) with gives and using hypothesis (iii), the first order necessary condition for the optimality of is Thus one has

The hypothesis implies and since and belong to one has Hence we obtain so That is

Corollary 5.2.

Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:

- (v)

then is a solution of (P3) (resp., (P4)).

Notice that if with we obtain that is a solution of (P5).

One can weaken the hypothesis of concavity of with respect to and and replace it by the concavity of with respect to as the following theorem shows. (See [23] for a quick survey of sufficient conditions.)

Let

The maximum is attained since is compact.

Theorem 5.3.

Let One assumes that there exists and that the following hypotheses are fulfilled.

- (i)
For all the mappings and for all are of class on

- (ii)
Also (iii) of the previous theorem.

- (iv)
The mapping is concave with respect to for all

Then is an optimal solution of (P1).

Proof.

(Notice that Now using (see Seierstad and Sydsaeter [24, page 390]) we obtain The concavity of with respect to gives Finally follows as in the proof of the previous theorem.

Corollary 5.4.

Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:

- (v)

then is a solution of (P3) (resp., (P4)).

Notice that if with we obtain that is a solution of (P5).

## Authors’ Affiliations

## References

- Pontryagin L, Boltianski V, Gramkrelidze R, Mitchenko E:
*Théorie Mathématique des Processus Optimaux*. Mir, Moscow, Russia; 1974.Google Scholar - Ekeland I, Scheinkman JA: Transversality conditions for some infinite horizon discrete time optimization problems.
*Mathematics of Operations Research*1986, 11(2):216-229. 10.1287/moor.11.2.216MATHMathSciNetView ArticleGoogle Scholar - Carlson DA, Haurie AB, Leizarowitz A:
*Infinite Horizon Optimal Control. Deterministic and Stochastic Systems*. 2nd edition. Springer, Berlin, Germany; 1991.MATHView ArticleGoogle Scholar - Zaslavski AJ:
*Turnpike Properties in the Calculus of Variations and Optimal Control, Nonconvex Optimization and Its Applications*.*Volume 80*. Springer, New York, NY, USA; 2006:xxii+395.Google Scholar - Blot J, Chebbi H: Discrete time Pontryagin principles with infinite horizon.
*Journal of Mathematical Analysis and Applications*2000, 246(1):265-279. 10.1006/jmaa.2000.6797MATHMathSciNetView ArticleGoogle Scholar - Blot J, Hayek N: Infinite-horizon Pontryagin principles with constraints. In
*Communications of the Laufen Colloquium on Science, Laufen, Austria, Berichte aus der Mathematik, 2*. Edited by: Ruffing A, Suhrer A, Suhrer J. Shaker, Aachen, Germany; 2007:1-14.Google Scholar - Araujo A, Scheinkman JA: Smoothness, comparative dynamics, and the turnpike property.
*Econometrica*1977, 45(3):601-620. 10.2307/1911677MATHMathSciNetView ArticleGoogle Scholar - Araujo A, Scheinkman JA: Notes on comparative dynamics. In
*General Equilibrium, Growth, and Trade*. Edited by: Green JR, Scheinkman JA. Academic Press, New York, NY, USA; 1979:217-226.Google Scholar - Michel P: Some clarifications on the transversality condition.
*Econometrica*1990, 58(3):705-723. 10.2307/2938197MATHMathSciNetView ArticleGoogle Scholar - Stokey NL, Lucas RE Jr., Prescott EC:
*Recursive Methods in Economic Dynamics*. Harvard University Press, Cambridge, Mass, USA; 1989:xviii+588.MATHGoogle Scholar - Grinold R: Convex infinite horizon programs.
*Mathematical Programming*1983, 25(1):64-82. 10.1007/BF02591719MATHMathSciNetView ArticleGoogle Scholar - Boltianski V:
*Commande Optimale des Systèmes Discrets. Traduit du Russe par A. Sossinsk*. Mir, Moscow, Russia; 1976:467.Google Scholar - Chichilnisky G: An axiomatic approach to sustainable development.
*Social Choice and Welfare*1996, 13(2):231-257. 10.1007/BF00183353MATHMathSciNetView ArticleGoogle Scholar - Chichlinisky G: What is sustainable development?
*Land Economics*1997, 73(4):46791.Google Scholar - Debreu G: Valuation equilibrium and Pareto optimum.
*Proceedings of the National Academy of Sciences of the United States of America*1954, 40(7):588-592. 10.1073/pnas.40.7.588MATHMathSciNetView ArticleGoogle Scholar - Peleg B, Ryder H: On optimal consumption plans in a multi-sector economy.
*Review of Economics Studies*1972, 39(2):159-169. 10.2307/2296868MATHView ArticleGoogle Scholar - Ioffe AD, Tihomirov VM:
*Theory of Extremal Problems, Studies in Mathematics and Its Applications*.*Volume 6*. North-Holland, Amsterdam, The Netherlands; 1979:xii+460.Google Scholar - Alexéev V, Tikhomirov V, Fomine S:
*Commande Optimale*. Mir, Moscow, Russia; 1982:447.Google Scholar - Blot J, Crettez B: On the smoothness of optimal paths.
*Decisions in Economics and Finance*2004, 27(1):1-34. 10.1007/s10203-004-0042-5MATHMathSciNetView ArticleGoogle Scholar - Aliprantis CD, Border KC:
*Infinite-Dimensional Analysis*. 2nd edition. Springer, Berlin, Germany; 1999:xx+672.MATHView ArticleGoogle Scholar - Blot J, Cartigny P: Optimality in infinite-horizon variational problems under sign conditions.
*Journal of Optimization Theory and Applications*2000, 106(2):411-419. 10.1023/A:1004611816252MATHMathSciNetView ArticleGoogle Scholar - Michel P: Programmes mathématiques mixtes. Application au principe du maximum en temps discret dans le cas déterministe et dans le cas stochastique.
*RAIRO Recherche Opérationnelle*1980, 14(1):1-19.MATHGoogle Scholar - Blot J, Hayek N: Sufficient conditions for infinite-horizon calculus of variations problems.
*ESAIM: Control, Optimisation and Calculus of Variations*2000, 5: 279-292.MATHMathSciNetView ArticleGoogle Scholar - Seierstad A, Sydsaeter K: Sufficient conditions in optimal control theory.
*International Economic Review*1977, 18(2):367-391. 10.2307/2525753MATHMathSciNetView ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.