 Research Article
 Open Access
 Published:
Infinite Horizon Discrete Time Control Problems for Bounded Processes
Advances in Difference Equations volume 2008, Article number: 654267 (2009)
Abstract
We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal control problems for bounded processes, for systems governed by difference equations. Results due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary conditions with weakened hypotheses of concavity and without invertibility, and we provide new results on the adjoint variable. We show links between bounded problems and nonbounded ones. We also give sufficient conditions of optimality.
1. Introduction
The first works on infinite horizon optimal control problems are due to Pontryagin and his school [1]. They were followed by few others [2–6]. We consider in this paper an infinite horizon Optimal Control problem in the discrete time framework. Such problems are fundamental in the macroeconomics growth theory [7–10] and see references of [11]. Even in the finite horizon case, the discrete time framework presents significant differences from the continuous time one. Boltianski [12] shows that in the discrete time case, a convexity condition is needed to guarantee a strong Pontryagin Principle while this last one can be obtained without such condition in the continuous time setting. We study our problem in the space of bounded sequences which allows us to use Analysis in Banach spaces instead of using reductions to finite horizon problems as in [5, 6]. According to Chichlinisky [13, 14], the space of bounded sequences was first used in economics by Debreu [15]. It can also be found in [7, 8, 16]. We obtain Pontryagin Maximum Principles in the strong form using weaker convexity hypotheses than the traditional ones and without invertibility [5]. When we study the problem in a general sequence space it turns out that the infinite series will not always converge. Therefore we present other notions of optimality that are currently used, notably in the economic literature, see [3, 4, 9] and we show how our problem can be related to these other problems. We end the paper by establishing sufficient conditions of optimality.
Now we briefly describe the contents of the paper. In Section 2 we introduce the notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary conditions of optimality namely the existence of the adjoint variable in the space satisfying the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove these theorems through some lemmas and using results due to IoffeTihomirov [17]. In Section 4 we introduce some other notions of optimality for problems in the nonbounded case and we show links with our problem. For example, we show that when the objective function is positive then a bounded solution is a solution among the unbounded processes. Finally we give sufficient conditions of optimality for problems in the bounded and unbounded cases adapting for each case the approprate transversality condition.
2. Pontryagin maximum principles for bounded processes
We first precise our notations. Let be a nonempty open convex subset of and a nonempty compact subset of Let and, for all We set
Define
For every define as the closure of the set of terms of the sequence If is compact. We set such that is thus the set of the bounded sequences which are in the interior of Note that is a convex open subset of since is open and convex. We set Define ; it is the set of admissible processes with respect to the considered dynamical system.
Let We consider first the following problem (P1):
which can be written as follows.
(P1) Maximize when
Theorem 2.1.
Let be a solution of (P1). Assume the following.

(i)
For all the mapping is of class on and for all the mapping is Fréchetdifferentiable on

(ii)
For all for all for all for all there exists such that
(2.1)

(iii)
For any compact set there exists a constant such that for all for all for all and

(iv)
There exists such that and for all
(2.2)
Then there exists such that
 (a)
 (b)
 (c)
Comments
For continuous time problems, one does not need conditions to obtain a strong Pontryagin maximum principle, both in the finite horizon case (see, e.g., [18]) and in the infinite horizon case (see, e.g., [5]). But for discrete time problems, strong Pontryagin principles cannot hold without an additional assumption namely a convexity condition, as Boltyanski shows in [12] for the finite horizon framework. Condition (ii) comes from the Ioffe and Tihomirov book [17]. It generalizes the usual convexity condition used to garantee a strong Pontryagin maximum principle. The usual condition is: convex subset, concave with respect to and for every affine with respect to It implies condition (ii). In (iii) the condition is satisfied when is continuous (since is compact) and the condition is satisfied when exists and is continuous.
Conclusion (a) is the adjoint equation, conclusion (b) is the strong Pontryagin maximum principle and conclusion (c) is a transversality condition at infinity. In our case (c) is immediately obtained since is in but in general (nonbounded cases) it is very delicate to obtain such a conclusion. [9]
In the next theorem we consider the autonomous case. Thus the hypotheses are simpler and easier to manipulate.
Theorem 2.2.
Let for all Let be a solution of (P1). Assume that the following conditions are fulfilled.

(i)
For all the mappings and are of class on

(ii)
For all for all for all for all there exists such that
(2.3)
 (iii)
Then there exists such that the assertions (a), (b), and (c) of Theorem 2.1 are satisfied.
3. Proofs of Theorems 2.1 and 2.2
Proof of Theorem 2.1
First Part
The first part of the proof goes through several lemmas.
Lemma 3.1.
is welldefined and under hypothesis (i) of Theorem 2.1, for all the mapping is of class and one has, for all
For the proof see [19].
We set for all
Lemma 3.2.
Assume that hypothesis (iii) of Theorem 2.1 holds. Then for one has Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold, then for all the mapping is of class on the ball in and for all one has
Proof.
Let Let be the constant of (iii) with So for all So one has
Assume now that hypotheses (i) and (iv) of Theorem 2.1 hold. Let us show that is of class on Take Let be given. Let be such that Then, for all under (iv), which implies that
Let us now show that is Fréchetdifferentiable on Take Let be given. Let be such that Then, for all under (iii). But this implies that Thus is Fréchetdifferentiable at and
To show the continuity of at let be the constant of hypothesis (iii) corresponding to Let be given and let be such that So is of class
Lemma 3.3.
Under hypothesis (ii) of Theorem 2.1, for all for all for all there exists such that
Proof.
Let and Hypothesis (ii) of Theorem 2.1 implies for all the existence of such that
Therefore we obtain
Set so and satisfies the required relations.
Lemma 3.4.
Under hypotheses (i) and (iv) of Theorem 2.1,
Proof.
Since the problem is a problem of bounded solutions of firstorder linear difference equations.
Let Assume that Then for all there exists a unique such that for all
where
Consider the operator such that for all
where
Recall that the norm of is defined by and that the norm of a linear operator between normed spaces is defined by
So So Since and is invertible so it is surjective.
Set Then under (iv) one has So is surjective that is
Recall that where consists of all singular functionals, see Aliprantis and Border [20]. In fact it consists (up to scalar multiples) of all extensions of the "limit functional" to
If then there exists such that for all ( being the space of convergent sequences having a limit in )
Lemma 3.5 ().
If then where for every So there exists such that for all
Second Part
Our optimal control problem can be written as the following abstract static optimisation problem in a Banach space:
that satisfies all conditions of Theorem 4.3, IoffeTihomirov [17]. So we can apply this theorem and obtain the existence of not all zero, such that:
(AE) denotes the adjoint equation of this problem and (PMP) the Pontryagin maximum principle. They can be written, respectively:
Set where and
(AE) becomes:
So we get
Let be arbitrarily chosen in and let be in Consider the sequence defined as follows:
So one has if hence
Thus, it holds that
Now
Therefore, for all and for all one has
which implies
that is,
(PMP) becomes:
So for all
Consider, for all the sequences defined as follows:
Since the inequality holds for every we obtain
using as is of finite support.
Lemma 3.6.
().
Proof.
Recall we obtained the existence of not all zero, such that:
If then since
Hence We can set it equal to one.
From Lemma 3.6 and the previous results, conclusions (a) and (b) are satisfied.
Conclusion (c) is a straightforward consequence of the belonging of to
Lemma 3.7.
().
Proof.
Indeed we obtained for all Using one has for all Thus
Proof of Theorem 2.2.
Define on such that Under hypothesis (i) of Theorem 2.2, for all the mappings and are of class on The proof can be found in [19].
We consider the proof of Lemma 3.4 and we set Then the proof goes like that of Theorem 2.1.
4. Results for unbounded problems
We study now problems of maximization over admissible processes which are not necessarily bounded when the optimal solution is bounded. So consider the following problems.
(P2) Maximize on
(P3) Find such that, for all
(P4) Find such that, for all
(P5) Find such that, for all
The optimality notion of (P3) is called "the strong optimality," that of (P4) is called "the overtaking optimality" and that of (P5) the "weak overtaking optimality" in [3] (in the continuoustime framework). Many existence results of overtaking optimal solutions and weakly overtaking optimal solutions are obtained in [3, 4]. In [4] there are also results in the discretetime framework.
Remark 4.1.
Notice that is an optimal solution of (P3) implies is an optimal solution of (P4) which implies is an optimal solution of (P5).
Moreover if is a bounded optimal solution of (P4) then (P3) and (P4) reduce to the same problem.
Lemma 4.2.
The two following assertions hold.

(a)
If is an optimal solution of problem (P2), (P3), (P4) or (P5) and then is an optimal solution of problem (P1). Therefore Theorem 2.1 applies.

(b)
Assume on If is an optimal solution of problem (P3) or (P4) and then
Proof. (a) Since a bounded optimal solution of (P2) or (P3) is an optimal solution of (P1). Suppose now that is a bounded optimal solution of (P4) that is for all Since this can be written for all and so in particular for all
In that case The proof is analogous for (P5).
(b) If is an optimal solution of problem (P3) and one has for all Since the sequence is increasing and since it is also upper bounded it converges in
So and
Theorem 4.3.
Let for all One assumes the following conditions fulfilled:

(i)
on

(ii)
For all there exists
Then one has
 (a)

(b)
If is an optimal solution of problem (P1), then it is an optimal solution of problems (P3), (P4), and (P5) which all reduce to the same problem.
Remark 4.4. (b) shows that under a nonnegativity assumption, solving the problem in the space of bounded processes provides solutions for problems in spaces of admissible processes which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny [21] where problems are studied in the continuoustime case.
Proof. (a) It is clear that the following inequality holds:
Let Let be given and let be such that Set
where is such that
and are bounded and
Since on one has
so we obtain
Since this is true fo all letting we obtain
(b) Since for all the sequence is nonnegative and nondecreasing so it converges in
So Hence (P3), (P4) reduce to the same problem. Similarly (P5) reduces to it. Let be an optimal solution of problem (P1) and suppose it is not an optimal solution of problem (P3). So there exists such that that is
Let and Construct and as in (a). Thus
Hence we obtain so which contradicts the hypothesis so is an optimal solution of problem (P3).
Following Michel, [22], for all and for all we define as the set of the for which there exists satisfying We also define as the set of the for which there exists satisfying for all
Theorem 4.5.
Let for all Let be an optimal solution of problem (P1). One assumes the following conditions fulfilled.

(i)
on

(ii)
For all there exists

(iii)
For all the mappings and are Fréchetdifferentiable at

(iv)
For all for all co where co denotes the convex hull.

(v)
For all is invertible.
Then there exists such that and
 (a)

(b)
for all for all
Remark 4.6.
Notice that condition (iv) is a convexity condition and that condition (ii) of Theorem 2.2 implies this condition (iv). Condition (ii) of Theorem 2.2 is equivalent to the following condition: for all the set is convex.
Proof.
Use Theorem 4.3 of this paper and apply Theorem 3 in BlotChebbi [5].
5. Sufficient conditions of optimality
Let for all
Theorem 5.1.
Let where is convex. One assumes that there exists and that the following conditions are fulfilled.

(i)
The mappings and for all are of class on
 (ii)
 (iii)

(iv)
The mapping is concave with respect to for all
Then is an optimal solution of (P1).
Proof.
Notice that from (ii), for all
Let For all one has
therefore, we obtain
Since for all is concave with respect to and one has Using hypothesis (iii) with gives and using hypothesis (iii), the first order necessary condition for the optimality of is Thus one has
The hypothesis implies and since and belong to one has Hence we obtain so That is
Corollary 5.2.
Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:
 (v)
then is a solution of (P3) (resp., (P4)).
Notice that if with we obtain that is a solution of (P5).
One can weaken the hypothesis of concavity of with respect to and and replace it by the concavity of with respect to as the following theorem shows. (See [23] for a quick survey of sufficient conditions.)
Let
The maximum is attained since is compact.
Theorem 5.3.
Let One assumes that there exists and that the following hypotheses are fulfilled.

(i)
For all the mappings and for all are of class on

(ii)
Also (iii) of the previous theorem.

(iv)
The mapping is concave with respect to for all
Then is an optimal solution of (P1).
Proof.
Let and let For all one has
by the definition of and noticing that So we obtain
(Notice that Now using (see Seierstad and Sydsaeter [24, page 390]) we obtain The concavity of with respect to gives Finally follows as in the proof of the previous theorem.
Corollary 5.4.
Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:
 (v)
then is a solution of (P3) (resp., (P4)).
Notice that if with we obtain that is a solution of (P5).
References
 1.
Pontryagin L, Boltianski V, Gramkrelidze R, Mitchenko E: Théorie Mathématique des Processus Optimaux. Mir, Moscow, Russia; 1974.
 2.
Ekeland I, Scheinkman JA: Transversality conditions for some infinite horizon discrete time optimization problems. Mathematics of Operations Research 1986, 11(2):216229. 10.1287/moor.11.2.216
 3.
Carlson DA, Haurie AB, Leizarowitz A: Infinite Horizon Optimal Control. Deterministic and Stochastic Systems. 2nd edition. Springer, Berlin, Germany; 1991.
 4.
Zaslavski AJ: Turnpike Properties in the Calculus of Variations and Optimal Control, Nonconvex Optimization and Its Applications. Volume 80. Springer, New York, NY, USA; 2006:xxii+395.
 5.
Blot J, Chebbi H: Discrete time Pontryagin principles with infinite horizon. Journal of Mathematical Analysis and Applications 2000, 246(1):265279. 10.1006/jmaa.2000.6797
 6.
Blot J, Hayek N: Infinitehorizon Pontryagin principles with constraints. In Communications of the Laufen Colloquium on Science, Laufen, Austria, Berichte aus der Mathematik, 2. Edited by: Ruffing A, Suhrer A, Suhrer J. Shaker, Aachen, Germany; 2007:114.
 7.
Araujo A, Scheinkman JA: Smoothness, comparative dynamics, and the turnpike property. Econometrica 1977, 45(3):601620. 10.2307/1911677
 8.
Araujo A, Scheinkman JA: Notes on comparative dynamics. In General Equilibrium, Growth, and Trade. Edited by: Green JR, Scheinkman JA. Academic Press, New York, NY, USA; 1979:217226.
 9.
Michel P: Some clarifications on the transversality condition. Econometrica 1990, 58(3):705723. 10.2307/2938197
 10.
Stokey NL, Lucas RE Jr., Prescott EC: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, Mass, USA; 1989:xviii+588.
 11.
Grinold R: Convex infinite horizon programs. Mathematical Programming 1983, 25(1):6482. 10.1007/BF02591719
 12.
Boltianski V: Commande Optimale des Systèmes Discrets. Traduit du Russe par A. Sossinsk. Mir, Moscow, Russia; 1976:467.
 13.
Chichilnisky G: An axiomatic approach to sustainable development. Social Choice and Welfare 1996, 13(2):231257. 10.1007/BF00183353
 14.
Chichlinisky G: What is sustainable development? Land Economics 1997, 73(4):46791.
 15.
Debreu G: Valuation equilibrium and Pareto optimum. Proceedings of the National Academy of Sciences of the United States of America 1954, 40(7):588592. 10.1073/pnas.40.7.588
 16.
Peleg B, Ryder H: On optimal consumption plans in a multisector economy. Review of Economics Studies 1972, 39(2):159169. 10.2307/2296868
 17.
Ioffe AD, Tihomirov VM: Theory of Extremal Problems, Studies in Mathematics and Its Applications. Volume 6. NorthHolland, Amsterdam, The Netherlands; 1979:xii+460.
 18.
Alexéev V, Tikhomirov V, Fomine S: Commande Optimale. Mir, Moscow, Russia; 1982:447.
 19.
Blot J, Crettez B: On the smoothness of optimal paths. Decisions in Economics and Finance 2004, 27(1):134. 10.1007/s1020300400425
 20.
Aliprantis CD, Border KC: InfiniteDimensional Analysis. 2nd edition. Springer, Berlin, Germany; 1999:xx+672.
 21.
Blot J, Cartigny P: Optimality in infinitehorizon variational problems under sign conditions. Journal of Optimization Theory and Applications 2000, 106(2):411419. 10.1023/A:1004611816252
 22.
Michel P: Programmes mathématiques mixtes. Application au principe du maximum en temps discret dans le cas déterministe et dans le cas stochastique. RAIRO Recherche Opérationnelle 1980, 14(1):119.
 23.
Blot J, Hayek N: Sufficient conditions for infinitehorizon calculus of variations problems. ESAIM: Control, Optimisation and Calculus of Variations 2000, 5: 279292.
 24.
Seierstad A, Sydsaeter K: Sufficient conditions in optimal control theory. International Economic Review 1977, 18(2):367391. 10.2307/2525753
Author information
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Received
Accepted
Published
DOI
Keywords
 Transversality Condition
 Adjoint Equation
 Pontryagin Maximum Principle
 Adjoint Variable
 Finite Horizon