Infinite Horizon Discrete Time Control Problems for Bounded Processes
© J. Blot and N. Hayek. 2008
Received: 15 October 2008
Accepted: 27 December 2008
Published: 9 February 2009
We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal control problems for bounded processes, for systems governed by difference equations. Results due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary conditions with weakened hypotheses of concavity and without invertibility, and we provide new results on the adjoint variable. We show links between bounded problems and nonbounded ones. We also give sufficient conditions of optimality.
The first works on infinite horizon optimal control problems are due to Pontryagin and his school . They were followed by few others [2–6]. We consider in this paper an infinite horizon Optimal Control problem in the discrete time framework. Such problems are fundamental in the macroeconomics growth theory [7–10] and see references of . Even in the finite horizon case, the discrete time framework presents significant differences from the continuous time one. Boltianski  shows that in the discrete time case, a convexity condition is needed to guarantee a strong Pontryagin Principle while this last one can be obtained without such condition in the continuous time setting. We study our problem in the space of bounded sequences which allows us to use Analysis in Banach spaces instead of using reductions to finite horizon problems as in [5, 6]. According to Chichlinisky [13, 14], the space of bounded sequences was first used in economics by Debreu . It can also be found in [7, 8, 16]. We obtain Pontryagin Maximum Principles in the strong form using weaker convexity hypotheses than the traditional ones and without invertibility . When we study the problem in a general sequence space it turns out that the infinite series will not always converge. Therefore we present other notions of optimality that are currently used, notably in the economic literature, see [3, 4, 9] and we show how our problem can be related to these other problems. We end the paper by establishing sufficient conditions of optimality.
Now we briefly describe the contents of the paper. In Section 2 we introduce the notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary conditions of optimality namely the existence of the adjoint variable in the space satisfying the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove these theorems through some lemmas and using results due to Ioffe-Tihomirov . In Section 4 we introduce some other notions of optimality for problems in the nonbounded case and we show links with our problem. For example, we show that when the objective function is positive then a bounded solution is a solution among the unbounded processes. Finally we give sufficient conditions of optimality for problems in the bounded and unbounded cases adapting for each case the approprate transversality condition.
2. Pontryagin maximum principles for bounded processes
For every define as the closure of the set of terms of the sequence If is compact. We set such that is thus the set of the bounded sequences which are in the interior of Note that is a convex open subset of since is open and convex. We set Define ; it is the set of admissible processes with respect to the considered dynamical system.
which can be written as follows.
For continuous time problems, one does not need conditions to obtain a strong Pontryagin maximum principle, both in the finite horizon case (see, e.g., ) and in the infinite horizon case (see, e.g., ). But for discrete time problems, strong Pontryagin principles cannot hold without an additional assumption namely a convexity condition, as Boltyanski shows in  for the finite horizon framework. Condition (ii) comes from the Ioffe and Tihomirov book . It generalizes the usual convexity condition used to garantee a strong Pontryagin maximum principle. The usual condition is: convex subset, concave with respect to and for every affine with respect to It implies condition (ii). In (iii) the condition is satisfied when is continuous (since is compact) and the condition is satisfied when exists and is continuous.
Conclusion (a) is the adjoint equation, conclusion (b) is the strong Pontryagin maximum principle and conclusion (c) is a transversality condition at infinity. In our case (c) is immediately obtained since is in but in general (nonbounded cases) it is very delicate to obtain such a conclusion. 
In the next theorem we consider the autonomous case. Thus the hypotheses are simpler and easier to manipulate.
3. Proofs of Theorems 2.1 and 2.2
Proof of Theorem 2.1
The first part of the proof goes through several lemmas.
For the proof see .
Assume that hypothesis (iii) of Theorem 2.1 holds. Then for one has Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold, then for all the mapping is of class on the ball in and for all one has
Recall that where consists of all singular functionals, see Aliprantis and Border . In fact it consists (up to scalar multiples) of all extensions of the "limit functional" to
From Lemma 3.6 and the previous results, conclusions (a) and (b) are satisfied.
Proof of Theorem 2.2.
Define on such that Under hypothesis (i) of Theorem 2.2, for all the mappings and are of class on The proof can be found in .
4. Results for unbounded problems
We study now problems of maximization over admissible processes which are not necessarily bounded when the optimal solution is bounded. So consider the following problems.
The optimality notion of (P3) is called "the strong optimality," that of (P4) is called "the overtaking optimality" and that of (P5) the "weak overtaking optimality" in  (in the continuous-time framework). Many existence results of overtaking optimal solutions and weakly overtaking optimal solutions are obtained in [3, 4]. In  there are also results in the discrete-time framework.
The two following assertions hold.
Proof. (a) Since a bounded optimal solution of (P2) or (P3) is an optimal solution of (P1). Suppose now that is a bounded optimal solution of (P4) that is for all Since this can be written for all and so in particular for all
Then one has
Remark 4.4. (b) shows that under a nonnegativity assumption, solving the problem in the space of bounded processes provides solutions for problems in spaces of admissible processes which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny  where problems are studied in the continuous-time case.
Following Michel, , for all and for all we define as the set of the for which there exists satisfying We also define as the set of the for which there exists satisfying for all
Notice that condition (iv) is a convexity condition and that condition (ii) of Theorem 2.2 implies this condition (iv). Condition (ii) of Theorem 2.2 is equivalent to the following condition: for all the set is convex.
Use Theorem 4.3 of this paper and apply Theorem 3 in Blot-Chebbi .
5. Sufficient conditions of optimality
One can weaken the hypothesis of concavity of with respect to and and replace it by the concavity of with respect to as the following theorem shows. (See  for a quick survey of sufficient conditions.)
Also (iii) of the previous theorem.
(Notice that Now using (see Seierstad and Sydsaeter [24, page 390]) we obtain The concavity of with respect to gives Finally follows as in the proof of the previous theorem.
- Pontryagin L, Boltianski V, Gramkrelidze R, Mitchenko E: Théorie Mathématique des Processus Optimaux. Mir, Moscow, Russia; 1974.Google Scholar
- Ekeland I, Scheinkman JA: Transversality conditions for some infinite horizon discrete time optimization problems. Mathematics of Operations Research 1986, 11(2):216-229. 10.1287/moor.11.2.216MATHMathSciNetView ArticleGoogle Scholar
- Carlson DA, Haurie AB, Leizarowitz A: Infinite Horizon Optimal Control. Deterministic and Stochastic Systems. 2nd edition. Springer, Berlin, Germany; 1991.MATHView ArticleGoogle Scholar
- Zaslavski AJ: Turnpike Properties in the Calculus of Variations and Optimal Control, Nonconvex Optimization and Its Applications. Volume 80. Springer, New York, NY, USA; 2006:xxii+395.Google Scholar
- Blot J, Chebbi H: Discrete time Pontryagin principles with infinite horizon. Journal of Mathematical Analysis and Applications 2000, 246(1):265-279. 10.1006/jmaa.2000.6797MATHMathSciNetView ArticleGoogle Scholar
- Blot J, Hayek N: Infinite-horizon Pontryagin principles with constraints. In Communications of the Laufen Colloquium on Science, Laufen, Austria, Berichte aus der Mathematik, 2. Edited by: Ruffing A, Suhrer A, Suhrer J. Shaker, Aachen, Germany; 2007:1-14.Google Scholar
- Araujo A, Scheinkman JA: Smoothness, comparative dynamics, and the turnpike property. Econometrica 1977, 45(3):601-620. 10.2307/1911677MATHMathSciNetView ArticleGoogle Scholar
- Araujo A, Scheinkman JA: Notes on comparative dynamics. In General Equilibrium, Growth, and Trade. Edited by: Green JR, Scheinkman JA. Academic Press, New York, NY, USA; 1979:217-226.Google Scholar
- Michel P: Some clarifications on the transversality condition. Econometrica 1990, 58(3):705-723. 10.2307/2938197MATHMathSciNetView ArticleGoogle Scholar
- Stokey NL, Lucas RE Jr., Prescott EC: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, Mass, USA; 1989:xviii+588.MATHGoogle Scholar
- Grinold R: Convex infinite horizon programs. Mathematical Programming 1983, 25(1):64-82. 10.1007/BF02591719MATHMathSciNetView ArticleGoogle Scholar
- Boltianski V: Commande Optimale des Systèmes Discrets. Traduit du Russe par A. Sossinsk. Mir, Moscow, Russia; 1976:467.Google Scholar
- Chichilnisky G: An axiomatic approach to sustainable development. Social Choice and Welfare 1996, 13(2):231-257. 10.1007/BF00183353MATHMathSciNetView ArticleGoogle Scholar
- Chichlinisky G: What is sustainable development? Land Economics 1997, 73(4):46791.Google Scholar
- Debreu G: Valuation equilibrium and Pareto optimum. Proceedings of the National Academy of Sciences of the United States of America 1954, 40(7):588-592. 10.1073/pnas.40.7.588MATHMathSciNetView ArticleGoogle Scholar
- Peleg B, Ryder H: On optimal consumption plans in a multi-sector economy. Review of Economics Studies 1972, 39(2):159-169. 10.2307/2296868MATHView ArticleGoogle Scholar
- Ioffe AD, Tihomirov VM: Theory of Extremal Problems, Studies in Mathematics and Its Applications. Volume 6. North-Holland, Amsterdam, The Netherlands; 1979:xii+460.Google Scholar
- Alexéev V, Tikhomirov V, Fomine S: Commande Optimale. Mir, Moscow, Russia; 1982:447.Google Scholar
- Blot J, Crettez B: On the smoothness of optimal paths. Decisions in Economics and Finance 2004, 27(1):1-34. 10.1007/s10203-004-0042-5MATHMathSciNetView ArticleGoogle Scholar
- Aliprantis CD, Border KC: Infinite-Dimensional Analysis. 2nd edition. Springer, Berlin, Germany; 1999:xx+672.MATHView ArticleGoogle Scholar
- Blot J, Cartigny P: Optimality in infinite-horizon variational problems under sign conditions. Journal of Optimization Theory and Applications 2000, 106(2):411-419. 10.1023/A:1004611816252MATHMathSciNetView ArticleGoogle Scholar
- Michel P: Programmes mathématiques mixtes. Application au principe du maximum en temps discret dans le cas déterministe et dans le cas stochastique. RAIRO Recherche Opérationnelle 1980, 14(1):1-19.MATHGoogle Scholar
- Blot J, Hayek N: Sufficient conditions for infinite-horizon calculus of variations problems. ESAIM: Control, Optimisation and Calculus of Variations 2000, 5: 279-292.MATHMathSciNetView ArticleGoogle Scholar
- Seierstad A, Sydsaeter K: Sufficient conditions in optimal control theory. International Economic Review 1977, 18(2):367-391. 10.2307/2525753MATHMathSciNetView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.