Infinite Horizon Discrete Time Control Problems for Bounded Processes

Abstract

We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal control problems for bounded processes, for systems governed by difference equations. Results due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary conditions with weakened hypotheses of concavity and without invertibility, and we provide new results on the adjoint variable. We show links between bounded problems and nonbounded ones. We also give sufficient conditions of optimality.

1. Introduction

The first works on infinite horizon optimal control problems are due to Pontryagin and his school [1]. They were followed by few others [2–6]. We consider in this paper an infinite horizon Optimal Control problem in the discrete time framework. Such problems are fundamental in the macroeconomics growth theory [7–10] and see references of [11]. Even in the finite horizon case, the discrete time framework presents significant differences from the continuous time one. Boltianski [12] shows that in the discrete time case, a convexity condition is needed to guarantee a strong Pontryagin Principle while this last one can be obtained without such condition in the continuous time setting. We study our problem in the space of bounded sequences which allows us to use Analysis in Banach spaces instead of using reductions to finite horizon problems as in [5, 6]. According to Chichlinisky [13, 14], the space of bounded sequences was first used in economics by Debreu [15]. It can also be found in [7, 8, 16]. We obtain Pontryagin Maximum Principles in the strong form using weaker convexity hypotheses than the traditional ones and without invertibility [5]. When we study the problem in a general sequence space it turns out that the infinite series will not always converge. Therefore we present other notions of optimality that are currently used, notably in the economic literature, see [3, 4, 9] and we show how our problem can be related to these other problems. We end the paper by establishing sufficient conditions of optimality.

Now we briefly describe the contents of the paper. In Section 2 we introduce the notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary conditions of optimality namely the existence of the adjoint variable in the space satisfying the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove these theorems through some lemmas and using results due to Ioffe-Tihomirov [17]. In Section 4 we introduce some other notions of optimality for problems in the nonbounded case and we show links with our problem. For example, we show that when the objective function is positive then a bounded solution is a solution among the unbounded processes. Finally we give sufficient conditions of optimality for problems in the bounded and unbounded cases adapting for each case the approprate transversality condition.

2. Pontryagin maximum principles for bounded processes

We first precise our notations. Let be a nonempty open convex subset of and a nonempty compact subset of Let and, for all We set

Define

For every define as the closure of the set of terms of the sequence If is compact. We set such that is thus the set of the bounded sequences which are in the interior of Note that is a convex open subset of since is open and convex. We set Define ; it is the set of admissible processes with respect to the considered dynamical system.

Let We consider first the following problem (P1):

(P1)

which can be written as follows.

(P1) Maximize when

Theorem 2.1.

Let be a solution of (P1). Assume the following.

• (i) For all the mapping is of class on and for all the mapping is Fréchet-differentiable on

• (ii) For all for all for all for all there exists such that

  

  (2.1)

• (iii) For any compact set there exists a constant such that for all for all for all and

• (iv) There exists such that and for all

  

  (2.2)

Then there exists such that

• (a)
• (b)
• (c)

Comments

For continuous time problems, one does not need conditions to obtain a strong Pontryagin maximum principle, both in the finite horizon case (see, e.g., [18]) and in the infinite horizon case (see, e.g., [5]). But for discrete time problems, strong Pontryagin principles cannot hold without an additional assumption namely a convexity condition, as Boltyanski shows in [12] for the finite horizon framework. Condition (ii) comes from the Ioffe and Tihomirov book [17]. It generalizes the usual convexity condition used to garantee a strong Pontryagin maximum principle. The usual condition is: convex subset, concave with respect to and for every affine with respect to It implies condition (ii). In (iii) the condition is satisfied when is continuous (since is compact) and the condition is satisfied when exists and is continuous.

Conclusion (a) is the adjoint equation, conclusion (b) is the strong Pontryagin maximum principle and conclusion (c) is a transversality condition at infinity. In our case (c) is immediately obtained since is in but in general (nonbounded cases) it is very delicate to obtain such a conclusion. [9]

In the next theorem we consider the autonomous case. Thus the hypotheses are simpler and easier to manipulate.

Theorem 2.2.

Let for all Let be a solution of (P1). Assume that the following conditions are fulfilled.

• (i) For all the mappings and are of class on

• (ii) For all for all for all for all there exists such that

  

  (2.3)

• (iii)

Then there exists such that the assertions (a), (b), and (c) of Theorem 2.1 are satisfied.

3. Proofs of Theorems 2.1 and 2.2

Proof of Theorem 2.1

First Part

The first part of the proof goes through several lemmas.

Lemma 3.1.

is well-defined and under hypothesis (i) of Theorem 2.1, for all the mapping is of class and one has, for all

For the proof see [19].

We set for all

Lemma 3.2.

Assume that hypothesis (iii) of Theorem 2.1 holds. Then for one has Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold, then for all the mapping is of class on the ball in and for all one has

Proof.

Let Let be the constant of (iii) with So for all So one has

Assume now that hypotheses (i) and (iv) of Theorem 2.1 hold. Let us show that is of class on Take Let be given. Let be such that Then, for all under (iv), which implies that

Let us now show that is Fréchet-differentiable on Take Let be given. Let be such that Then, for all under (iii). But this implies that Thus is Fréchet-differentiable at and

To show the continuity of at let be the constant of hypothesis (iii) corresponding to Let be given and let be such that So is of class

Lemma 3.3.

Under hypothesis (ii) of Theorem 2.1, for all for all for all there exists such that

(3.1)

Proof.

Let and Hypothesis (ii) of Theorem 2.1 implies for all the existence of such that

(3.2)

Therefore we obtain

(3.3)

Set so and satisfies the required relations.

Lemma 3.4.

Under hypotheses (i) and (iv) of Theorem 2.1,

Proof.

Since the problem is a problem of bounded solutions of first-order linear difference equations.

Let Assume that Then for all there exists a unique such that for all

(3.4)

where

Consider the operator such that for all

(3.5)

where

(3.6)

Recall that the norm of is defined by and that the norm of a linear operator between normed spaces is defined by

So So Since and is invertible so it is surjective.

Set Then under (iv) one has So is surjective that is

Recall that where consists of all singular functionals, see Aliprantis and Border [20]. In fact it consists (up to scalar multiples) of all extensions of the "limit functional" to

If then there exists such that for all ( being the space of convergent sequences having a limit in )

Lemma 3.5 ().

If then where for every So there exists such that for all

Second Part

Our optimal control problem can be written as the following abstract static optimisation problem in a Banach space:

(3.7)

that satisfies all conditions of Theorem 4.3, Ioffe-Tihomirov [17]. So we can apply this theorem and obtain the existence of not all zero, such that:

(3.8)

(AE) denotes the adjoint equation of this problem and (PMP) the Pontryagin maximum principle. They can be written, respectively:

(3.9)

Set where and

(AE) becomes:

(3.10)

So we get

(3.11)

Let be arbitrarily chosen in and let be in Consider the sequence defined as follows:

(3.12)

So one has if hence

Thus, it holds that

Now

Therefore, for all and for all one has

(3.13)

which implies

(3.14)

that is,

(3.15)

(PMP) becomes:

(3.16)

So for all

Consider, for all the sequences defined as follows:

(3.17)

Since the inequality holds for every we obtain

(3.18)

using as is of finite support.

Lemma 3.6.

().

Proof.

Recall we obtained the existence of not all zero, such that:

(3.19)

If then since

Hence We can set it equal to one.

From Lemma 3.6 and the previous results, conclusions (a) and (b) are satisfied.

Conclusion (c) is a straightforward consequence of the belonging of to

Lemma 3.7.

().

Proof.

Indeed we obtained for all Using one has for all Thus

Proof of Theorem 2.2.

Define on such that Under hypothesis (i) of Theorem 2.2, for all the mappings and are of class on The proof can be found in [19].

We consider the proof of Lemma 3.4 and we set Then the proof goes like that of Theorem 2.1.

4. Results for unbounded problems

We study now problems of maximization over admissible processes which are not necessarily bounded when the optimal solution is bounded. So consider the following problems.

(P2) Maximize on

(P3) Find such that, for all

(P4) Find such that, for all

(P5) Find such that, for all

The optimality notion of (P3) is called "the strong optimality," that of (P4) is called "the overtaking optimality" and that of (P5) the "weak overtaking optimality" in [3] (in the continuous-time framework). Many existence results of overtaking optimal solutions and weakly overtaking optimal solutions are obtained in [3, 4]. In [4] there are also results in the discrete-time framework.

Remark 4.1.

Notice that is an optimal solution of (P3) implies is an optimal solution of (P4) which implies is an optimal solution of (P5).

Moreover if is a bounded optimal solution of (P4) then (P3) and (P4) reduce to the same problem.

Lemma 4.2.

The two following assertions hold.

• (a) If is an optimal solution of problem (P2), (P3), (P4) or (P5) and then is an optimal solution of problem (P1). Therefore Theorem 2.1 applies.

• (b) Assume on If is an optimal solution of problem (P3) or (P4) and then

Proof. (a) Since a bounded optimal solution of (P2) or (P3) is an optimal solution of (P1). Suppose now that is a bounded optimal solution of (P4) that is for all Since this can be written for all and so in particular for all

In that case The proof is analogous for (P5).

(b) If is an optimal solution of problem (P3) and one has for all Since the sequence is increasing and since it is also upper bounded it converges in

So and

Theorem 4.3.

Let for all One assumes the following conditions fulfilled:

• (i) on

• (ii) For all there exists

Then one has

• (a)

• (b) If is an optimal solution of problem (P1), then it is an optimal solution of problems (P3), (P4), and (P5) which all reduce to the same problem.

Remark 4.4. (b) shows that under a nonnegativity assumption, solving the problem in the space of bounded processes provides solutions for problems in spaces of admissible processes which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny [21] where problems are studied in the continuous-time case.

Proof. (a) It is clear that the following inequality holds:

(4.1)

Let Let be given and let be such that Set

(4.2)

where is such that

and are bounded and

Since on one has

(4.3)

so we obtain

(4.4)

Since this is true fo all letting we obtain

(4.5)

(b) Since for all the sequence is nonnegative and nondecreasing so it converges in

So Hence (P3), (P4) reduce to the same problem. Similarly (P5) reduces to it. Let be an optimal solution of problem (P1) and suppose it is not an optimal solution of problem (P3). So there exists such that that is

(4.6)

Let and Construct and as in (a). Thus

Hence we obtain so which contradicts the hypothesis so is an optimal solution of problem (P3).

Following Michel, [22], for all and for all we define as the set of the for which there exists satisfying We also define as the set of the for which there exists satisfying for all

Theorem 4.5.

Let for all Let be an optimal solution of problem (P1). One assumes the following conditions fulfilled.

• (i) on

• (ii) For all there exists

• (iii) For all the mappings and are Fréchet-differentiable at

• (iv) For all for all co where co denotes the convex hull.

• (v) For all is invertible.

Then there exists such that and

• (a)

• (b) for all for all

Remark 4.6.

Notice that condition (iv) is a convexity condition and that condition (ii) of Theorem 2.2 implies this condition (iv). Condition (ii) of Theorem 2.2 is equivalent to the following condition: for all the set is convex.

Proof.

Use Theorem 4.3 of this paper and apply Theorem 3 in Blot-Chebbi [5].

5. Sufficient conditions of optimality

Let for all

Theorem 5.1.

Let where is convex. One assumes that there exists and that the following conditions are fulfilled.

• (i) The mappings and for all are of class on

• (ii)
• (iii)

• (iv) The mapping is concave with respect to for all

Then is an optimal solution of (P1).

Proof.

Notice that from (ii), for all

Let For all one has

(5.1)

therefore, we obtain

(5.2)

Since for all is concave with respect to and one has Using hypothesis (iii) with gives and using hypothesis (iii), the first order necessary condition for the optimality of is Thus one has

The hypothesis implies and since and belong to one has Hence we obtain so That is

Corollary 5.2.

Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:

• (v)

then is a solution of (P3) (resp., (P4)).

Notice that if with we obtain that is a solution of (P5).

One can weaken the hypothesis of concavity of with respect to and and replace it by the concavity of with respect to as the following theorem shows. (See [23] for a quick survey of sufficient conditions.)

Let

The maximum is attained since is compact.

Theorem 5.3.

Let One assumes that there exists and that the following hypotheses are fulfilled.

• (i) For all the mappings and for all are of class on

• (ii) Also (iii) of the previous theorem.

• (iv) The mapping is concave with respect to for all

Then is an optimal solution of (P1).

Proof.

Let and let For all one has

(5.3)

by the definition of and noticing that So we obtain

(5.4)

(Notice that Now using (see Seierstad and Sydsaeter [24, page 390]) we obtain The concavity of with respect to gives Finally follows as in the proof of the previous theorem.

Corollary 5.4.

Let (resp., ). If the hypotheses of the previous theorem are satisfied except that is replaced by and if the following hypothesis is also satisfied:

• (v)

then is a solution of (P3) (resp., (P4)).

Notice that if with we obtain that is a solution of (P5).