A Note on Optimality Conditions for Control Problems with Parameters

In this paper we study optimal control problems characterized by two different kinds of decision instruments: control functions and parameters. The former are piecewise continuous functions, the latter are constants in the whole programming interval. We suggest that, even if the theory for these problems has been developed a few decades ago, the standard sufficient conditions are not completely satisfactory. The critical issue is strictly connected with the approach used to prove the necessary conditions. A historical review of these conditions contextualizes our discussion. We believe that the sufficiency issue for this class of problems is still open to developments. Mathematics Subject Classification: 49J15, 49K30


Introduction
Optimal control problems with parameters are well known and were initially called "Optimal Processes with Parameters" in § 25 of the fundamental book "The Mathematical Theory of Optimal Processes" by Pontryagin [7, pp. 191-196].In that section Pontryagin and his coauthors introduce this kind of problems, for which they prove a version of the standard necessary conditions.Consistently with the interest at those times, they do not discuss sufficient conditions: only later this kind of conditions will be introduced by Mangasarian for general control problems [6], and generalized by Arrow [1, p. 92] and Arrow and Kurz [2, p. 43].A satisfactory proof of the latter was provided by Seierstad and Sydsaeter [12].
Pontryagin's book has been translated and reprinted several times and it may be surprising to know that its § 25, on Optimal Processes with Parameters, has appeared in two different versions.We can read the first version of statement and proof of the necessary conditions for optimal processes with parameters in the English translations by Trigoroff [7], published in 1962, and by Brown [8], published in 1964.By the way, Trigoroff's translation is the most known and cited, and has been reprinted in 1986 [11].We can find the second version in the French translation by Embarek [9], published in 1974, and in the fourth edition [10] in Russian, published in 1983.It is remarkable that the first version (see [7,8] in English) contained a mistake, whereas the second one (see [9] in French, and [10] in Russian) presents the right result.As far as we know, nobody signalled the change, perhaps because it concerns a special topic without any consequence on the remaining parts of an extraordinary book.Therefore, the wrong version could be reprinted, ignoring the correction already made by the authors.
Other authors tackled the same question of the necessary conditions for optimal control problems with parameters through a different approach, perhaps because they felt the available version unsatisfactory.Leitmann, in his book [4, § 13.14], proposed a straightforward proof which we will recall in Section 2 of this paper.The same idea can be found exploited in the book by Léonard and Long [5, § 7.11].In neither of these references there is anything about sufficient conditions.To the best of our knowledge, the first book where sufficient conditions for optimal control problems with parameters are discussed is that by Seierstad and Sydsaeter [13, § 3.2, Exercise 3.2.8].In fact, the proof of these conditions, as well as the necessary ones, is left as an exercise in [13], with a hint which goes in the direction of the proof of necessary conditions proposed by Leitmann [4].We believe that Leitmann's idea to prove the necessary conditions is very smart, but it also introduces a bias in the reasoning for the sufficient conditions.In this paper we try to analyze the logical process on which the sufficient conditions are based.
In Section 2, we describe a very simple instance of optimal control problem with parameters, in order to illustrate the idea introduced by Leitmann to prove the necessary conditions.In Section 3, following the same idea, we prove the sufficient conditions.Then, using a simple example, we explain why we think that something is still missing about this topic.

Optimal Processes with Parameters
We consider the simplest optimal control problem with parameters.It is both simple to analyze and essential in representing the relevant features to discuss.Moreover, it is useful to understand the idea behind the proof of necessary conditions and, at the same time, the statement and proof of the sufficient conditions.Let us consider the following optimal control problem max where decisions are represented by the control function u (•) and the parameter q.We make strong assumptions on the functions g and f as we want to use the standard necessary conditions avoiding troubles with regularity.More in detail, we assume that: →R is continuously differentiable and Lipschitz continuous in the variable x, uniformly with respect to all the other variables: i.e. for all Under these assumptions for all piecewise continuous functions u (•) from [0, T ] to U and for all q ∈ Q there exists a unique continuous function x (•) which satisfies the motion equation.Moreover, the objective functional is well-defined for all state functions x (•) associated to feasible controls u (•) and parameter values q (see [3, Ch. 22, Hypothesis 22.1] for a more general discussion on the required assumptions for the data of an optimal control problem).
The necessary conditions for this problem are stated in the following theorem.
Theorem 2.1 Let us define the Hamiltonian function h (t, u, q, x, p) := g (t, u, q, x) + p f (t, u, q, x) If the control-parameter-state triplet (u * (•) , q * , x * (•)) is optimal, then the following inequality holds for all u ∈ U h (t, u * (t) , q * , x * (t) , p (t)) ≥ h (t, u, q * , x * (t) , p (t)) where the adjoint function p (•) satisfies the adjoint equation and transversality condition and the parameter q * satisfies the condition The proof of Theorem 2.1 is based on the idea presented by Leitmann in [4, § 13.14].Along the same lines is the proof by Léonard and Long [5, p. 257].We recall it in detail because it is the basis for stating the sufficient conditions, and we believe that it is the reason why the sufficient conditions that we know may not appear so satisfactory.
Proof.Let us introduce the auxiliary problem max where we define Here, the parameter q becomes a constant state function y (•) with free initial value.We observe that the control-parameter-state triplet (u * (•) , q * , x * (•)) is optimal for (1) if and only if the control-state triplet (u * (•) , x * (•) , y * (•)) with y * (•) ≡ q * is optimal for (5).Therefore we can use the necessary conditions for the standard optimal control problem (5) to obtain the necessary condition for problem (1) with parameters.We notice that in (5) We know that y * (•) ≡ q * ; hence, recalling also the definitions of H and S, we observe that the first adjoint function and transversality condition can be written as and this implies that the adjoint function p (•) introduced in the statement of the theorem coincides with p 1 (•).
Seierstad and Sydsaeter [13, § 3.2, Exercise 3.2.8]suggest a formally different approach, which requires to add a state function and an auxiliary control, assigning to the latter the role of decision variable originally possessed by the parameter.On the other hand, the approach by Pontryagin et al. [9,10], is based on the same variational argumentation as used to prove the Maximum Principle.

Sufficient conditions
As we said, Seierstad and Sydsaeter [13, Exercise 3.2.8,p. 192] provide also sufficient conditions for problem (1), leaving the proof as an exercise.In order to prove them, we refer again to Leitmann's auxiliary problem, already used for the necessary conditions.This will clarify how deeply the sufficiency result is connected with the proof of the necessary conditions.
Proof.In the proof of Theorem 2.1, we have stated the equivalence of problems (1) and (5).Therefore, we need now to prove sufficient conditions for the auxiliary problem (5) and state them in the frame of the original problem (1).The Arrow-type sufficient conditions for it, in the general form presented in [3,Ch. 24,Th.-the scrap value function S(x, y) is concave.This is equivalent to the statement of the theorem.
The discussion of both Theorems 2.1 and 3.1 makes it clear that the parameter q must be formally treated as a constant state function.This implies that the Hamiltonian function must be maximized with respect to the control variable u only, instead of the control-parameter pair (u, q), and the resulting function must be concave in (x, q), and not simply in x.We have the feeling that the condition of concavity in (x, q) to guarantee optimality is too strong, because it seems to overlook the special simplicity of the constant q.Therefore, we believe that it is reasonable to search for sufficient conditions which pose weaker restrictions on the functions involved with the parameter q.
We conclude this section providing an example where we cannot apply the sufficient conditions because of their restrictive hypotheses.
Example.Let us consider the following control problem with a parameter: with U = Q = (0, +∞).The Hamiltonian function is h (t, u, q, x, p) := −q 2 u 2 /2 + pu and therefore arg max u∈R h (t, u, q, x, p) = p/q 2 , the adjoint equation is and the optimal state must satisfy the following equation: After some manipulations we obtain u * (t) ≡ q + 1 q 2 + 2 (8) p (t) ≡ q 2 (q + 1) q 2 + 2 x * (t) = q + 1 q 2 + 2 t Even if it is not surprising that the sufficient conditions cannot apply to all optimal control problems, it is quite unusual that a direct computation can prove the optimality without any troubles whereas the sufficient conditions fail.
In our personal opinion, the sufficient conditions for optimal control problems with parameters have very strong assumptions and these assumptions are biased by the proof of the necessary conditions.Further analysis is needed: we think that more special sufficient conditions could be obtained working without the auxiliary problem, for example facing the problem with the approach introduced by Pontryagin and his coauthors.