Professional Documents
Culture Documents
ISSN: 2010-1791
1. Introduction
The theory of dynamics games is concerned with multi-person decision making. The
principal characteristic of a dynamic game is that involves a dynamic decision process evolving
in time (continuous or discrete), with more than on decision maker, each with its own cost
function and possibly having access to different information. Dynamic game theory adopts
characteristics from game theory and optimal control theory, although it is much more versatile
than each of.
Differential games belong to a subclass of dynamic games called games in the state space.
In a game in the state space, the modeler introduces a set of variables to describe the state of a
dynamic system, at any particular instant of time in which the game takes place. The systematic
study of the problems of differential games was initiated by Isaacs in 1954.
After development of the maximum principle of Pontryagin's maximum principle, it became
clear that there was a connection between differential games and optimal control theory. In fact,
the differential game problems are a generalization of the optimal control problems in cases
where more than one driver or player. However, differential games are conceptually much more
complex than optimal control problems in that it is not as what constitutes a solution. There are
different kinds optimal solutions for problems such as differential games minimax solution,
Nash equilibrium, Pareto equilibrium, depending on the characteristics of the games (see e.g.,
Tolwinski (1982) and Haurie, Tolwinski, and Leitman (1983)).
We present some results on differential games cooperative and non-cooperative differential
games, and theirs "optimal" solutions. In particular we will study those that relate Pareto
equilibrium and Nash equilibrium (non-cooperative games), although other types of cooperative
and non-cooperative games, for example, commitment games, Stackelberg games, to name a
few.
2. Preliminary in optimal control theory
As mentioned above, optimal control problems are a special class of differential games
played and a cost criterion. In this section we study some basic results on optimal control theory:
dynamic programming and the maximum principle, since these results are determining in
dynamic game theory.
397
ISSN: 2010-1791
(1)
where
is called the objective function or cost criteria. This is, in own words, the problem is find the
admissible control , which Maximizes the objective function, subject to the state equation and the
control constraints
(2)
Usually the set
is determined by constraints (physical, economic, biological, etc.) on the
values of the control variables
at time . The control is called the optimal control
and , determined by means of state equation with
, is called the optimal trajectory or an
optimal path.
2.2. Dynamic Programming and the Maximum Principle.
Dynamic programming is based on Bellman's principle of optimality (Richard Bellman in
1957 stated this principle in his book on dynamic programming)
An optimal policy has the property that, whatever the initial
state and initial conditions are, the remaining decision must
constitute an optimal policy with regard to the outcome
resulting from the first decision.
Let us consider the optimal control problem (1). The principle of maximum can be derived from
Bellman's principle of optimality (see [45]). We state the principle of maximum as follows
398
ISSN: 2010-1791
(3)
(4)
(5)
where the so-called Hamiltonian is defined as
(6)
The maximum principle states that under certain assumptions there exists for every optimal
control path
a trajectory
such that the maximum condition, the adjoin equation, and
transversality condition (eq. 4) are satisfied. To obtain a sufficiency theorem we augment these
conditions by convexity assumptions. This yields the following theorem.
Theorem 2. Consider the optimal control problem given by the equation (1), (2), (3) and define
the Hamiltonian function like in (7), and the maximized Hamiltonian function
(7)
Assume that the state space is a convex set and that is continuously differentiable and
concave. Let
be a feasible control path with corresponding state trajectory
. If there exists
an absolutely continuous function
such that the maximum condition
(8)
the adjoint equation
(9)
399
ISSN: 2010-1791
(11)
3.
The general -player (deterministic) differential game time is described by the state equation
(12)
and the cost functional for each player
for
It is assumed that all players know the state equation as well as the cost functionals.
Example 1. In a two-firm differential game with one state variable
time according to the differential equation
in which
are scalar control variables of firm 1 and 2, respectively. The state variable
represents the number of customers that firm 1 has at time and
is the constant size of the
total market. Hence
is the number of customers of firm 2. The control variables
are the firm`s respective advertising effort rates at time . The differential equation, in this case, can
be interpreted in the following way: the number of customers of firm 1 tends to increase by the
advertising efforts of firm 1 since these efforts attract customers from firm 2. On the other hand, the
advertising efforts of firm 2 tend to draw away customers from firm 1.
Payoffs are given by
400
ISSN: 2010-1791
in which
represent firm i's unit revenues. The second term in the integrand of is a convex
advertising cost function of firm . Feasibility requires that
and
are not negative. Each
firm wishes to choose its advertising strategy over
so as to maximize its payoff. The payoff is
simply the present value of a firm's profit on the horizon.
Remark. In this game, the rival firm's actions do not influence a firm's payoff directly but only
indirectly through the state dynamics.
where
is nondecreasing in .
belongs to
for
of player
Some types of standard information structures that arise in deterministic differential games are
stated in following definition.
401
(i) open-loop if
(ii) closed-loop if
(iii)
memory less perfect state if
(iv) feedback if
, for
ISSN: 2010-1791
,
for
The following theorem provides a set of conditions under which the problem given by
equations (12) and (13), admits a unique solution for every -tuple
Theorem 3. Let the information structure for each pair be any one of the information patterns of
the definition above. Furthermore, let
then if
(i)
is continuous in
for each
,
(ii)
(iii)
for
uniformly Lipschitz in
is uniformly Lipschitz in
is continuous in for each
the differential equation (12) admits a single state trajectory for every
, and furthermore this unique trajectory is continuous.
,
and
, so that
As mentioned above, the differential games can be classified in two classes: Cooperative
games and non-cooperative games. In a cooperative game the players wish to cooperate to reach a
result that will be beneficial to all.
3.2.1 Pareto Equilibrium
Definition 2. Let us consider a game with players. Let
) be the player cost function,
given the initial state
and that the players follow the multi-strategy
. Let
be the set of admissible strategies for the player
and
and
(14)
where
. An admissible strategy
is called Pareto-optimal if there is if it does not exist another
such that
(15)
402
ISSN: 2010-1791
Figure 1.
We can to see in figure 1, that the pair ( , ) with the cost vector V=V( , ) is not a Pareto
equilibrium, since there exist other points in that are "below" V.
Let
be the set of Pareto equilibrium (which supposed to be not empty). The set of vectors
that minimizes
, i.e.
(17)
then
is a Pareto equilibrium.
3.3. Non-cooperative Games
In a non-cooperative game the players act independently and each one wishes to optimize his
own objective function, i.e. players are rivals and all players act in their own best interest, paying no
attention whatsoever to the fortunes of the other players.
403
ISSN: 2010-1791
where
and
and
which player
wants to maximize and player
wants to minimize. Since the gain of player
represents a loss to player , such games are named zero-sum games (because the sum of their cost
functional is identically zero). Thus, we are looking for admissible control trajectories
and
such that
(20)
The solution
is known as a saddle point but some authors call it the minimax solution.
Here
and
stand for
and ,
, respectively.
3.3.2. Nash equilibrium
First, we consider the case
Definition 3. Let
to the strategy
as
(two players)
(21)
Similarly, the set of the optimal responses of player 2 to a strategy
as
(22)
A multi-strategy
404
Equivalently,
ISSN: 2010-1791
is a Nash equilibrium if
(23)
and
(24)
In our words, in Nash equilibrium, a player can't improve his situation if he alters his strategy
unilaterally.
Generalizing to any finite number
Definition 4. A multi-strategy
the following inequalities hold for all
The interpretation of a Nash equilibrium solution is as follows: If one player tries to alter his
strategy unilaterally, he can't improve his performance by such a change. In this sort of situation
each player is just interested in his own performance, that is, the game is played non-cooperatively.
Definition 5. Let we consider a dynamical game with players and let it be
objective function for the player
since the initial condition of the game is
in the time
. Let it be
, a markovian multi-strategy, that is to say, each
markovian (or feedback). It's said that is a perfect Nash Equilibrium if, for each
any initial condition
, is hold that
the
it is
and
(25)
where the infimum is calculated above all the marcovians strategies
of the player .
405
ISSN: 2010-1791
Let
be the
is a
We can note that this reduce the problem to one border problem with two border conditions
that in some instances it can be solved explicitly. For example, Clemhout and Wan (1974) consider
games three-linear, called thus because the Hamiltonian is linear in the state, in the controls, and in
the variable attaches. Also, Dockner et al. (1985) identify several types of differential games that
they are soluble, in the sense of the fact that they can be determined balances of Nash of open loop,
either explicitly.
Example of perfect Nash equilibrium
Let us consider a differential game with
players and finite horizon . To save on
notation, denote the control variables of the two players by and instead of and . The state
space is
, the initial state is a fixed number
, and the set of feasible controls is
for player 1 and
for player 2. The objective functionals are
406
ISSN: 2010-1791
and
Let us first try to find open-loop Nash equilibrium of the above game, that is, a pair
where
and
are the strategies for player 1 and player 2,
respectively. If player 2 chooses to play
then player 1's problem can be written as
Maximize
Subject to
Since
(27)
to the maximization of
the problem
Maximize
Subject to
with
(28)
and
407
where
ISSN: 2010-1791
still to be determined.
Maximize
Subject to
(30)
Denoting by the costate variable of player 2, the Hamiltonian function for this problem is
given by
. Maximization with respect to
yields
if
and
if
. If
then
, independently of . These properties imply that the maximized Hamiltonian function is
given by
The adjoin equation and transversality condition for player 2's problem are
Using
The boundary value problem consisting of (28) and (31) has a unique solution which is given
by
with
from (29). The function
is easily seen to be non-positive and
strictly increasing on
It depends on the parameters
and whether
for all
or whether
) can be smaller than
for some
. Because of the monotonicity
of
, however, we know that in the latter case there exists a number
such that
for
and
for all
Careful analysis of (31) reveals that such a
number exists if and only if
and
408
where
and
ISSN: 2010-1791
To verify that the above candidate is an open-loop Nash equilibrium it suffices therefore to
prove that
is an optimal control path in player 2's problem. This, however, follows from
theorem 2 by noting that
for all , which shows that the maximized Hamiltonian
function
) of player 2's problem is a concave function with respect to . This concludes
the derivation of an open-loop Nash equilibrium for this example.
4. References
[1]
Amir, R. (2003). Stochastic games in economics and related fields: an
overview. In Neyman and Sorin (2003), Chapter 30.
[2]
Arkin, V.L, Evstigneev, LV. (1987). Stochastic Models of Control and
Economic Dynamics. Academic Press, London.
[3]
Balbus, L., Nowak, A.S. (2004). Construction of Nash equilibrium in
symmetric stochastic games of capital accumulation. Math. Meth. Opero Res. 60, pp.
267-277.
[4]
Basar, T. Editor (1986). Dynamic Games and Applications in Economics.
Lecture Notes in Economics and Mathematical Systems 265, Springer-Verlag, Berlin.
[5]
Basar, T, Olsder, G.J. (1999). Dynamic Non-cooperative Game Theory,
Second Edition. SIAM, Philadelphia. (The first edition was published by Academic
Press, in 1982.)
[6]
Bellman, R. (1956). Dynamic Programming. Princeton University Press,
Princeton, N.J.
[7]
Bernhard, P. (2005). Robust control approach to option pricing, including
transaction costs. In Nowak and Szajowski (2005), pp. 391-416.
[8]
Dockner, E., Feichtinger, G., Jorgensen, S. (1985). Tractable classes of
nonzero sum open-loop Nash differential games: theory and examples. J. Optim. Theory
Appl. 45, pp. 179-197.
[9]
Dockner, E.J., Jorgensen, S., Long, N.V., Sorger, G. (2000). Differential
Games in Economics and Management Science. Cambridge University Press,
Cambridge, U.K.
[10]
Feichtinger, G., Jorgensen, S. (1983). Differential game models in
management. Euro. J. Opero Res. 14, pp. 137-155.
[11]
Fershtman, c., Mullar, E. (1986). Tumpike properties of capital accumulation
games. J. Econ.' Th. 38, pp. 167-177.
409
ISSN: 2010-1791
[12]
Filar, J.A., Petrosjan, L.A. (2000). Dynamic cooperative games. Internatl.
Game Theory Rev. 2, pp. 47-65.
[13]
Filar, J., Vrieze, K. (1997). Competitive Markov Decision Processes.
Springer Verlag, New York.
[14]
Fleming, W.H., Rishel, R.W. (1975). Deterministic and Stochastic Optimal
Control. Springer-Verlag, NewYork.
[15]
Fleming, W.H., Soner, H.M. (1992). Controlled Markov Processes and
Viscosity Solutions. Springer-Verlag, New York.
[16]
Fudenberg, D., Tirole, J. (1991). Game Theory. MIT Press, Cambridge,
MA.Gaidov, S.D. (1986). Pareto-optimality in stochastic differential games. Problems of
Control and Information Theory 15, 439-450.
[17]
Gaidov, S.D. (1990). On the Nash-bargaining solution in stochastic
differential games. Serdica 16, pp. 120-125.
[18]
Isaacs, R. (1965). Differential Games. Wiley, New York.
[19]
Jorgensen, S., Sorger, G. (1990). Feedback Nash equilibrium in a problem of
optimal fishery management. J. Optim. Theory Appl. 64, pp. 293-310.
[20]
Jorgensen, S., Yeung, D.W.K. (1996). Stochastic differential game model of a
common property fishery. J. Optim. Theory Appl. 90 pp. 381-403.
[21]
Kaitala, V., Hamalainen, RP., Ruusunen, J. (1985). On the analysis of
equilibrium and bargaining in a fishery game. In Feichtinger (1985), pp. 593-606.
[22]
Kalai, E., Smorodinsky, M. (1975). Other solutions to Nash's bargaining
problem. Econometrica 43, pp. 513-518.
[23]
Kannan, D., Lakshmikantham, v., editors (2002). Handbook of Stochastic
Analysis and Applications. Dekker, New York.
[24]
Karatzas, L., Shreve, S.E. (1998). Methods of Mathematical Finance.
Springer-Verlag, New York.
[25]
Kirman, A.P., Sobel, M.J. (1974). Dynamic oligopoly with inventories.
Econometrica 42, pp. 279-287.
[26]
Kuhn, H.W. Szego, G.P., editors (1971). Differential Games and Related
Topics. North-Holland, Amsterdam.
[27]
Leitmann, G. (1974). Cooperative and Non-cooperative Many Players
Differential Games. Springer-Verlag, New York.
[28]
Le Van, E., Dana, R-A. (2003). Dynamic Programming in Economics.
Kluwer, Boston.
[29]
Nash, J. (1950a). Equilibrium points in N-person games. Proc. Natl. Acad.
Sci. 36, pp. 48-49.
[30]
Nash, J. (1950b). The bargaining problem. Econometrica 18, pp. 155-162.
[31]
Nash, J. (1951). Non-cooperative games. Ann. Math. 54, pp. 286-295.
[32]
Nash, J. (1953). Two-person cooperative games. Econometrica 21, pp. 128140.
410
ISSN: 2010-1791
[33]
Neck, R (1982). Dynamic systems with several decision makers. In
Operations Research in Progress, ed. by G. Feichtinger and P. Kall, Reidel, New York,
pp. 261-284.
[34]
Neyman, A, Sorin, S., editors (2003). Stochastic Games and Applications.
Kluwer, Dordrecht.
[35]
Nowak, AS., Szajowski, P. (2003). On Nash equilibrium in stochastic games
of capital accumulation. In Stochastic Games and Applications, Volume 9, edited by L.A
Petrosjan and Y.Y. Mazalov, Nova Science, pp. 118-129.
[36]
Nowak, AS., Szajowski, K., editors (2005). Advances in Dynamic Games.
(Annals of the International Society of Dynamic Games, vol. 7) Birkhauser, Boston.
[37]
Petrosyan, L.A (2003). Bargaining in dynamic games. In Petrosyan and
Yeung (2003), pp. 139-143.
[38]
Petrosyan, L.A. (2005). Cooperative differential games. In Nowak and
Szajowski (2005), pp. 183-200.
[39]
Petrosyan, L.A. Zenkevich, N.A (1996). Carne Theory. World Scientific,
Singapore.
[40]
Petrosyan, L.A. Yeung, D.W.K, Editor s (2003). ICM Millennium Lectures
on Games. Springer-Verlag, Berlin.
[41]
Pohjola, M. (1983). Nash and Stackelberg solutions in a differential game
model of capitalism J. Economic Dynamics and Control 6, pp. 173-186.
[42]
Ricci, G., Editor (1991). Decision Processes in Economics. Lecture Notes in
Economics and Mathematical Systems 353, Springer-Verlag, Berlin.
[43]
Roth, AE. (1979). Axiomatic Models of Bargaining. Springer-Verlag, Berlin.
[44]
Roth, AE. (1985). Game-Theoretic Models of Bargaining. Cambridge
University, Cambridge, U.K.
[45]
Sethi, S.P., Thompson, G.L. (2000). Optimal Control Theory: Applications to
Management Science and Economics, 2nd Edition. Kluwer, Boston.
[46]
Shapley, L. (1953). Stochastic games. Proc. Natl. Acad. Sci. 39, pp. 10951100.
[47]
Stokey, N.L., Lucas, R.E. (1989). Recursive Methods in Economic Dynamics.
Harvard University Press, Cambridge, MA.
[48]
Tabak, D., Kuo, B.C. (1971). Optimal Control by Mathematical Programming.
Prentice-Hall, Englewood Cliffs, N.J.
[49]
Tolwinski, B., Haurie, A., Leitmann, G. (1986). Cooperative equilibrium in
differential games. J. Math. Anal. Appl. 119, pp. 182-202.
[50]
Toussaint, B., (1985). The transversality condition at infinity applied to a
problem of optimal resource depletion. In Feichtinger (1985), pp. 429-440.
[51]
Vaisbord, E.M., Zhukovskii, V.I. (1988). Introduction to Multi-Player
Differential Games and Their Applications. Gordon and Breach, New York.
[52]
Vega-Amaya, O. (2003). Zero-sum average semi-Markov games: fixed-point
solutions of the Shapley equation. SIAM J. Control Optimal. 42, pp. 1876
411
ISSN: 2010-1791
[53]
Von Newmann, J., Morgenstern, O. (1944). The Theory of Games and
Economic Behavior. Princeton University Press, Princeton, N.J.
[54]
Young, J., Zhou, X.Y. (1999). Stochastic Controls: Hamiltonian Systems and
HJB Equations. Springer-Verlag, New York.
[55]
Yu, P.L., Leitmann, G. (1974). Compromise solutions, domination structures
Salukvadze's solution. J. Optim. Theory Appl. 3, pp. 362-378.
[56]
Zhukavskiy. V.L, Salukvadze, M.E. (1994). The Vector-Valued Maximin.
Academic Press, Boston.