The Dynamic Programming Approach
THIS CHAPTER PRESENTS portfolio choice and asset pricing in the framework of dynamic programming, a technique for solving dynamic optimization problems with a recursive structure. The asset pricing implications go little beyond those of the previous chapter, but there are computational advantages. After introducing the idea of dynamic programming in a deterministic setting, we review the basics of a finite-state Markov chain. The Bellman equation is shown to characterize optimality in a Markov setting. The first-order condition for the Bellman equation, often called the “stochastic Euler equation,” is then shown to characterize equilibrium security prices. This is done with additive utility in the main body of the chapter, and extended to more general recursive forms of utility in the exercises. The last sections of the chapter show the computation of arbitrage-free derivative security values in a Markov setting, including an application of Bellman’s equation for optimal stopping to the valuation of American securities such as the American put option. An exercise presents algorithms for the numerical solution of term-structure derivative securities in a simple “binomial” setting.
To get the basic idea, we start in the T-period setting of the previous chapter, with no securities except those permitting short-term riskless borrowing at any time t at the discount dt > 0. The endowment process of a given agent is e. Given a consumption process c, it is convenient to define the agent’s wealth process Wc byand