This paper considers the maximization of certain equivalent reward generated by a Markov decision process with constant risk sensitivity. First, value iteration is used to optimize possibly time-varying processes of finite duration. Then a policy iteration procedure is developed to find the stationary policy with highest certain equivalent gain for the infinite duration case. A simple example demonstrates both procedures.

This paper shows how the decision analysis approach can be used to determine the most economic method of carrying out computations or analyses. A primary decision problem is first formulated to obtain a structure for the analysis. Then several computational or analytical procedures, which can be used to analyze the primary decision problem in greater detail, are evaluated to select the most economic procedure. The purpose of each of these procedures is to increase the available information about uncertain parameters before making the primary decision, thereby yielding a "better" decision. Each procedure is evaluated by combining the value structure of the primary decision problem with a model of that procedure. The procedures considered in this paper are clairvoyance, complete analysis, Monte Carlo analysis, and numerical analysis. An example of a bidding problem is used to illustrate the results.