Let dx = g(x, u, t) dt + dz be a general dynamical system with control u and where z is Brownian motion (dzdt = ξ is white Gaussian noise). Let the loss function be E ∝t0T k(x, u, τ) dτ, and let dydt = x(t) + ϵ(t), where ϵ(t) is white Gaussian noise, be the only observable quantity. In addition, let the initial probability density function of x(t0), P(a, t0), be given. Two forms of (Markov process) state space representations of this system are studied. The first is the conditional probability density function of x; the second is the set of moments of the density function.

A versatile and practical method of searching a parameter space is presented. Theoretical and experimental results illustrate the usefulness of the method for such problems as the experimental optimization of the performance of a system with a very general multipeak performance function when the only available information is noise-distributed samples of the function. At present, its usefulness is restricted to optimization with respect to one system parameter. The observations are taken sequentially; but, as opposed to the gradient method, the observation may be located anywhere on the parameter interval. A sequence of estimates of the location of the curve maximum is generated. The location of the next observation may be interpreted as the location of the most likely competitor (with the current best estimate) for the location of the curve maximum. A Brownian motion stochastic process is selected as a model for the unknown function, and the observations are interpreted with respect to the model. The model gives the results a simple intuitive interpretation and allows the use of simple but efficient sampling procedures. The resulting process possesses some powerful convergence properties in the presence of noise; it is nonparametric and, despite its generality, is efficient in the use of observations. The approach seems quite promising as a solution to many of the problems of experimental system optimization.

Let ξn be a sequence of independent vector-valued random variables. Let xn be a vector with components x0(n), …, xr(n), and let θn be a vector control. The maximum principle and canonical equations of Pontryagin are derived for the vector system xn = F(xn − 1, θn, ξn) with loss function Ex0(N), where N, the control time, is fixed. In the continuous case, it is derived for the form ẋ = ƒ(x, θ) + σξ, and loss Ex0(T), where σ is a nonnegative definite matrix and ξ is vector-valued white Gaussian noise. Under suitable smoothness assumptions, the expectation (conditioned upon xn) of the adjoint variables are the derivatives of the loss (min Ex0(N) conditioned upon xn).

Properties of a random walk model of an unknown function are studied. The model is suitable for use in the following (among others) problem. Given a system with a performance function of unknown, time varying, and possibly multipeak form (with respect to a single system parameter), and given that the only information available are noise perturbed samples of the function at selected parameter settings, then determine the successive parameter settings such that the sum of the values of the observations is maximum. An attempt to avoid the optimal search problem through the use of several intuitively reasonable heuristics is presented.