Inversion of multilayer synchronous networks is a method which tries to answer questions like “What kind of input will give a desired output?” or “Is it possible to get a desired (output under special input/output constraints)?”. We will describe two methods of inverting a connectionist network. Firstly, we extend inversion via backpropagation (Linden/Kindermann [4], Williams [11]) to recurrent (Elman [1], Jordan [3], Mozer [5], Williams/Zipser [10]), time-delayed (Waibel at al. Secondly, we introduce a new inversion method for proving the non-existence of an input combination under special constraints, e.g. in a subspace of the input space. This method works by iterative exclusion of invalid activation values. It might be a helpful way to judge the properties of a trained network. We conclude with simulation results of three different tasks: XOR, morse signal decoding and handwritten digit recognition.
This paper examines the use of two characteristic neural network architectures to signal and image processing applications. Digital image halftoning and seismic event detection are treated as optimization problems, to which symmetric Hopfield-type networks with near-neighbor-connections provide efficient solutions. A solution to the halftoning problem, provided by simulated annealing, is also examined and compared to the neural network one. Feedforward multilayered networks are examined in the form of auto-associative memories, using the same input and desired output data. The ability of such networks to compress sequences of image frames, having been trained over a small number of them, is specifically examined in the paper. The performance of a network trained by the backpropagation learning algorithm is compared to that of a counterpropagation network applied to the same sequence of real images.
A simple method for training the dynamical behavior of a neural network is derived. It is applicable to any training problem in discrete-time networks with arbitrary feedback. The algorithm resembles back-propagation in that an error function is minimized using a gradient-based method, but the optimization is carried out in the hidden part of state space either instead of, or in addition to weight space. A straightforward adaptation of this method to feedforward networks offers an alternative to training by conventional back-propagation. Computational results are presented for some simple dynamical training problems, one of which requires response to a signal 100 time steps in the past.
