The design of a supervised neural network may be pursued in a variety of different ways. The back-propagation algorithm for the design of a multilayer perceptron (under supervision) as described in the previous chapter may be viewed as an application of an optimization method known in statistics as stochastic approximation. In this chapter we take a different approach by viewing the design of a neural network as a cumelfitting (approximation) problem in a high-dimensional space. According to this viewpoint, learning is equivalent to finding a surface in a multidimensional space that provides a best fit to the training data, with the criterion for “best fit” being measured in some statistical sense. Correspondingly, generalization is equivalent to the use of this multidimensional surface to interpolate the test data. Such a viewpoint is indeed the motivation behind the method of radial-basis functions in the sense that it draws upon research work on traditional strict interpolation in a multidimensional space. In the context of a neural network, the hidden units provide a set of “functions” that constitute an arbitrary “basis” for the input patterns (vectors) when they are expanded into the hidden-unit space; these functions are called radial-basisfunctions. Radial-basis functions were first introduced in the solution of the real multivariate interpolation problem. The early work on this subject is surveyed by Powell (1985). It is now one of the main fields of research in numerical analysis. Broomhead and Lowe (1988) were the first to exploit the use of radial-basis functions in the design of neural networks. Other major contributions to the theory, design, and application of radial-basis function networks include papers by Moody and Darken (1989), Renals (1989), and Poggio and Girosi (1990a). The paper by Poggio and Girosi emphasizes the use of regularization theory applied to this class of neural networks as a method for improved generalization to new data. The construction of a radial-basis function (RBF) network in its most basic form involves three entirely different layers. The input layer is made up of source nodes (sensory units). The second layer is a hidden layer of high enough dimension, which serves a different purpose from that in a multilayer perceptron. The output layer supplies the response of the network to the activation patterns applied to the input layer. The transformation from the input space to the hidden-unit space is nonlinear, whereas the transformation from the hidden-unit space to the output space is linear. A mathematical justification for this rationale may be traced back to an early paper by Cover (1965). In particular, we note from this paper that a pattern-classification problem cast in a high-dimensional space nonlinearly is more likely to be linearly separable than in a low-dimensional spacehence the reason for making the dimension of the hidden-unit space in an RBF network high. Through careful design, however, it is possible to reduce the dimension of the hidden-unit space, especially if the centers of the hidden units are made adaptive.

Magnetic objects can cause local variations in the Earth's magnetic field that can be measured with a magnetometer. Here we used triaxial magnetometer measurements and an analysis method employing wavelet techniques to determine the "signature" or "fingerprint" of different iron objects. Clear distinctions among the iron samples were observed. The time-dependent changes in the frequency powers were extracted by use of the Morlet wavelet corresponding to frequency bands from 0.1 to 100 Hz. (c) 2007 Elsevier B.V. All rights reserved.

The goal of the research presented in this thesis is to extract features, to filter and get fingerprints from signals detected by infrasound, seismic and magnetic sensors. If this can be achieved in a real time system, then signals from various events can be detected and identified in an otherwise torrent data. Several approaches have been analyzed. Wavelet transform methods are used together with ampligram and time scale spectrum to analyze infrasound, seismic and magnetic data. The energy distribution in the frequency domain may be seen in wavelet scalograms. A scalogram displays the wavelet coefficients as a function of the time scale and of the elapsed time. The ampligram is a useful method of presentation of the physical properties of the time series. The ampligram demonstrate the amplitude and phase of components of the signal corresponding to different spectral densities. The ampligram may be considered as an analogy to signal decomposition into Fourier components. In that case different components correspond to different frequencies. In the present case different components correspond to different wavelet coefficient magnitudes, being equivalent to spectral densities. The time scale spectrum is a forward wavelet transform of each row (wavelet coefficient magnitude) in the ampligram. The time scale spectrum reveals individual signal components and indicates the statistical properties of each component: deterministic or stochastic. Next step is to distinguish between different sources of infrasound on-line. This will require signal classification after detection is made. The implementation of wavelet – neural network in hardware may be a first choice. In this work the Independent Component Analysis is presented to improve the quality of the infrasonic signals by removing background noise before the hardware classification. The implementation of the discrete wavelet transform in a Field Programmable Gate Array (FPGA) is also included in this thesis using Xilinx System Generator and Simulink software. A study of using infrasound recordings together with a miniature 3-axis fluxgate magnetometer to find meteorites as soon as possible after hitting the earth is also presented in this work.

There are many reasons for using infrasound, i.e. low frequency sound, to monitor various events. Inherent features like its long-distance propagation and the use of simple, ground based equipment in very flexible system are some. The disadvantage is that it is a slow system due to the speed of sound. In this paper we try to show that there are several other advantages if one can extract all the features of the signal. In this way it is hoped that we can get a fingerprint of the event that caused the infrasound. Rayleigh waves and sound from epicentre may be obtained for earthquakes, pressure pulses and electrojets from aurora, core radius and funnel shape from tornados, etc. All these possibilities are suggestive for further R&D of the infrasound detection systems

A new method for extracting features from photographic images has been developed. The input image is through a pulse coupled neural network transformed to a set of signatures, well suited for classification by unsupervised neural networks. A strategy using multiple self-organizing feature maps in a hierarchical manner is developed. With this approach, using a certain degree of supervision, an acceptable classification is obtained when applied to test images. The method is applied to license plate recognition.

Sonar detection and classification of sunken wrecks and other objects is of keen interest to many. This paper describes the use of neural networks (NN) for locating, classifying and determining the alignment of objects on a lakebed in Sweden. A complex program for data preprocessing and visualization was developed. Part of this program, The Sonar Viewer, facilitates training and testing of the NN using (1) the MATLAB Neural Networks Toolbox for multilayer perceptrons with backpropagation (BP) and (2) the neural network O-Algorithm (OA) developed by Age Eide and Thomas Lindblad. Comparison of the performance of the two neural networks approaches indicates that, for this data BP generalizes better than OA, but use of OA eliminates the need for training on non-target (lake bed) images. The OA algorithm does not work well with the smaller ships. Increasing the resolution to counteract this problem would slow down processing and require interpolation to suggest data values between the actual sonar measurements. In general, good results were obtained for recognizing large wrecks and determining their alignment. The programs developed a useful tool for further study of sonar signals in many environments. Recent developments in pulse coupled neural networks techniques provide an opportunity to extend the use in real-world applications where experimental data is difficult, expensive or time consuming to obtain.

In this paper we discuss the use of massively parallel hardware for data mining. An introduction to the concept is followed by some benchmark results.

The approach of using a conventional neural network as well as one utilizing the O-algorithm are compared in an application of particle tracking. In the latter case we control the confidence level of the results, which is a major advantage when the network is implemented in hardware.

We briefly review the use of the wavelet and wavelet packet transforms. We describe their application to signal processing as devices for feature extraction and reduction of data for neural networks, in particular their implementation in hardware for signal identification.

An implementation of the new IBM Zero Instruction Set Computer (ZISC036) on a PC/ISA-bus card is reported. This circuit has 36 processing elements of a type similar to that of Radial Basis Function or RBF-like neurons. It is a highly parallel and cascadeable building block with on-chip learning capability, and is well suited for pattern recognition, signal processing, etc. A card with two ZISC036 was built and tested with a noisy character recognition "benchmark". Some future implementations and ideas are presented.

The problem of facial recognition from grey-scale video images is approached using a two- stage neural network implemented in software. The first net finds the eyes of a person and the second neural network uses an images of the area around the eyes to identify the person. In a second approach the first network is implemented in hardware using the IBM ZISC036 RBF- chip to increase processing speed. Other implementations in hardware are also discussed, and includes preprocessing using wavelet (packet) transforms.

Implementation of the new IBM Zero Instruction Set Computer (ZISC036) on a PC/ISA-bus card as well as on a VME-card is reported. The ZISC circuit has 36 processing elements of a type similar to that of Radial Basis Function (RBF) neurons, It is a highly parallel and cascadable building block with on-chip learning capability and is well suited for pattern recognition, signal processing, etc. Results of a test on identification of simulated Higgs events are given.

This paper describes the implementation of an analog neural network chip (Intel 80170NX ETANN) to filter a multi-signal read-out from flash-ADCs used in a liquid argon time projection chamber (ICARUS). The training and subsequent testing of the network is discussed in some detail. A network consisting of multiple sub-networks is proposed to perform signal filtering, determination of peak position and track finding.

This paper describes the implementation of an analog neural network chip (Intel 80170NX) to the pad readout of a Cherenkov ring imaging detector system. A similar system has previously been tested in software in order to separate proton and pion events. The architecture, training and testing in analog hardware is described.

This paper demonstrates the implementation in hardware of an electrically trainable analog neural network (ETANN) for finding the position and width (FWHM) of an ion-beam hitting a strip-detector. This is accomplished using a single ETANN chip with 32 neurons in one hidden layer. The network finds the maximum and the FWHM, with an error of 0.1 and 0.2, respectively, of the 16 wire input pitch. An extension of this linear peak-fitting problem to include finding the height is presented. Extensions to larger nets with 64 and 128 inputs are presented as multi-chip solutions. A track-finding problem using several chips is briefly discussed.

A neural network, yielding full control of the results of the uncertainties in the data set, is developed. The network may be used as a tool in order to calculate physical parameters, as well as a “traditional” neural network for pattern recognition.

The performance of a back-propagation neural network for particle identification using a RICH detector has been studied. When trained on 8000 simulated events in 14 iterations using a general back-propagation algorithm it correctly identifies 86% of the events out of a sample of 1000 experimentally measured pion and proton events at 3.5 GeV/c beam momentum. The identification efficiency is 70%. This is compatible with what is obtained by conventional, but mathematically much more complicated, identification algorithms.