The design of a supervised neural network may be pursued in a variety of different ways. The back-propagation algorithm for the design of a multilayer perceptron (under supervision) as described in the previous chapter may be viewed as an application of an optimization method known in statistics as stochastic approximation. In this chapter we take a different approach by viewing the design of a neural network as a cumelfitting (approximation) problem in a high-dimensional space. According to this viewpoint, learning is equivalent to finding a surface in a multidimensional space that provides a best fit to the training data, with the criterion for “best fit” being measured in some statistical sense. Correspondingly, generalization is equivalent to the use of this multidimensional surface to interpolate the test data. Such a viewpoint is indeed the motivation behind the method of radial-basis functions in the sense that it draws upon research work on traditional strict interpolation in a multidimensional space. In the context of a neural network, the hidden units provide a set of “functions” that constitute an arbitrary “basis” for the input patterns (vectors) when they are expanded into the hidden-unit space; these functions are called radial-basisfunctions. Radial-basis functions were first introduced in the solution of the real multivariate interpolation problem. The early work on this subject is surveyed by Powell (1985). It is now one of the main fields of research in numerical analysis. Broomhead and Lowe (1988) were the first to exploit the use of radial-basis functions in the design of neural networks. Other major contributions to the theory, design, and application of radial-basis function networks include papers by Moody and Darken (1989), Renals (1989), and Poggio and Girosi (1990a). The paper by Poggio and Girosi emphasizes the use of regularization theory applied to this class of neural networks as a method for improved generalization to new data. The construction of a radial-basis function (RBF) network in its most basic form involves three entirely different layers. The input layer is made up of source nodes (sensory units). The second layer is a hidden layer of high enough dimension, which serves a different purpose from that in a multilayer perceptron. The output layer supplies the response of the network to the activation patterns applied to the input layer. The transformation from the input space to the hidden-unit space is nonlinear, whereas the transformation from the hidden-unit space to the output space is linear. A mathematical justification for this rationale may be traced back to an early paper by Cover (1965). In particular, we note from this paper that a pattern-classification problem cast in a high-dimensional space nonlinearly is more likely to be linearly separable than in a low-dimensional spacehence the reason for making the dimension of the hidden-unit space in an RBF network high. Through careful design, however, it is possible to reduce the dimension of the hidden-unit space, especially if the centers of the hidden units are made adaptive.

Recognition of human activity from sensor data is a research field of great potential. Giving autonomous systems the ability to identify what a human subject is doing at a given time is highly useful in many industries, particularly in health care and security monitoring. Our results, using a public domain dataset, show that the state- of-the-art decision tree ensemble algorithm XGBoost gives an accuracy of 94.6% validated on an independent test set. Previously published results using support vector machines (SVM) gave an accuracy of 90.2%. As far as we know, our result is the new state of the art for this data set. Recognition of human activity carries potential privacy concerns, which to some degree constrain the choice of sensor technology. Therefore, systems such as ours which can identify activities from simple inertial sensors, e.g. accelerators and gyroscopes are of particular interest. Data from such inertial sensors are difficult to interpret using mechanistic models; hence the field of Machine Learning is particularly interesting for this application.

The PCNN can be a very powerful front-end processor for an image recognition system. This is not surprising since the PCNN is based on the biological version of a pre-processor. The PCNN has the ability to extract edge information, texture information, and to segment the image. This type of information is extremely useful for image recognition engines. The PCNN also has the advantage of being very generic. Very few changes (if any) to the PCNN are required to operate on different types of data. This is an advantage over previous image segmentation algorithms, which generally require information about the target before they are effective.

Humans have an outstanding ability to recognise, classify and discriminate objects with extreme ease. For example, if a person was in a large classroom and was asked to find the light switch it would not take more than a second or two. Even if the light switch was located in a different place than the person expected or it was shaped differently than expected it would not be difficult to find the switch. Humans also do not need to see hundreds of exemplars in order to identify similar objects. A person needs to see only a few dogs and then he is able to recognise dogs even from species that he has not seen before. This recognition ability also holds true for animals, to a greater or lesser extent. A spider has no problem recognising a fly as even a baby spider can do that. At this level we are talking about a few hundred to a thousand processing elements or neurons. Nevertheless the biological systems seem to do their job very well.

Consider the image in the upper left corner of Fig. 7.1. This image gives a grey scale input to a PCNN and the subsequent frames show the temporal series of outputs from a PCNN. Close to perfect edge detection is obtained and there are no problems identifying the aircraft (e.g. from image number 3). Here the damaged wing tip is also easily seen.

With the advent of the cheap digital camera we have the ability to overwhelm ourselves with digital images. Thus, there is a need to be able to describe the contents of images in a condensed manner. This description must contain information about the content of the images rather than just a statistical description of the pixels. Measurements of the activity in the brain of small mammals indicate that image information is converted to small one-dimensional signals. These signals are dependent upon the shapes contained in the input stimulus. This is a drastic reduction in the amount of information used to represent the input and therefore is much easier to process.

Previous chapters considered the PCNN and ICM operating on grey scale images. This chapter considers data that has multiple channels such as colour images and multi-spectral images. In these cases the neural model is expanded in order to handle the new dimension in the input space.

In this section two digital models evolved from biological cortical models will be presented. The first is the Pulse-Coupled Neural Network (PCNN) which for many years was the standard model for many image processing applications. The PCNN is based solely on the Eckhorn model but there are many other cortical models that exist. These models all have a common mathematical foundation, but beyond the common foundation each also had unique terms. Since the goal here is to build image processing routines and not to exactly simulate the biological system a new model was constructed. This model contained the common foundation without the extra terms and is therefore viewed as the intersection of several cortical models and it is named the Intersecting Cortical Model (ICM).

Two related third-party packages named NumPy and SciPy [3] provide tools for creating arrays (vectors, matrices, and tensors) and a plethora of tools for manipulating these arrays. These tools provide very efficient codes that make programming easier and computations quick. Since neural models consider arrays of neurons these tools are essential. A third package named Python Image Library (PIL) provides tools or reading and writing image data.

Many cancers are detected at a late stage resulting in high mortality rates. Thus, it is essential to develop inexpensive and simple methods for early diagnosis. Detection of different malignancies using canine scent, as well as other technical methods, has been reported in peer-reviewed journals, indicating that this may represent a new diagnostic tool for malignancies. Aim: This study aims to test the detection of different volatile organic compound signals emitted by ovarian carcinoma and normal tissues. Materials & methods: A previously tested electronic nose is used in the pilot study to analyze human grade 3 seropapillary ovarian carcinoma samples. The recorded signals were compared with healthy human Fallopian tube specimens. A variety of algorithms were tested and confusion matrices compared. In parallel, an external validation study was performed using the same type and grade of human ovarian carcinomas with healthy myometrium (first part) and postmenopausal ovarium (second part) specimens as controls. Both sample types were obtained from individuals who did not participate in the pilot study. Results: Method sensitivity was 100% (15 of 15) in the pilot study. The first part of the validation study demonstrated that 84.8% of cancer tissues (sensitivity: 84.8%) and 88.6% of the control samples (specificity: 88.6%) were correctly classified. In the second part the JRip algorithm correctly classified 75% of cancer tissues (sensitivity: 75%) and 80% of the control ovarian tissues (specificity: 80%). Collating results gives a sensitivity of 84.4%, whereas overall specificity was 86.8%. Conclusion: Although based on a limited number of samples, our results strongly suggest that specific volatile organic compound signals emitted by ovarian carcinomas may be used for early diagnosis of the disease.

Ovarian carcinoma is one of the most deadly diseases, especially in the case of late diagnosis. This paper describes the result of a pilot study on an early detection method that could be inexpensive and simple based on data processing and machine learning algorithms in an electronic nose system. Experimental analysis using real ovarian carcinoma samples is presented in this study. The electronic nose used in this pilot test is very much the same as a nose used to detect and identify explosives. However, even if the apparatus used is the same, it is shown that the use of proper algorithms for analysis of the multi-sensor data from the electronic nose yielded surprisingly good results with more than 77% classification rate. These results are suggestive for further extensive experiments and development of the hardware as well as the software.

Ovarian cancer is one of the leading causes of death from cancer in women. The lifetime risk is around 1.5%, which makes it the second most common gynecologic malignancy (the first one being breast cancer). To have a definitive diagnose, a surgical procedure is generally required and suspicious areas (samples) will be removed and sent for microscopic and other analysis. This paper describes the result of a pilot study in which an electronic nose is used to ldquosmellrdquo the aforementioned samples, analyze the multi-sensor signals and have a close to real-time answer on the detection of cancer. Besides being fast, the detection method is inexpensive and simple. Experimental analysis using real ovarian carcinoma samples shows that the use of proper algorithms for analysis of the multi-sensor data from the electronic nose yielded surprisingly good results with more than 77% classification rate. The electronic nose used in this pilot study was originally developed to be used as a ldquobomb dogrdquo and can distinguish between e.g. TNT, Dynamex, Prillit. However, it was constructed to be a flexible multi-sensor device and the individual (16) sensors can easily be replaced/exchanged. This is suggestive for further investigations to obtain even better results with new, specific sensors. In another pilot experiment, headspace of an ovarian carcinoma sample and a control sample were analyzed using gas chromatography-mass spectrometry. Significant differences in chemical composition and compound levels were recorded, which would explain the different response obtained with the electronic nose.

Infrasound is a low frequency acoustic phenomenon that typically ranges from 0.01 to 20 Hz. The data collected from infrasound microphones are presented online by the infrasound monitoring system operating in Northern Europe, i.e., the Swedish-Finnish Infrasound Network (SFIN). Processing the continuous flow of data to extract optimal feature information is important for real-time signal classification. Performing wavelet decomposition on the real-time signals is an alternative. The purpose of this paper is to present the design and FPGA implementation of discrete wavelet transforms (DWT) for real-time infrasound data processing; our approach uses only two FIR filters, a high-pass and a low-pass filter. A compact implementation was realized with pipelining techniques and multiple use of generalized building blocks. The design was described in VHDL and the FPGA implementation and simulation were performed on the QUARTUS II platform.

Advancements in electronics provide a vital new option for implementation of low-frequency smart sensors that can perform signal processing close to the sensors and transmit the data wirelessly. These smart sensors can improve the efficiency of an automatic classification system and reduce the cost of actual infrasound microphones. The design of a digital wireless data acquisition system using a QF4512 programmable signal converter from Quickfilter Technologies, a MSP430 microcontroller from Texas Instruments and a F2M03GLA Bluetooth module from Free2move for infrasonic records is presented in this paper. The digital wireless data acquisition system has passed extensive laboratory and field tests (e.g. with man-made explosions).

Automatic Design of Algorithms through Evolution (ADATE) is a program synthesis system that creates recursive programs in a functional language with automatic invention of recursive help functions and self-adaptive optimization of numerical values. We implement a neuron in a pulse coupled neural network (PCNN) as a recursive function in the ADATE language and then use ADATE to automatically evolve better PCNN neurons for image segmentation.Our technique is generally applicable for automatic improvement of most image processing algorithms and neural computing methods. It may be used either to generally improve a given implementation or to tailor that implementation to a specific problem, which with respect to image segmentation for example can be road following for autonomous vehicles or infrared image segmentation for heat seeking missiles that are to distinguish the heat source of the target from flares. (C) 2008 Elsevier B.V. All rights reserved.

Signal processing and feature extraction are investigated using the Empirical Mode Decomposition (EMD). It is believed that this approach is well suited for non-linear and non-stationary data. With EMD any complicated set of data can be decomposed into a finite, and usually small number, of functions called Intrinsic Mode Functions (IMFs). A new discriminating system is presented here that is capable of discriminating between different seismic signals from nuclear testing sites based on the IMFs and the multi-modal data space. The advantage of this space is that multiple metrics of similarity are converted into one single Euclidean space. This space is capable of extracting similarities among several signals through a combination of multiple metrics. This is a new way of associating data. After illustrating the technique with an investigation of an audio data example (piano), we examine the characteristics of seismic signals from nuclear testing (explosions). The results presented in this paper indicate that a relatively simple discriminating system can successfully cluster and classify seismic events.