method is at an early stage of investigation, and development in many directions is possible. Challenging open problems include the relationship between the model types and their discriminating powers, convergence of classiication performance under approximate uniformity, and alternative methods for model combination. Moreover , the study of the method reveals some fundamental problems in pattern recognition that are important to many classiication methodologies. These include the meaning of representativeness of a training set, measures of mutual dependencies among features, characterizations of the complexity of a given recognition problem, as well as analytical studies of classiier performance versus computational complexity. Acknowledgements The authors would like to thank Evelyn Kleinberg and Hing-Kai Hung for many useful discussions, and Jonathan Hull for assistance in obtaining the NIST database. 9 outperformed both 1NN and 3NN in some of tests with 250 models, and more with 750 models. Using the results of stochastic modeling on set 2, a logistic regression model was estimated for the combination of the rankings using three feature sets. The estimated coeecients for the rankings using pseudo-gray, binary, and convolution response data with 250 models are 2.3019, 1.3813, and 1.8869, and with 750 models are 2.4079, 1.4571, and 1.9707 respectively. It can be seen from Table 3 that, the classiiers derived using diierent feature sets are suuciently independent, so that further improvement on performance is possible by their combination. 6 Conclusions We have presented a method for automatic classiier construction for an arbitrary pattern recognition problem. The method is unique in that the power of the classiier improves as training continues, and, modulo the quality of a given training set, there is little tradeoo between accuracy on the training set and the projectability to an unseen test set. Experimental results show that the method is eeective on handwritten digit recognition using real-world images, and it achieves a level of accuracy that is close to expensive nearest-neighbor matching procedures. models took 1192 seconds on a SUN SPARC 2, whereas a run of 1NN with the same data took 5635 seconds on the same machine. A run with SM using 750 models took 3609 seconds. 100 Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set 2) Training (set 1) Testing (set …

We describe a system which can recognize digits and uppercase letters handprinted on a touch terminal. A character is input as a sequence of [x(t), y(t)] coordinates, subjected to very simple preprocessing, and then classified by a trainable neural network. The classifier is analogous to “time delay neural networks” previously applied to speech recognition. The network was trained on a set of 12,000 digits and uppercase letters, from approximately 250 different writers, and tested on 2500 such characters from other writers. Classification accuracy exceeded 96% on the test examples.

Through a series of experiments in optical character recognition, an understanding is beginning to emerge of the general nature of the hardware required. Rather than the fully-connected layered neural nets conceived by most hardware researchers, many machine perception tasks require local connectivity and repeated weight patterns between layers to support computing of convolutions. No current-day hardware is available to evaluate in parallel all the connections in a character recognition system. Fortunately, the repetitive nature of the convolution operation makes time-division multiplexing of the hardware possible and even efficient. To avoid I/O bottlenecks, the hardware must contain substantial input data buffers and shift registers. I/O requirements are further relaxed if several layers of the net are processed in a pipelined fashion without recourse to external storage. This paper will discuss hardware architectures for character recognition and will outline choices for possible circuits. An advanced (and working) reconfigurable neural-net chip, that mixes analog and digital processing, will be described.

It is shown that a neural net can perform handwritten digit recognition with state-of-the-art accuracy. The solution required automatic learning and generalization from thousands of training examples and also required designing into the system considerable knowledge about the task-neither engineering nor learning from examples alone would have sufficed. The resulting network is well suited for implementation on workstations or PCs and can take advantage of digital signal processors (DSPs) or custom VLSI

Hardware architectures for character recognition are discussed, and choices for possible circuits are outlined. An advanced (and working) reconfigurable neural-net chip that mixes analog and digital processing is described. It is found that different approaches to image recognition often lead to neural-net architectures that have limited connectivity and repeated use of the same set of weights. This architecture is ideal for time-multiplexing (a combined parallel-series processing) on hardware systems that would be too small to evaluate the entire network in parallel. To make this process efficient, a chip needs to have shift registers to format the input data and additional registers to store intermediate results. Within this framework, it is possible to design chips that have broad utility, large connection capacity, and high speed. This was demonstrated by a new chip with 32000 reconfigurable connections

The ability of learning networks to generalize can be greatly enhanced by providing constraints from the task domain. This paper demonstrates how such constraints can be integrated into a backpropagation network through the architecture of the network. This approach has been successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service. A single network learns the entire recognition operation, going from the normalized image of the character to the final classification.

Two novel methods for achieving handwritten digit recognition are described. The first method is based on a neural network chip that performs line thinning and feature extraction using local template matching. The second method is implemented on a digital signal processor and makes extensive use of constrained automatic learning. Experimental results obtained using isolated handwritten digits taken from postal zip codes, a rather difficult data set, are reported and discussed.< >

MOS charge storage has been demonstrated as an effective method to store the weights in VLSI implementations of neural network models by several workers. However, to achieve the full power of a VLSI implementation of an adaptive algorithm, the learning operation must built into the circuit. We have fabricated and tested a circuit ideal for this purpose by connecting a pair of capacitors with a CCD like structure, allowing for variable size weight changes as well as a weight decay operation. A 2.5µ CMOS version achieves better than 10 bits of dynamic range in a 140µ × 350µ area. A 1.25µ chip based upon the same cell has 1104 weights on a 3.5mm × 6.0mm die and is capable of peak learning rates of at least 2 × 109 weight changes per second.

. This paper describes a 1MHz CMOS implementation of neural networkused for acoustical attention. Signals are coded into binary spikes modelling the biology.A uniform model of pulse propagating cell is used in different performing stages. Thesignal processing is distributed to neurons and synapses realized as analog circuits.I. IntroductionHandling the signals in sensoric driven systems accounts for separation of information fromdata overcrowded environment. The acoustical signal...

Early results from exploring alternative computer architectures based on hints from neurobiology suggest that networks of highly-interconnected, simple, low-precision processors can give tools for tackling problems that have been hard or impossible to do on standard computers. The authors describe an electronic neural model and show how this model is readily adapted for use in pattern-recognition tasks. They also describe a chip, implementing this model, that is used for handwritten digit recognition.< >

We have studied the dynamics of a totally interconnected network of nonlinear amplifiers by building model electronic circuits using dense arrays of resistors and discrete amplifiers. Such models have been discussed recently in the context of spin glasses and neural networks. Even without optimization for speed, these circuits easily reproduce and extend the results of computer simulations in considerably less time.

A high-density matrix of α-Si resistors was made to demonstrate a new type of parallel-processing associative memory consisting of an interconnected array of analog amplifiers. The 22 × 22 resistor matrix was made using a technology compatible with conventional VLSI processing. This demonstration circuit can recall up to four 22- bit memories in 1 to 10 µs while correcting errors in the input word of at least 5 bits. This function is difficult to perform efficiently in conventional digital hardware and is the basis for solving a variety of pattern-recognition problems including vision and speech.

We designed an Electronic Neural Network (ENN) memory with 256 neurons on a single chip using a combination of analog and digital VLSI technology plus a custom microfabrication process. Amplifiers with inverting and noninverting outputs are used for the neurons to make inhibitory and excitatory connections. The connections between the individual neurons are provided by amorphous‐silicon resistors which are placed on the CMOS chip in the last fabrication step. This technique allows a very dense packing of the neurons. Electron‐beam direct‐writing is used to pattern the resistors making it easy to change the information stored in the network from one chip to the next.