One of the driving forces behind the development of new, highly parallel architectures is the need for embedded high-performance computing. The development of advanced applications on such architec ...

This paper presents a real-time spectral classification system based on the PGP spectrograph and a smart image sensor. The PGP is a spectrograph which extracts the spectral information from a scene and projects the information on an image sensor, which is a method often referred to as Imaging Spectroscopy. The classification is based on linear models and categorizes a number of pixels along a line. Previous systems adopting this method have used standard sensors, which often resulted in poor performance. The new system, however, is based on a patented near-sensor classification method, which exploits analogue features on the smart image sensor. The method reduces the enormous amount of data to be processed at an early stage, thus making true real-time spectral classification possible. The system has been evaluated on hardwood parquet boards showing very good results. The color defects considered in the experiments were blue stain, white sapwood, yellow decay and red decay. In addition to these four defect classes, a reference class was used to indicate correct surface color. The system calculates a statistical measure for each parquet block, giving the pixel defect percentage. The patented method makes it possible to run at very high speeds with a high spectral discrimination ability. Using a powerful illuminator, the system can run with a line frequency exceeding 2000 line/s. This opens up the possibility to maintain high production speed and still measure with good resolution.

Analog Sensor Processing Using Exposure Control (ASPEC) is a new concept for high speed image processing. By using an addressable image array with integrating output amplifiers, signal processing can be performed directly on the sensor. The major gain in using ASPEC techniques is that the operations are fast, the approach can be implemented using existing hardware, and that the processing is executed in parallel on the sensor array. Furthermore, the data reduction is carried out early in the signal processing chain. In this paper we present a novel programmable camera architecture based on the CMS CMOS integrating addressable image sensor which is well suited for ASPEC applications.

In the original near-sensor image processing (NSIP) concept, the intensity level for an individual sensor element is mapped onto the time axis so that the time to reach the threshold is inversely proportional to the intensity. In many cases this is favorable, in some cases there is a need for another mapping between time and intensity. We show that such mappings can be achieved by either inserting delays or by varying the threshold voltage. The result is that to achieve a linear mapping it is much more favorable to use a variable threshold since the processing power of the sensor is then better utilized. An interesting result, when it comes to adaptive mapping, is that the traditionally time consuming operation histogram equalization can be accomplished almost for free in the NSIP concept. We also show that other histogram-based, i.e., data-dependent, mapping can be easily implemented.

This chapter discusses the concept of near sensor image processing (NSIP). The photoelectric effect is the basic physical principle used in all light sensors. Under certain conditions, it is possible for a photon that hits a material to leave its energy to an electron. This causes a charge to build up in the sensor material. In a charge-coupled device (CCD), photo element used in many TV cameras, the charge is collected in a well created by applying suitable voltages to a system of electrodes. The sensors are interrogated at even or uneven time intervals in the main NSIP loop during ongoing exposure. In NSIP, the exposure time depends on the light condition as it is exposed until at least one position accumulates enough light energy. Thus, tolerance to various light conditions has been traded for data-dependent non-fixed exposure time. To obtain such adaptation in traditional systems, an outer control loop is needed to modify the exposure time and maintain a reasonable signal-to-noise (S/N) ratio. All this comes free with NSIP.

In this paper we introduce the concept of analog sensor processing using exposure control, ASPEC. We give a number of our previous image processing applications which have utilized programmable exposure control. We also show a new application which is center of gravity calculation. This is demonstrated for one and for several objects.

We discuss a device for real time compensation of image quality deterioration induced by atmospheric turbulence. We device will permit ground based observations with very high image resolution. We propose can instrument with two channels. One is an ordinary image detection channel, while the other uses a Hartmann-Shack wavefront detector to measure image degradation. This information is obtained in the form of a set of lenslet focus shifts, each corresponding to the local tilt of the wavefront. Through modelling, the entire wavefront is reconstructed. Consequently, we can estimate the optical transfer function and its corresponding point spread function. Through convolution techniques, the distorted image can subsequently be restored. Thus, image correction is performed in software, eliminating the need for expensive live optics designs.Due to the nature of atmospheric turbulence, detection and correction have to be made with 50-100 frames per second. This implies a need for very high computing capacity. A study of the mathematical operations involved has been made with special emphasis on implementation in the hardware architecture known as Radar Video image Processor (RVIP). This hardware utilises a high degree of parallelism. Results available show that RVIP together with complementary units provide the necessary high-speed computing capacity.The detection system in both channels must meet very high demands. We mention high quantum efficiency, fast readout at lour noise levels and a wide spectral range. A preliminary investigation evaluates suitable detectors. ICCDs are so far the most promising candidates.

In this paper we present a system for high speed pixelwise spectral classification. The system is based on the line imaging PGP (prism-grating-prism) spectrograph combined with the smart image sensor MAPP2200. The classification is implemented using a near-sensor approach where linear discriminant functions are calculated using exposure time modulation and analog summation of pixel data. After A/D-conversion the sums are compared and classified pixels are output from the sensor chip. The theoretical maximum classified pixel rate of the system is around 1 MHz depending on number of classes, etc. In most practical applications however, the limit will be set by the available amount of light.

The near-sensor image processing concept, which has earlier been theoretically described, is here verified with an implementation. The NSIP describes a method to implement a two-dimensional (2-D) image sensor array with processing capacity in every pixel. Traditionally, there is a contradiction between high spatial resolution and complex processor elements, In the NSIP concept we have a nondestructive photodiode readout and we can thereby process binary images without loosing gray-scale information. The global image processing is handled by an asynchronous Global Logical Unit. These two features makes it possible to have efficient image processing in a small processor element. Electrical problems such as power consumption and fixed pattern noise are solved. All design is aimed at a 128/spl times/128 pixels NSIP in a 0.8 /spl mu/m double-metal single-poly CMOS process. We have fabricated and measured a 32/spl times/32 pixels NSIP. We also give examples of image processing tasks such as gradient and maximum detection, histogram equalization, and thresholding with hysteresis. In the NSIP concept automatic light adaptivity within a 160 dB range is possible.

Today, it is accepted to say that the more bits you have got in your micro processor the better performance you will have. This is probably true if only the performance is concerned. However, if the chip size of the processor is taken into account this might not be the case. In massively parallel architecture, chip area is an important figure. This is especially true for air-borne and, to certain extend, industrial real-time applications. In this paper we study the impact of multi bit processors on the linear SIMD array called RVIP which is an architecture used for real-time radar signal processing. In RVIP, each processing element is bit serial. The results show that the gain in using multi bit processors is very little and that the optimal bit number probably is one or two. We believe that the results from this study can be transferred to other similar systems.

Near-Sensor Image Processing, NSIP, is a concept where the temporal behavior of the photo diode is used to perform image processing. It has been shown that many conventional image processing operations like convolution and grey scale morphology can easily be implemented in NSIP. In this paper we will describe the basis of NSIP and how the sensor/processor architecture is used to perform local as well as global operation. An implementation of an NSIP chip will also be described. Finally, we will show a number of algorithms and applications which have been implemented in our NSIP camera system.

A range image is an image where each pixel represents a measurement of the distance from the camera to the object. Typical applications for range imaging are inspection and dimension measurement in industrial processes, e.g. in the forest products industry. Range image acquisition can be made in many different ways. In this paper we use an active triangulation method where a sheet-of-light illuminates the scene. The sensor level signal processing task is to extract the light impact position in each sensor column. Two novel algorithms implemented on the commercially available smart image sensor MAPP2200 are presented. Both algorithms give 256 pixel width resolution at a line frequency of 15 - 20 kHz, corresponding to range pixel rates of 4 - 5 MHz, and range resolution varying from 8 up to 13 bits in special cases. This is considerably faster than other proposed methods. One of the methods also gives intensity data concurrent and in perfect registration with the range data and the other has an error detection capability to detect multiple peaks on the sensor.

We show that the linear SIMD architecture FVIP is capable of performing a typical MPD radar signal processing including FFT and the resolving algorithm. A conventional DSP system clocked at 50 MHz would require approximately 200 DSPs to have the same performance as our FVIP system. We have estimated the size and power consumption of our FVIP system to be 3 dm3 and 200 W. With these MPD studies together with previous LPD studies we are confident that a “VIP-type” architecture can handle all typical pulse Doppler radar waveforms currently used in airborne radar

We present a single chip circuit solution to a concept called Near-Sensor Image Processing (NISP), which includes image sensing, image processing and feature extraction. We give solutions to the three main implementation problems. A small photodiode read-out unit, which is locally compensated for process variations, a low power processor element and an instruction line driver, suitable for massively parallel processors are described. A 16/spl times/16 elements prototype has been built. However most of the results come from simulations of an improved 128/spl times/128 matrix.

A range image is an image where each pixel represents a measurement of the distance from the camera to the object. Typical applications for range imaging are inspection and dimension measurement in industrial processes, e.g. in the forest products industry. Range image acquisition can be made in many different ways. In this paper we use an active triangulation method where a sheet-of-light illuminates the scene. The sensor level signal processing task is to extract the light impact position in each sensor column. Two novel algorithms implemented on the commercially available smart image sensor MAPP2200 are presented. Both algorithms give 256 pixel width resolution at a line frequency of 15 - 20 kHz, corresponding to range pixel rates of 4 - 5 MHz, and range resolution varying from 8 up to 13 bits in special cases. This is considerably faster than other proposed methods. One of the methods also gives intensity data concurrent and in perfect registration with the range data and the other has an error detection capability to detect multiple peaks on the sensor.

The paper introduces the concept of near-sensor image processing. By this, the authors mean techniques in which the physical properties of the image sensor itself is utilized to do part of the signal processing task. It is shown that the analog-temporal behavior of photodiodes combined with thresholding amplifiers can be used favorably to do certain low-level image processing tasks including median filtering and convolution. The given examples also show how adaptivity to different light levels can be achieved in a natural way. To extract features from the image, such as moments and shape factors, the authors introduce a simple measurement function.

Image processing is used in the forest products industry to detect various defects on wood surfaces. Normally, several different sensors are needed, which makes the systems complicated and expensive. In this paper we present a highly integrated sensor system for wood defect detection based on a single MAPP2200 smart sensor. Five different measuring principles are simultaneously utilized to detect surface grayscale, surface roughness, deviating grain direction, 3D-profile and surface density. Using a pixel resolution of 1×0.5 mm, the sensor permits scanning of boards at the speed of several meters per second

The authors describe sensor/processor design which is based on the near-sensor image processing concept. With this concept it is possible to integrate a large number of sensor/processing elements in the sensor array. This is possible since each sensor/processing element can be reduced to a minimal number of transistors and still perform a number of image processing operations, which is shown here. The authors also give an overall description of the architecture as well as some descriptions of the more global operations which are required in this concept

The authors present the radar video image processor (RIVP) architecture and its performance. RIVP is an SIMD (single instruction multiple data) linear processor array with 128 bit-serial processing elements (PEs) integrated on one chip. Each PE incorporates a serial-parallel multiplier and a bit-serial ALU with a 32-b accumulator register. The special multiplier-ALU-accumulator design makes convolutions, which is a basic image processing operation, very effective. Large IO and inter processor communication bandwidth is obtained by the use of four 32-b double-ported IO registers and a 16-b internal bidirectional shift register in each PE. Four RIVP chip and an internal micro controller are packaged n a 2" × 2" multi chip module (MCM) and is a stand-alone 512 PE SIMD processor array. A 512 PE MCM module is suited for real-time video-input processing of 512 × 512 images. Each RIVP MCM is capable of 1 Giga multiply-accumulations per second on 10 by 16 bit words at 50 MHz clock frequency. In some applications several RIVP MCM modules are needed. For instance, in the pulse Doppler radar example given eight modules are used to obtain a total of 4096 processing elements in series