Image enhancement has been proposed as a potential aid for the visually impaired (1). Digital image enhancement may be used to improve visibility of video images and printed pictures. We demonstrated its value with optically simulated cataracts (1). Use of enhancement filters based on the patient's contrast sensitivity function was shown to modestly reduce magnification demands for reading of patients with central scotoma (2). However, the value of image enhancement in recognizing and perceiving gray scale images has not been demonstrated. If improvement is noted, these techniques may be implemented in various ways. Television programs can be enhanced either at a central broadcasting location or the patient's receiver, or a portable system may be used with a head-mounted, closed-circuit TV system to aid mobility (3). Difficulty with face recognition is a frequent early complaint of many patients with macular disease (4). Faces can be recognized both when low pass filtered to a large degree (5) or high pass filtered (6,7). Since most image enhancement techniques are in the form of high pass filtering, this study evaluated improvement in face recognition with the adaptive en~ hancement algorithm (1). To evaluate actual face recognition rather than the ability to discriminate among test faces, we tested patient ability to recognize celebrities.

Lightness constancy in complex scenes requires that the visual system take account of information concerning variations of illumination falling on visible surfaces. Three experiments on the perception of lightness for three-dimensional (3-D) curved objects show that human observers are better able to perform this accounting for certain scenes than for others. The experiments investigate the effect of object curvature, illumination direction, and object shape on lightness perception. Lightness constancy was quite good when a rich local gray-level context was provided. Deviations occurred when both illumination and reflectance changed along the surface of the objects. Does the perception of a 3-D surface and illuminant layout help calibrate lightness judgments? Our results showed a small but consistent improvement between lightness matches on ellipsoid shapes, relative to flat rectangle shapes, under illumination conditions that produce similar image gradients. Illumination change over 3-D forms is therefore taken into account in lightness perception.

Lightness constancy requires that a surface retain its Lightness not only when the illumination is changed but also when the surface is moved from one background to another. Occlusion of one surface by another frequently results in a retinal juxtaposition of patches under different illuminations. At such edges, retinal luminance ratios can be much higher than in scenes with a single illumination. We demonstrate that such retinal adjacencies can produce failures of lightness constancy. We argue that they are responsible for departures from perfect lightness constancy in two prior experiments that examined the effects of depth relations on lightness constancy.

Changes of annulus luminance in traditional disk-and-annulus patterns can be perceived to be either reflectance or illuminance changes. In the present experiments, we examined the effect of varying annulus reflectance. In Experiment 1, we placed test and standard patch-and-surround patterns in identical Mondrian patchworks. Only the luminance of the test surround changed from trial to triaL., appearing as reflectance variation under constant illumination. Lightness matches were identical to brightness matches, as expected. In Experiment 2, we used only the patch and surround (no Mondrian). Instructions said that the illumination would change from trial to trial. Lightness and brightness-contrast data were identical; illumination gradients were indistinguishable from reflectance gradients. In Experiment 3, the patterns were the same, but the instructions said that the shade of gray of the test surround would change from trial to trial. Lightness matches were identical to brightness matches, again confirming the ambiguity of disk-and-annulus patterns.

At mesopic mean luminances, a fixed luminance contrast produces less brightness contrast than it does at photopic luminances. This suggests that lightnesses of surfaces might also be altered at low luminances. I measured lightness, brightness, and brightness contrast in CRT simulations of achromatic paper patchworks. The illuminance of the standard pattern was fixed, producing 0.12,1.2, or 12 cd/m2. The illuminance on the test pattern was varied in a lightness constancy paradigm. Constant brightness contrast required more luminance contrast at lower mean luminances. Failures of lightness constancy occurred at the lowest mean luminances, but they were minor in comparison with the loss of brightness contrast in the same pattern. These results have implications for imaging applications. Often, image content falls in both the photopic and the mesopic ranges. Our results indicate that brightness contrast may decrease substantially in low-luminance regions without large changes of surface lightness.

Changes of annulus luminance in traditional disk-and-annulus patterns are perceptually ambiguous; they could be either reflectance or illuminance changes. In more complicated patterns, apparent reflectances are less ambiguous, letting us place test and standard patchesjnpxsurrounds perceived to be different grays. Our subjects matched the apparent amounts of light coming from the patches (brightnesses), their apparent reflectances (lightnesses), or the brightness differences between the patches and their surrounds (brightness contrasts). The three criteria produced quantitatively different results. Brightness contrasts matched when the patch/surround luminance ratio of the test was approximately equal to that of the standard. Lightness matches were illumination invariant but were not exact reflectance matches; the different surrounda of test and standard produced a small illumination-invariant error. This constant error was negligible for increments, but, for decrements, it was approximately 1.5 Munsell value steps. Brightness matches covaried substantially with illuminance.

Models of the spatial response of human vision are important for applied work, but the available contrast sensitivity function (CSF) data vary widely due to the diverse spatiotemporal stimuli used over the years. To assist selection, this paper: (1) reports measurements of the effects on the CSF of varying the spatial and temporal windows of grating patches; (2) demonstrates that the widely discrepant CSFs from previous studies can be accounted for by using these results; and (3) discusses simple criteria for choosing CSFs for practical applications. CSFs were measured for several combinations of spatial and temporal waveforms, using the same subjects under otherwise identical conditions. The CSF was measured over the range of 0.5-10 c/deg using Gabor-type patches of 1.0-, 0.5-, 0.25-, and 0.125-octave spatial bandwidths using both abrupt and gradual temporal presentations. The results were compared with the CSF obtained with a fixed aperture (4 deg x 4 deg) grating pattern. Increasing the number of cycles resulted in increased sensitivity at intermediate frequencies, changing the CSF to a narrower bandpass shape. For each patch bandwidth, the gradual presentation CSF had a narrower spatial pass band than with the abrupt presentation. The relevance of the large differences in the CSFs obtained with different stimuli to our understanding of visual performance is discussed.

We have investigated the extent of color constancy in complex (Mondrian) and simple (center-surround) displays. Our critical finding is that the extent of lightness and color constancy in such displays depend heavily on the instructions given to subjects. We also find, not surprisingly, that adaptation has a powerful effect on color constancy. Stimulus conditions, particularly spatial complexity, which have been thought critical in lightness and color constancy, however have little or no effect. This is true for judgments made with successive as well as with simultaneous displays.

A number of researchers have reported that patterns have lower apparent contrast when surrounded by high-physical-contrast patterns with similar spatial frequency properties than when surrounded by lower-contrast patterns. This contrast contrast has been interpreted as revealing lateral interactions among neural gain signals. We report several cases where the phenomena seem to be closely connected to figural properties not captured by the neural interaction account. The test pattern’s apparent contrast can also be described in terms of the maximum and minimum brightnesses of it elements. We find that uniform bright patches are brighter and dim patches are dimmer on low contrast surrounds that on high contrasts; this is hard to reconcile with patternspecific neural interactions.

The influence of sensory processes on perception of pictures has long interested graphics scientists and engineers. Adaptation, illumination, and surround variables affect chromatic and achromatic apparent contrast and other aspects of color appearance. My recent experiments on apparent surface colors in complex patterns have led to a model of surface appearance in which early visual processes (e.g., adaptation, contrast) are only the first stage. Their role in surface perception is to relatively accurately encode the physical contrasts in the retinal image. Higherorder processes then compute surface properties from these contrast signals. It is, however, well known from neurophysiological and psychophysical measurements that early processes only approximate ideal encoding of image contrasts. Constant response amplitudes require larger luminance contrasts at low mean luminances. I have recently measured local apparent contrasts, lightnesses (apparent reflectances), and brighmesses (apparent luminances) in complex patterns at a variety of luminances that occur frequently in modern display devices. Apparent contrast decreased at low luminances, but this did not distort apparent reflectances (as one might expect from a number of recent lightness models). These results have several interesting implications for imaging applications.

The response of visual filters has been used to simulate vision and to design image enhancement techniques for the visually impaired. For the simulations and enhancements to be useful, they must be based on appropriate measurements of the filters being studied. Most previous measurements of spatial contrast sensitivity functions (CSFs) have been based on stimuli that give maximal contrast sensitivity. We measured CSF using stimuli that are better suited for analyzing perception of images. Use of the different types of CSF yields notably different simulations.

Three experiments were conducted in an attempt to replicate and clarify Gilchrist's (1977, 1980) experiments on the effects of depth information on judgments of achromatic surface color. Gilchrist found that coplanarity, and not retinal adjacency, was the dominant factor in determining achromatic color matches. Because such matches can be made on the basis of either brightness or lightness, we obtained judgments of both qualities. Stereopsis was added to enhance the perceived depth effect of Gilchrist's display, which was otherwise simulated closely on a high-resolution CRT. The results for lightness followed the same pattern as those of Gilchrist, but were smaller in magnitude. This discrepancy may reflect reduced extraneous lighting effects in our displays. Our results therefore agree with related studies in suggesting that lightness matches are based on relationships among coplanar surfaces. Brightness matches, however, were not influenced by perceived depth.

We simultaneously perceive at each visible surface point (at least) a surface color (with several dimensions), an illuminant color, surface orientation, and surface specularity. Perceived surface and lighting variables are a multidimensional function of the past several minutes of chromatic exposure and the current retinal images. Models with outputs that are single-valued functions of image spatial and temporal derivatives (e.g., edge ratio models of Wallach, Land, Horn, Arend) describe sensory processes providing important relational information. However, they are easily shown to be incomplete as models of surface color constancy. Traditional color appearance models are similarly incomplete. This multidimensionality of surface perception was known to early theorists and has been further elaborated by recent human perception and machine vision theorists. The human vision data needed for refinement of computational color constancy models are currently in short supply for both theoretical and methodological reasons. We have recently measured human color constancy using both spatially simple and complex patterns. Our procedure allowed separate measurement of local sensory color (hue, saturation, and brightness) and apparent surface colors (surface chromatic color and lightness). Even in the simplified scenes of our experiments there is a complicated relationship between the sensory color at an image point and the apparent color of the surface perceived on the corresponding sight line.

The influence of surrounding colors on the appearance of a focal region is one of the oldest topics of vision research, and a wide variety of experimental paradigms has produced large quantities of data. From the beginning investigators were interested in the role of contrast in everyday color appearance. For the sake of close experimental control, most of that work involved very simple stimulus patterns. As a consequence relatively little systematic information has been available concerning the role of color contrast in patterns more closely approximating the conditions of everyday scenes. Most natural scenes involve many different chromaticies varying over the relatively small range produced by reflective surfaces under nearly white illumination. My colleagues and I have recently performed a series of experiments on contrast in complex patterns1,2.

Gradient illusions require that models of suprathreshold appearance include a spatial integration that fills areas between edges. We describe a structural problem inherent in such models; for many scenes there are inconsistencies (nonzero curl) in thresholded derivatives that prevent simple spatial integration. Our experiments show that the human visual system does encounter curl problems and that it uses two different types of perceptual solution: field segmentation and lightness-gradient manipulation. The latter occurs under conditions where field segmentation is impossible. At least two such conditions can occur: failure to form a segmenting contour and topological problems in potential segmenting contours.

Gilchrist’s 1 subjects saw a test paper as adjacent to either a paper in a distant brightly lit room or a second paper in a closer dimly lit room (the apparent depth of the test being determined by interposition cues). The test and two other papers were always retinally adjacent. Gilchrist reported that grey-scale matches to the test were strongly influenced by the apparently coplanar paper; the other noncoplanar paper had virtually no effect. We wondered what Gilchrist's subjects had matched: brightness or lightness. We accurately simulated the rooms (excluding shadows) on a high-quality monitor. Luminance ratios (unlike Gilchrist's) under each illuminant were natural, not exceeding 30:1. Lightness matches showed Gilchrist's effect but to a lesser degree whether depth was cued by interposition alone or by stereo in addition. Brightness matches showed no effect of perceived depth. Results were replicated with a method of adjustment.

Image enhancement potentially provides low-vision patients better use and enjoyment of printed photographs and TV. A variety of equipment and techniques exist for enhancements, but there is no systematic method of designing appropriate enhancement for low-vision patients. Most image enhancement techniques may be classified as either contrast modification or spatial-filtering techniques. Although global contrast modification may be beneficial for the visually impaired, there is no way to optimize the selection of the modification paradigm or the parameters to be used. We propose a linear model of impaired vision in which the degradation transfer function resulting from the impairment is described by the ratio of the spatial contrast sensitivity function (CSF) of the low-vision eye to the CSF of a normal eye. The inverse of the degradation transfer function may be used as a spatial filter to preemphasize the image before it is displayed to the low-vision patient. Based on the CSF, the appropriate filter may be tailored for each patient or group of patients. Both threshold and suprathreshold CSF data may be required for the filter design. Results with optically simulated cataracts indicate that this approach may be feasible for patients with high spatial frequency loss.

Recent data require nonlinear spatial summation processes in models for detection of spatially periodic and aperiodic patterns. Contrast matching experiments with suprathreshold grating and spot patterns suggest that nonlinear models may be required for suprathreshold contrast data, but differences between the psychophysical tasks used with periodic and aperiodic patterns make comparison difficult. A series of experiments are reported in which subjects matched local brightnesses and brightness differences within single cycles of grating patterns. This task closely resembles the matching task in classical contrast experiments with aperiodic stimuli, allowing comparison of the data from the two types of experiments. Brightnesses within a 5-cycle/degree (c/deg) sinusoidal grating were largely unaffected by addition of a large 15-c/deg modulation component in either of two phases, in spite of the resulting large change in local luminances within the pattern. As at threshold, complex models are required in order to account for apparent differences between spatial interactions within periodic and aperiodic patterns.

Contour information generated by moving retinal images has been shown by others to be the principal determinant of perceived color. The data presented here show that, for brightness, this information reflects only differences between adjacent stimulus areas. The entire distribution of difference information from contours in the visual field must be specified in order to predict the brightness at any point.