Notes in Computer Science 5646

Significant color signal transformation occurs in the primary visual cortex and neurons tuned to various direction in the color space are generated. The resulting multi-axes color representation appears to be the basic principle of color representation throughout the visual cortex. Color signal is conveyed through the ventral stream of cortical visual pathway and finally reaches to the inferior temporal (IT) cortex. Lesion studies have shown that IT cortex plays critical role in color vision. Color discrimination is accomplished by using the activities of a large number of color selective IT neurons with various properties. Both discrimination and categorization are important aspects of our color vision, and we can switch between these two modes depending on the task demand. IT cortex receives top-down signal coding the task and this signal adaptively modulates the color selective responses in IT cortex such that neural signals useful for the ongoing task is efficiently selected. 1 Neural Pathway for Color Vision Visual systems in the human and monkey brains have functional differentiation and consists of multiple parallel pathways [1]. Color information is carried by specific types of retinal cells and transmitted along specific fibers in the optic nerve [2][3]. Visual signals leaving the eye are relayed at the lateral geniculate nucleus (LGN) and then reach to the primary visual cortex (or V1) situated at the most posterior part of the cerebral cortex. LGN has multi-layered organization, and color information is coded only at specific layers. Cerebral cortex contains a number of visual areas, and these areas consist of two major streams of visual signals. Of these, color information is carried by the ventral visual stream that is thought to be involved in visual recognition of objects. Ventral visual stream starts from sub-regions in V1, include sub-regions of area V2, area V4 and finally reaches to the inferior temporal cortex (or IT cortex) (Fig. 1). In humans, damage in the ventral cortical area around fusiform gyrus results in the loss of color sensation (achromatopsia), so this area should play a critical role in color vision. In the macaque monkey, a very good animal model of human color vision, IT cortex plays a very important role in color vision because selective damage in the IT ∗ This work is supported by a Japanese Grant-in-Aid for Scientific Research (B) and a grant for Scientific Research on Priority Areas from MEXT of Japan. 2 H. Komatsu and N. Goda Fig. 1. Visual pathway in the monkey brain related to color vision. V1: primary visual cortex, V2: area V2, V4: area V4, IT: inferior temporal cortex. TE and TEO correspond to the anterior and posterior part of IT. cortex results in severe deficit in color discrimination [4-6]. In this paper, we will describe our researches on how the color information is represented and transformed at different stages of the visual pathway, and how the neuron activities in the IT cortex are related to the behavior using color signals. 2 Representation of Color Information Color vision originates from the comparison of signals of photoreceptors with different spectral sensitivity functions. Humans and macaque monkeys have three types of cone photoreceptors that are maximally sensitive to long (L), middle (M) and short (S) wavelengths, and they are called L cone, M cone and S cone, respectively. Comparison of signals from different types of cones occurs in the retinal circuit, and the resulting difference signals are sent to LGN through the optic nerve. At this stage, it has been known that color information is carried by two types of color selective neurons, namely, red-green (R/G) color opponent neuron, and blue-yellow (B/Y) color opponent neuron. The former type of cells code the difference between L-cone and M-cone signals (either L-M or M-L). On the other hand, the latter type of cells code the difference between S-cone signal and the sum of the signals from the remaining two types of cones (S-(L+M)). Different laboratories have used different color stimuli to characterize the color selectivity of neurons. In our laboratory, we have used color stimuli based on the CIExy chromaticity diagram [7][8]. To study the color selectivity of a neuron, we used a set of color stimuli that were systematically distributed on the chromaticity diagram and mapped the responses on the diagram (Fig.2). Each color stimulus had the same luminance, shape and area. Color stimuli were presented on the computer display one by one at the same position in the receptive field of the recorded neuron. We employed CIE-xy chromaticity diagram because of the general familiarity of this Color Information Processing in Higher Brain Areas 3 Fig. 2. Color stimuli used in our laboratory that were systematically distributed in the chromaticity diagram. A: Colors plotted on the CIE-xy chromaticity diagram. B: Colors replotted on the MacLeod-Boynton (MB) chromaticity diagram. In both A and B, + indicates the chromaticity coordinates of color stimuli distributed regularly on the CIE-xy chromaticity diagram, ○ those of color stimuli distributed regularly on the MB chromaticity diagram, and Δ the equal-energy white point. Cardinal axes in the MB diagram [L-M and S-(L+M)] are also shown. From [8] with modification. diagram, and because we can easily describe the color selectivity in terms of the combination of cone signals because XYZ space on which CIE-xy diagram is based and LMS space representing cone signals are connected by linear transformation. By using this method, comparison of the color selectivity of neurons in LGN and V1 was conducted [8]. Figure 3 left shows typical examples of color selectivity of LGN neurons. Response magnitude to each color stimulus is expressed as the diameter of the circle and plotted at the position in the chromaticity diagram that corresponds to the chromaticity coordinates of the color. Open circle represents excitatory response and filled circle represents inhibitory response. Cell 1 showed strong response to red colors and exhibited no response to cyan to green colors. This is an example of R/G color opponent neuron. Cell 2 showed strong excitatory responses to blue colors and strong inhibitory responses to colors around yellow. This is an example of B/Y color opponent neurons. In these diagrams, contour lines of the equal-magnitude responses are also plotted. Like these example neurons, LGN neurons generally had straight response contours. To examine how the cone signals are combined to generate these neural responses, response contours were re-plotted on the MacLeod-Boynton (MB) chromaticity diagram [9] by using the cone spectral sensitivities as a transformation matrix [10], and the direction in which the response magnitude most steeply changes (tuning direction) was determined. Lower half of Figure 3 left shows the tuning directions of 38 LGN neurons recorded. They were concentrated only at very limited directions in color space: two large peaks were observed at 0 deg and 180 deg. These correspond to the difference signal between L and M cones: 0 deg corresponds to L-M signal, and 180 deg to M-L signal. Altogether, these peaks correspond to the R/G color opponent 4 H. Komatsu and N. Goda Fig. 3. Left: Color selectivity of two example LGN neurons (top) and distribution of the tuning directions of LGN neurons (bottom). Right: Color selectivity of two example V1 neurons (top) and distribution of the tuning directions of V1 neurons (bottom). See text for the detail. From [8] with modification. neurons. There was a smaller peak at around 90 deg. This corresponds to the difference signal between S cone signal and the sum of L and M cone signals, namely S-(L+M) signal, and represent the B/Y color opponent neurons. We can think that, at this stage, color is decomposed along two axes that consists of MB chromaticity diagram, namely L-M and S-(L+M) axes. Different colors correspond to different weighs on each of these axes. Color is represented in V1 in a way quite different from that in LGN. Right half of Figure 3 shows the color selectivity of two example neurons in V1. Compared with LGN neurons, there were two major differences. First, tuning directions of V1 neurons widely vary and are not restricted in certain directions as observed in LGN. The response contours of cell 3, for example, have orientation that is never observed in LGN. Figure 3 right bottom shows tuning directions of 73 V1 neurons. They are widely distributed across many directions in the color space. This indicates that different hues are represented by different neurons in V1. These results indicate that there is dramatic change in the way hue is represented between LGN and V1. Secondly, many color selective V1 neurons had clearly curved response contours (e.g. cell 4) that were unusual in LGN where neurons in principle had straight response contours. Filled parts of bar graphs in Figure 3 indicate neurons in which a model yielding curved response contours make the data fitting significantly better than any model having only straight response contours. The curved response contours enable to restrict the responses in any region in the chromaticity diagram, and can generate sharp tuning to any hue. Therefore, the neural process involved in forming the curved response contour must be closely related to the process of generating selectivity to various hues in the cerebral cortex. We can also think about the difference in color representation between LGN and V1 in the following way. The Color Information Processing in Higher Brain Areas 5 neural pathway connecting the eye and V1 through LGN consists of only a limited number of nerve fibers compared with the number of photoreceptors. In order to transmit visual information efficiently under such constraint, color information is encoded in a compressed form to reduce redundancy. In contrast, in the cerebral cortex, the constraint of capacity is less severe