On cross-modal similarity: Perceiving temporal patterns by hearing, touch, and vision

How does the similarity perceived between two brief temporal patterns depend on properties of the stimulus (durations of pulses and intervals between pulses) and on the sensory modality of presentation? Multidimensional scaling of similarity ratings obtained in a series of six experiments revealed four main findings: (1) Two-dimensional solutions provided good fits to all of the data, with the major dimension representing the order of intervals between pulses (long-short vs. short-long), the minor dimension representing overall duration (primarily of the pulses). (2) Essentially the same two-dimensional representation characterized similarity in intramodal comparisons (hearing, touch, vision) and cross-modal comparisons (hearing-touch, hearing-vision, vision-touch). But, (3) the major dimension-interval order-was even more prominent in crossmodal comparisons. Finally, (4) although individual subjects varied widely as to which dimension they weighted more heavily, individuals were consistent in this over all six (intramodal and cross-modal) tasks. Despite some differences between intramodal and cross-modal perception of similarity in temporal patterns, a large component of the similarity judgments transcends individual modalities.