The development of Artificial Intelligence has led to models that can process natural language like humans in a wide range of tasks. These Large Language Models (LLMs) have also been shown to be able to predict human behavior on several psycholinguistics tests. This poses the question of whether LLMs can mimic human behavior on tasks that go beyond language and involve sensory experiences. Some individuals experience different forms of synesthesia, a phenomenon in which the stimulation of a sense triggers another sensory pathway. For example, grapheme–color synesthetes automatically and involuntarily associate specific colors to specific letters/numbers. Non-synesthetes, when forced to match colors to letters/numbers, show implicit associations that are remarkably similar to synesthetes’. Would this also be the case for LLMs? And can we observe patterns even when concepts are presented in a language other than English, the main language used to train these models? In this work we present a preliminary study on how humans and LLMs make grapheme–color (upper- and lower-case letters, numbers), weekday-color, and month-color associations in Spanish. The results show that although different LLMs do not always choose the same colors, they show some common patterns. Crucially, these patterns reflect those observed in humans. These results suggest a potential linguistic origin of some of the patterns observed in grapheme-color, weekday-color and month-color associations. More broadly, these findings reinforce the idea that human language reflects cross-modal relationships inherent to human perception, even when the "study subjects" are not real people but artificial intelligence models.

Synesthesia is a neurological phenomenon in which healthy individuals experience additional, automatic, and consistent perceptions unrelated to veridical sensory input. For most (but not all) synesthetes, this additional sensation is a color: for example, grapheme-color synesthetes experience colors for letters of the alphabet. Measuring these color associations is of central importance to synesthesia research, but there is no standard color picker “tool” that researchers can adapt to use in their own experiments: each researcher must code their own. This is a barrier to entry for synesthesia research, and additionally creates potential methodological confounds because different researchers make color pickers with different properties. Here, I present SynesthesiaColorPicker, an open-source, mobile-friendly color picker tool that can be integrated with two popular online experiment platforms (Qualtrics and LabJS/ Open Lab) without any prior programming knowledge. The templates, underlying JavaScript code, and detailed instructions are available for download on a GitHub repository. I then use experimental data to show how two methodological design choices in SynesthesiaColorPicker overcome measurable confounds in existing color picker methodology.

In the past few decades, researchers have established synesthesia as a genuine phenomenon, identified its characteristics (in particular, its automatic, specific and consistent nature), and developed "gold standard" inclusion criteria for research: synesthetes are participants that self-report synesthetic experiences and have consistent (beyond a "cutoff" score) inducer-to-concurrent pairings. While this approach has significantly advanced scientific progress, it can confuse interpretation of research findings due to its inherent circularity: consistency will always appear to be a defining characteristic of synesthesia so long as it is also an inclusion criterion for synesthesia studies. Here, we aim to clarify the relationship between self-report and consistency in "diagnosing"1 synesthesia. In four experiments, we find that: (1) the optimal consistency cutoff score differs across languages; (2) self-reported synesthetes that "fail" consistency tests can still behave like synesthetes - to our knowledge the first objective evidence that "inconsistent synesthesia" is a genuine phenomenon; (3) Using self-report as the sole inclusion criterion does not significantly change the effect size of two measures of synesthetic behavior (the synesthetic Stroop and synesthetic color Palette); and (4) Consistency influences Stroop effect size in self-reported synesthetes only, but influences the Palette in both synesthetes and non-synesthete controls. We conclude that (in certain cases) self-report alone is a sufficient diagnostic criterion for synesthesia, and that synesthesia studies can increase explanatory power by using raw consistency scores as a covariate in analyses, rather than as an inclusion criterion.

We replicate and extend findings on tactile-to-color cross-modal associations. We used nano-engineered material constructions, facilitating future cross-study comparisons on how tactile properties affect visual (color) associations. We obtained systematic associations in normal population, between tactile surface (e.g. rough, soft, sticky) and color properties (e.g., color categories). While gradual color properties (e.g. brightness) also related to tactile properties, these findings were less robust.

This study investigates the perception of tactile wetness, a complex sensation experienced by humans. Previous research has primarily focused on either thermal or mechanical cues separately, or has used textiles as stimuli whose parameters are difficult to control. Here, we employed polyacrylamide hydrogels with varying stiffness levels soaked in liquids of distinct thermal conductivities. By psychophysically evaluating participants' perception of wetness, we showed that the wetness judgments for the samples exhibit a transitive relationship based on the mechanical and thermal cues from an intrinsically tunable organic material. We developed a prediction model of human wetness judgment with an accuracy of 90% and found that the best metrics for the most accurate model were those that were the most human-adjacent: change in temperature at the skin-sample interface (thermal) and compressive force from 2 mm indentation of the sample (mechanical). Given these parameters, we developed a perceptual space capable of recreating 7 distinct levels of wetness perception with the physical parameters used in this study. The results provide insights into the relative contributions of mechanical and thermal stimulus properties in wetness perception. Most notably, this work highlights that the physical characteristics of the skin-stimulus interface can provide ample information for creating a wetness perceptual space, as opposed to the chemical composition of the hydrogels. ### Competing Interest Statement The authors have declared no competing interest.

The haptic sense captures information arising from the somatosensory system-the sensor system of the body excluding the eyes, ears, nose, and tongue. That is, it captures stimuli arising from the skin (i.e., touch) and from internal structures (i.e., the musculoskeletal system and internal organs). The field of research called haptics is concerned with understanding and manipulating this sense, often using engineered technology, and usually for creating novel or realistic touch sensations. Fundamental to every tactile interaction is an interface between the skin and a material. Given that essentially all material objects are composed of or covered in organic media, we reasoned that we, as organic materials scientists, might be able to contribute to the understanding of the sense of touch by manipulating material properties on the molecular scale. Over time, our research group acquired additional skills in electrical engineering and developed strong collaborations with cognitive and behavioral scientists. With a shared curiosity about the sense of touch, we made what we believe are original contributions to the field of haptics.Our approach is guided by a paradigm consisting of four layers from which hypotheses can be generated, experiments can be designed, and whose analytical techniques may be applied. The layers are (1) material composition, (2) material properties, (3) interfacial properties between the skin and the material, and (4) human perception. For example, a material may be composed of one part silicon and two parts oxygen (material composition), which leaves the surface terminated in dipoles and thus a high surface polarizability (material properties). These dipoles may then interact with the skin with a strong van der Waals interaction and high friction (interfacial properties). This friction may lead to stick-slip behavior and could possibly be perceived as fine texture or roughness, even if the surface is smooth (perception).Another useful organizing principle is that of active vs passive touch. That is, engaging with an object with intent vs having an object brush up against one's skin without expectation. In either case, the sensation perceived can be described as either fundamental (e.g., roughness, coldness, compliance, and slipperiness) or blended (e.g., wetness). Beginning with an example of how our approach can be used to understand active touch of a blended sensation, we show how polyacrylamide hydrogels can be tuned by adjusting both the mechanical compliance and thermal conductivity to elicit different levels of perceived wetness. We then show how a purpose-designed conductive polymer can render sensations of roughness in the context of a virtual reality simulation that operates by both passive and active modalities. Lastly, we demonstrate a form of haptic "holography" using the photoacoustic effect; that is, pi-conjugated materials coated on the skin can render sensations of vibration (perceived passively) when exposed to pulsed light. Throughout this Account, we describe how control of materials can be used to elicit senses artificially for both fundamental knowledge and for the ultimate development of richer human-machine interfaces.

Synaesthesia has often been linked to an artistic or creative temperament, but the nature of this link (and, hence, the possible underpinning mechanisms) are poorly understood. This study focusses primarily on people with synaesthesia who have visual experiences, including color, that are induced by music. We determine how this impacts their musical preferences and musical sophistication using previously validated self-report measures and contrast them against non-synaesthetes and synaesthetes with non-musical types. Our data show that people with music-color synaesthesia gravitate toward certain genres (e.g., Reflective and Complex) and show more active engagement with music relative to controls and other synaesthetes. However, synaesthesia as a whole is also linked to greater musical sophistication (e.g., perceptual abilities). A second study examines in detail the nature of associations from musical notes to colors in synaesthetes relative to non-synaesthetes. Synaesthetes have a distinctive way of associating colors with notes: They are more consistent over time, show a more sensitive pitch-luminance correspondence, and have a distinctive color palette (e.g., more browns, fewer greens). These indicative features can be used to determine the presence of this form of synaesthesia.

Abstract The generation of pressure perturbations in matter stimulated by pulsed light is a method widely recognized as the photoacoustic or light‐induced thermoelastic effect. In a series of psychophysical experiments, the robustness of the tactile perception generated with a variety of light sources is examined: a diverging pulsed laser used for photoacoustic tomography optical parameter oscillation (OPO), a miniature diode laser (MDL), and a commercial digital light processing (DLP) projector. It is demonstrated that participants can accurately detect, categorically describe the sensations, and discern the direction of pulsed light travel. High detection accuracy is reported as follows: (d′ = 4.95 (OPO); d′ = 2.78 (modulated MDL); d′ = 2.99 (DLP)) of the stimulus on glabrous skin coated with a thin layer of dye absorber. For all light sources, the predominant sensation is felt as vibration at the distal phalanx (i.e., fingertip, 55.21–57.29%) and the proximal phalanx (41.67–44.79%). At the fingertip, thermal sensations are perceived less frequently than mechanical ones. Moreover, these haptic effects are preserved under a wide range of pulse widths, spot sizes, optical energies, and wavelengths of the light sources. This form of sensory stimulation demonstrates a generalizable non‐contact, non‐optogenetic, in situ activation of the mechanosensory system.

Electrotactile stimulus is a form of sensory substitution in which an electrical signal is perceived as a mechanical sensation. The electrotactile effect could, in principle, recapitulate a range of tactile experience by selective activation of nerve endings. However, the method has been plagued by inconsistency, galvanic reactions, pain and desensitization, and unwanted stimulation of nontactile nerves. Here, we describe how a soft conductive block copolymer, a stretchable layout, and concentric electrodes, along with psychophysical thresholding, can circumvent these shortcomings. These purpose-designed materials, device layouts, and calibration techniques make it possible to generate accurate and reproducible sensations across a cohort of 10 human participants and to do so at ultralow currents (≥6 microamperes) without pain or desensitization. This material, form factor, and psychophysical approach could be useful for haptic devices and as a tool for activation of the peripheral nervous system.

The photoacoustic effect refers to the generation of pressure waves in matter stimulated by light[1]. In the context of radiology (i.e., photoacoustic imaging) waves generated by pulsed laser light are detected by an ultrasound transducer[2–4]. It has been shown that photoacoustic waves produce a mechanical, tactile sensation in humans on bare skin[5]. In a series of psychophysical experiments, performed with both medical grade and off-the-shelf pulsed light systems, participants could detect, categorically describe, and discern the direction of travel of pulsed optical stimuli with the use of a dye as an optical absorber on the skin. To a large extent, the sensations were perceived as localized vibration on the glabrous surface of the fingers, when sensitized with the thin film of dye. This form of sensory stimulation demonstrates an enhanced non-contact, non-optogenetic, in situ activation of the mechanosensory system. This modality of sensation may provide a tool that leads to new insights in psychology, neuroscience, mechanobiology, and the health sciences. Finally, it has many advantageous characteristics for human interaction with artificial environments, as optical signals can be projected onto the skin across distances.

Grapheme-color synesthetes experience linguistic symbols (e.g., letters of the alphabet) as having a consistent color (e.g., “The letter S is burgundy red”). Intriguingly, when non-synesthetes are forced to choose colors for letters, similar non-random distributions are observed. Why are certain letters likelier to be associated with certain colors? Researchers have long sought to explain these trends, and in the past few decades numerous studies have reported correlations between synesthetic colors and various properties of letters, such as ordinal position, frequency in the language, and even pronunciation. These influences, which we call “Regulatory Factors” (RFs), each explain some fraction of the variation in observed associations. In the present work, we provide an updated review of the literature, covering all known studies of RFs. We describe each RF and the operationalization that was used to measure it. For each RF, we also replicate the results in our own database of synesthetes and non-synesthetes, in some cases testing for the first time whether the RF influences the associations of non-synesthete controls. Finally, we introduce a new statistical model of synesthetic associations, that can evaluate the effect of all RFs in a single model.

Grapheme-color synesthetes experience graphemes as having a consistent color (e.g., "N is turquoise"). Synesthetes' specific associations (which letter is which color) are often influenced by linguistic properties such as phonetic similarity, color terms ("Y is yellow"), and semantic associations ("D is for dog and dogs are brown"). However, most studies of synesthesia use only English-speaking synesthetes. Here, we measure the effect of color terms, semantic associations, and non-linguistic shape-color associations on synesthetic associations in Dutch, English, Greek, Japanese, Korean, Russian, and Spanish. The effect size of linguistic influences (color terms, semantic associations) differed significantly between languages. In contrast, the effect size of nonlinguistic influences (shape-color associations), which we predicted to be universal, indeed did not differ between languages. We conclude that language matters (outcomes are influenced by the synesthete's language) and that synesthesia offers an exceptional opportunity to study influences on letter representations in different languages.

Author(s): Root, Nicholas; Rouw, Romke | Abstract: Grapheme-color synesthetes experience linguistic symbols as having a consistent color (e.g., “The letter R is burgundy.”). Intriguingly, certain letters tend to be associated with certain colors, and these biases are not random: numerous properties of letters influence which letter is associated with which color. These influences, called “Regulatory Factors” (RFs), each explain some fraction of the variation in observed associations. No comprehensive model of the influences on grapheme-color associations exists: RFs have only been measured in isolation, are not always operationalized consistently, and often make competing predictions that cannot be accounted for in a univariate model. Here, we describe a statistical framework that integrates the predictions of all RFs into a single model, and thus yields a unified account of their influence on grapheme-color associations. Our model also links these predictions to measurable properties of language, offering a window into the multifactorial contributions to letter representation in the brain.

This paper describes a type of haptic device that delivers two modes of stimulation simultaneously and at the same locations on the skin. The two modes of stimulation are mechanical (delivered pneumatically by inflatable air pockets embedded within a silicone elastomer) and electrical (delivered by a conductive polymer). The key enabling aspect of this work is the use of a highly plasticized conductive polymer based on poly(3,4-ethylenedioxythiphene) (PEDOT) blended with elastomeric polyurethane (PU). To fabricate the "electropneumotactile" device, the polymeric electrodes are overlaid directly on top of the elastomeric pneumatic actuator pockets. Co-placement of the pneumatic actuators and the electrotactile electrodes is enabled by the stretchability of the PEDOT:OTs/PU blend, allowing the electrotactiles to conform to underlying pneumatic pockets under deformation. The blend of PEDOT and PU has a Young's modulus of ~150 MPa with little degradation in conductivity following repeated inflation of the air pockets. The ability to perceive simultaneous delivery of two sensations to the same location on the skin are supported by experiments using human subjects. These results show that participants can successfully detect the location of pneumatic stimulation and whether electrotactile stimulation is delivered (yes/no) at a rate significantly above chance (mean accuracy = 94%).

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. Herein, a haptic glove is described which is capable of producing sensations reminiscent of three types of near‐surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators are combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in the laboratory. This polymer consists of a stretchable polyanion which serves as a scaffold for the polymerization of poly(3,4‐ethylenedioxythiophene). The scaffold is synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity, and low impedance relative to metals. The glove is equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants are able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature have an overall accuracy of 98%, whereas untrained participants have an accuracy of 85%. Sensations can similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.

Grapheme-color synesthetes experience graphemes (e.g., letters of the alphabet) as having a specific, consistent color. Most studies of grapheme-color synesthesia have only examined synesthetes in English, leaving underexplored the question of how synesthetic phenomenology might differ in languages that do not use alphabets. In particular, grapheme-color synesthesia in an abugida (a segmental writing system in which vowels are added to consonant graphemes using 'accent'-like diacritical marks) has never been studied. Here, we present a case study of a Bengali synesthete, MJ, the first report of a grapheme-color synesthete in an abugida. First, we show that for MJ, diacritics influence the overall color of the consonant grapheme they modify, 'pulling' it toward the color she experiences for the vowel. Second, we describe and analyze the complex synesthetic experiences reported by MJ for conjunct graphemes, a unique orthographic feature of Brahmi-derived scripts (such as Bengali) in which multiple graphemes are visually combined into a single 'merged' grapheme. Finally, we show that in addition to these language-specific features, MJ's synesthetic associations are influenced by some of the same linguistic properties (such as orthography and phonology) that influence synesthetic associations in other languages. We conclude that the idiosyncratic features of MJ's synesthesia reflect unique properties of the Bengali writing system, that more studies of synesthesia in non-alphabetic scripts are needed, and that synesthetic phenomenology can offer insights into how linguistic properties shape grapheme representation in the brain.

Grapheme-colour synaesthesia is a neurological phenomenon in which linguistic symbols evoke consistent colour sensations. Synaesthesia is believed to be influenced by both genetic and environmental factors, but how these factors interact to create specific associations in specific individuals is poorly understood. In this paper, we show that a grapheme-colour association in adult synaesthetes can be traced to a particular environmental effect at a particular moment in childhood. We propose a model in which specific grapheme-colour associations are 'locked in' during development in children predisposed to become synaesthetes, whereas grapheme-colour associations remain flexible in non-synaesthetes. We exploit Western gender-colour stereotypes to test our model: we found that young girls in general tend to associate their first initial with the colour pink. Consistent with our model, adult female synaesthetes are influenced by their childhood environment: they associate their first initial with pink. Adult female non-synaesthetes do not show this bias. Instead, in our study, non-synaesthetes tended to associate their first initial with their current favourite colour. The results thus support the 'locking in' model of synaesthesia, suggesting that synaesthetic associations can be used as a 'time capsule', revealing childhood influences on adult linguistic associations. Grapheme-colour synaesthesia may thus offer an extraordinary opportunity to study linguistic development. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

The goal of the field of haptics is to create technologies that manipulate the sense of touch. In virtual and augmented reality, haptic devices are for touch what loudspeakers and RGB displays are for hearing and vision. Haptic systems that utilize micromotors or other miniaturized mechanical devices (e.g., for vibration and pneumatic actuation) produce interesting effects, but are quite far from reproducing the feeling of real materials. They are especially deficient in recapitulating surface properties: fine texture, friction, viscoelasticity, tack, and softness. The central argument of this Progress Report is that to reproduce the feel of everyday objects requires chemistry: molecular control over the properties of materials and ultimately design of materials which can change these properties in real time. Stimuli-responsive organic materials, such as polymers and composites, are a class of materials which can change their oxidation state, conductivity, shape, and rheological properties, and thus might be useful in future haptic technologies. Moreover, the use of such materials in research on tactile perception could help elucidate the limits of human tactile sensitivity. The work described represents the beginnings of this new area of inquiry, in which the defining approach is the marriage of materials science and psychology.

In engineering, the "softness" of an object, as measured by an indenter, manifests as two measurable parameters: (i) indentation depth and (ii) contact area. For humans, softness is not well defined, although it is believed that perception depends on the same two parameters. Decoupling their relative contributions, however, has not been straightforward because most bulk-"off-the-shelf"-materials exhibit the same ratio between the indentation depth and contact area. Here, we decoupled indentation depth and contact area by fabricating elastomeric slabs with precise thicknesses and microstructured surfaces. Human subject experiments using two-alternative forced-choice and magnitude estimation tests showed that the indentation depth and contact area contributed independently to perceived softness. We found an explicit relationship between the perceived softness of an object and its geometric properties. Using this approach, it is possible to design objects for human interaction with a desired level of perceived softness.