This article explores staircases as multimodal sites for semiotic and epistemic reflection on meaning-making. Building on the shared etymology of staircase and scale , and drawing on a scalar understanding of semiosis, I analyse three installations— A Stumbling Staircase , A Staircase of Affects and A Staircase of Memories —to examine how built environments solicit embodied engagement and texture social experience. Rather than serving as neutral architectural structures, the staircases orchestrate distinct multimodal configurations of movement, vision, touch, affect and memory. The first produces instability and fragmentation, disrupting bodily orientation and the visual field. The second semiotizes relations among space, time and people through affective scaling. The third indexes chronotopic complexity and racialized TimeSpace, inviting embodied performances that resignify the staircase, its surroundings and those who traverse it, while fostering memory, resistance and alternative ways of inhabiting place. I argue that these multimodal semiotic-scaling practices emerge through textu(r)alities, i.e., the ongoing weaving of bodies, material environments and semiotic resources through which worlds become apprehensible, inhabitable and transformable. By foregrounding the texturing of embodied experience, the paper challenges modernist understandings of meaning-making that marginalize bodies, affect and materiality.
Academic environments heavily restrict movement and sensory exploration. However, researchers in multiple disciplines have challenged this instructional norm, showing the role of bodily action in developing concepts ( embodied learning in the learning sciences) and of sensory experiences in regulating affect ( sensory regulation and sensory seeking in occupational therapy/psychology). Drawing on embodiment theory, I problematize the intersection of these two perspectives to propose an integrative framework for inclusive pedagogical design that fosters bodily action as both sensory/affective and embodied/conceptual, centering the somatic senses: sensory-adaptive embodied design (SA-ED). I present findings from a task-based interview study with sensory-neurodiverse children exploring a proof-of-concept SA-ED: Balance Graphing. Balance Graphing invites bodily movement, specifically rocking on balance boards, as means of both exploration of math concepts through dynamic graphing activities, and of dynamic sensory regulation through flexible-intensity activation of the vestibular sensory modality. Findings suggest that SA-ED can (1) allow children to adapt stimulation to their sensory needs, (2) legitimize and amplify movement as a meaningful dimension of thinking and communication.
This article examines how narrative emotion and embodied experience take shape as textural atmospheres in olfactory multisensory story design. Drawing on two Norwegian digital picturebooks enhanced with four abstract scents, the analysis traces how smell organises the felt rhythm of reading, orientation, suspension, crisis, and closure, through shifts in affective texture rather than explicit description. Guided by critical sensory multimodal literacy (CSML) as the primary interpretive framework, and informed by phenomenological perspectives on embodiment, the analysis reads each scent encounter as a moment of embodied attunement in which meaning emerges through sensing, not only through language. We situate olfaction within multimodal and socio-material relations, demonstrating how designed scent cues mediate attention, proximity, and belonging in shared reading experiences. The article contributes to discussions on corporeality and texture by framing smell as an atmosphere of narrative experience, an inclusive pathway where the somatic becomes semantic. Rather than treating odour as novelty, we argue that olfactory pacing and valence weave coherent textures of anticipation, tension, and repose that make stories lived as much as read.
In this essay, I discuss my experiences translating poetry by Víctor Cabrera – an author originally from the Chiapas State of Mexico. Cabrera’s work engages with a variety of art forms, including film and music. In the case studies discussed in this essay, I focus on my process for translating A Garden Razed to Ashes [Un jardín arrasado de cenizas] (2014), a work in dialogue with the music of American jazz composer Thelonious Monk, and WIDE SCREEN (2009), a work in dialogue with the films of American arthouse director Jim Jarmusch. In this essay, I discuss my translations in relation to theories of intertextuality, intratextuality, pastiche, and palimpsest. These translation projects raise interesting questions regarding how a translator might recognize the ways in which a literary author incorporates the voices of artists in other media and how these elements can be re-created in a translation to highlight the complexity and transdisciplinarity of the source author. My principal aim in writing this reflection is to illustrate the creative and interpretive activity carried out by translators.
In this study, we explore how students in Year 4 (10 years old) utilize different semiotic modes while designing a comic displaying the evolution of a lizard with a particular trait: armor. Situated use of comic design for natural science didactics is an understudied practice, and, thus, based on video observations of three case students working throughout seven lessons, we draw on designs for learning when exploring how different semiotic modes are utilized in meaning making to create an evolutionary trait in designing a comic. The analysis demonstrates how the lizards’ armor is designed, and how this semiotic content is transformed and transduced by the students through different semiotic modes. The results demonstrate that the students’ process develops from mainly verbal discussions to drawn images, to writing, to finally producing the multimodal ensembles of their comics. However, this process is not linear, and the analysis shows how sometimes an individual student’s drawing, gestures, or writing can also become the origin of an idea that is developed through transductions to other modes. Thus, in comic design, transductions between modes support students’ understanding of the biological content, as well as their ability to formulate more general conclusions in the classroom.
AI-generated images now function as routine multimodal texts, yet we know little about how they construct meaning beyond minimal prompts. Using an SFL-informed multimodal framework, this study analyses intersemiotic expansion, projection, and affect in a purpose-built corpus of 120 text-to-image outputs produced with a strictly controlled two-line prompt. Four subject types differing in animacy and conventional agency were crossed with six minimally specified predicates spanning material, mental, verbal, and projective processes (five independent generations each). Coding shows that divergence from prompts follows stable semiotic patterns rather than random variation. Action-oriented predicates strongly license specification and addition, whereas predicates foregrounding internal states rely more heavily on projection and selective specification. Subject type modulates these strategies: schema-rich human subjects constrain variation, while biologically misaligned or non-agentive subjects trigger compensatory environmental reconfiguration, explicit projection, and heightened affective loading. Projection peaks where subject plausibility is lowest, and affect intensifies for prompts involving desire or internal states. These findings suggest that AI-generated images operate as multimodal meaning-making artefacts that actively reconstrue ideational, interpersonal, and textual meanings, rendering culturally sedimented visual schemas analytically visible.
Abstract This review examines Olfactory Worldmaking (2026) by Hsuan L. Hsu as a pivotal intervention in the environmental humanities, reconceptualizing olfaction as a medium of relational and reparative worldmaking. Building on but reorienting The Smell of Risk (2020), Hsu moves beyond modern regimes that equate odor with toxicity and contamination to theorize smell as transcorporeal, affective, and atmospheric, enabling humans and more-than-human environments to co-constitute worlds. Published in the wake of COVID-19, when widespread anosmia rendered smell perceptible through absence, the book frames olfactory deprivation as a structural feature of modern deodorized spaces rather than an anomaly, highlighting contemporary environmental estrangement and sensory normalization. Across chapters on smellscapes, Black diasporic “microclimates,” and speculative aesthetic practices, Hsu shows how olfactory experience functions as a multimodal site of ethical and political imagination. Smell activates distributed memory, sustains life within racialized atmospheres, and facilitates alternative modes of sensing that challenge anthropocentric perception. Drawing on cross-disciplinary scholarship, Hsu advances an atmospheric ethics grounded in opacity and “solidarity in incommensurability,” emphasizing participatory, embodied engagement. While the book’s breadth is a major strength, the reviewer suggests it could benefit from more sustained engagement with specific case studies and further theorization of non-Western sensory epistemologies, offering fruitful avenues for future research.
This paper presents the first case study to apply social semiotic analysis to the spatial text of a museum in Japan. Focusing on the Kanazawa Yasue Gold Leaf Museum building and its Permanent Exhibition Room, the study examines how this single-theme museum dedicated to gold leaf constructs meaning. Using the framework of Spatial Discourse Analysis, it analyzes meaning-making processes through three metafunctions: representational, interactional, and organizational. The analysis shows that the museum, through the multimodal orchestration of linguistic, visual, and spatial resources and their interaction with visitors’ movement and experience, produces a narrative that extends beyond a technical explanation of gold leaf. This narrative highlights the lifeways of gold-leaf craftspeople, the aesthetics that emerge from the interplay of technique and material, and a regional identity shaped by the integration of tradition and creativity. The study contributes a new empirical case to social semiotic research and offers fresh perspectives for understanding regional culture in Kanazawa and for advancing museological practice in Japan.
We always remember the movements. The Harlem Renaissance. Motown. The Bronx in the summer of 1973. But we oftentimes forget what makes the art that we love, art. We forget the rooms before the recordings. The experiments that never became albums. The joy that had no audience. This essay documents “Flow-State,” a creative retreat with five (six) musical and multimedia artists from New Orleans, Pittsburgh, Chicago, Detroit, (and Norfolk). For 3 days, we had only time and space—nothing to produce, no one to please. Here, I ask: What does creative practice sound like when it refuses to become a finished product? What frequencies emerge when the self is allowed to wander? Drawing on sound studies and Black feminist aesthetics, I take up the unsold interiority of the studio, and the affective densities of just wandering through music. The first note played when no one is listening. The laughs between takes. Those moments before anyone decides what it means. The flow-state, as I understand it, is less a neurological event than a prefigurative relation, one where trust becomes the condition of possibility. In writing about these moments, I want to elevate what our archives often miss: the everyday happiness of artists just being together, in real time.
Prints have long been associated with language and communication, yet the subtle processes of production that scaffold their graphic traces often remain a mystery, unseen and entangled with the transitory and social stages of their making. This article follows the collective and multimodal semiosis of a printmaking workshop during a residency with Barbadian artist Annalee Davis, and the co-development of a print edition titled An Impossible Map . This residency was in partnership with the National Trust for Scotland’s project Facing our Past and Dundee Contemporary Arts Print Studio. Analysis focuses on the significance of texture and touch and the journey towards Davis’s selection of an inkless process resulting in a ‘blind emboss’, offering insights into the critical potential of print to evoke what Davis called the ‘inadequacies of the archive’ regarding historic forced banishments of indentured labourers from Scotland to the Caribbean to support the prosperity of the sugar trade.
This practitioner reflection draws from a community-based comic initiative to examine how participatory and collaborative storytelling unfolds in multimodal production. I introduce semiotic divisibility to describe how meaning-making in comics may be distributed across linguistic, visual, or spatial modes, which, in turn, allows collaborators to contribute according to the semiotic mode that aligns best with their storytelling capacities and resources. Through selected vignettes, I reflect on how this distribution democratizes creative control and intuitively opens up spaces for participatory storytelling. I also make a case for why comics, as multimodal artifacts, serve as sites to negotiate multiple collaborating voices, a quality that remains particularly insightful for community-based storytelling initiatives.
This paper introduces a novel method for investigating how public service, market-driven, and populist news outlets differ in their construction of ideological positions in audiovisual reporting. A semi-automatic framework of multimodal semiotic content analysis is presented that combines computational audiovisual feature detection with theory-driven manual annotation of discourse and narrative elements and shows early results concerning the construction of distinct positions on COVID-19. The method presented suggests a powerful reconceptualisation of what counts as “large-scale” analysis in multimodal video research, arguing that scale should be defined not merely by the number of videos but be explicitly related to analytical complexity, levels of abstraction, and the density of annotated units, such as shots and many other multimodal features. Based on a corpus of 107 German news reports (January–March 2022, containing around 5000 shots) from Tagesschau (public service), BILD TV (market-driven), and CompactTV (populist/right-wing), we systematically conduct shot-by-shot annotation for multimodal semiotic content analysis. The analysis identifies recurring narrative patterns across channels and uncovers how multimodal configurations materialise distinct ideological orientations. Statistical comparisons show clear cross-channel differences. Tagesschau foregrounds expert voices and largely avoids sen-sationalist framing. BILD TV emphasises emotionalised language associated with shock and “craziness”, while CompactTV amplifies oppositional actors and frequently frames COVID-19 measures in terms of “dictatorship” and “coercion”. These findings demonstrate that multimodal discourse patterns function as ideological performances, shaping whose voices are legitimised, how events are staged, and which emotions are mobilised. The study illustrates how semi-automatic multimodal analysis enables scalable, fine-grained investigation of narrative form and ideological signalling in contemporary news environments.
This article develops a new materialist approach to the representation of mystical-type experiences in Virtual Reality (VR). It investigates how sensory modalities and virtual environments are employed to translate the indeterminate and invisible qualities of mental states and evoke experiences of liminal consciousness that are characteristic of mystical-type experiences associated with psychedelic use or psychosis. Drawing on research in atmosphere and new materialism, I argue that atmosphere serves as a central mechanism in VR for rendering invisible and relational processes tangible. To support this claim, I analyze the VR works Manic VR and Ayahuasca – Kosmik Journey , focusing on how the medium-specific affordances of VR are used to construct immersive, affectively charged atmospheres. I show how these atmospheres enable imaginative and felt experiences that resonate with the mental states they seek to represent. The two pieces are discussed as part of a spectrum of non-dualist, relational aesthetics, ranging from restrained, apprehensive attunement to deep somatic absorption.
Of the five physical senses, touch is perhaps the most broadly used in metaphorical and analogic meanings. To be “touched,” or for something to be “touching,” can be interpreted in a variety of ways depending on context and shared understanding. In aesthetic criticism, hapticality has been interpreted similarly broadly, particularly as regards photography and film. But much of this work has only hinted at the possibilities for the hapticality of sound. On a parallel disciplinary track, trans studies have produced important work in haptics and the constructedness of bodies and gender, which has also been used in some trans aesthetic philosophy. This paper attempts to join the two in interrogating a trans haptic aesthetics through the work of Venezuelan pop musician Arca, and particularly her five-album KiCk series. Through these albums, I propose a trans haptic aesthetics based on a capacious understanding of texture: through the sonic, thematic, and self-presentational elements of the music. In so doing, I work towards a resistant, liberatory aesthetics that uses texture to reach out and touch communities of listeners, and speculate on the effects of listeners being touched and touching back.
In spoken interactions, repair-the process by which interactants identify and resolve communication trouble-has been studied extensively. Fewer studies have explored the inherently multimodal repair process in written interactions, and almost none have examined it within the semiotic landscape, that is, on publicly visible signs. From a corpus of around 5500 images collected in Germany and Iran, we present a qualitative analysis of six selected cases to examine the functions of multimodal repair in social and political commentary. Using ethnographic linguistic landscape analysis (ELLA) and geosemiotics, the analysis focuses on images from Mannheim, Cottbus, Leipzig, Tehran, and Zanjan. Results indicate that actors in the semiotic landscape use different multimodal repair strategies for functions that go beyond identifying trouble in communication. Instead, they rely on repair strategies that are recognizable to viewers, but use these strategies to perform social and political commentary in different sociopolitical contexts. Implications of these findings for theories of multimodality are considered, especially in showing that repair is not fundamentally temporal but is shaped by the multimodal affordances (such as spatial organization) of the communicative context in which it occurs.
This practitioner reflection examines how painting in plein-air-meaning outdoors from life-inspired multimodal fieldnotes that became the node between research, teaching, and place. This work was conducted during a 4-year, community-engaged art education study where the community art classroom was employed to develop participatory arts-based research methods by inviting participants to reflect on changes in Pointe-Saint-Charles, a post-industrial neighborhood in Montréal's Sud Ouest. I began painting individually using an iPad, easel, and small mint-tins, which forced close looking while capturing deeper affectual, experiential, material, and spatial insights that transcended the written word. I share three ways painted fieldnotes provided insights into the neighborhood. The first is socio-spatial insights that revealed the deeper changes in infrastructure and their cultural meanings. The second is painting as theory generation through meditative, ongoing engagement throughout my research. The third is painting as a multimodal public discourse, where the act of painting stoked reflection and engagement from the public, students, and colleagues. I also share how these fieldnotes changed my approach to my own teaching and how I teach pre-service teachers by inviting closer engagement between personal practice and teaching. This paper argues that the painted fieldnote offers a valuable, multimodal method for thinking, noticing, and teaching through place.
By stating their ideology and policies, new parties are able to affirm their authority as legitimate participants in the democratic process. Defining the political party’s identity through affirmation means also acknowledging the boundaries of this identity. This can be identified through markers of negation. Socially, a community’s identity can be defined not only by what it is, but also by what it is not. Linguistically, the use of negative structures implies that there is a corresponding affirmative statement. Gesturally, it has been argued that negation involves specific types of gestures. The verbal and visual markers of negation can therefore be transposed on to the discursive interface. This article focuses on two speeches given by Nigel Farage, the leader of the political party Reform UK. It argues that negation serves an important role in the construction of the new political party’s identity by facilitating the process of identifying the available and desirable options for identity construction. Gesture analysis is integrated into the linguistic analysis of the adverbs of negation not , -n’t and never in the verbal group, with a focus on the force-dynamic properties of modal auxiliary verbs in relation to negation.
Silas Malafaia is one of the most influential televangelists in Brazilian history. This study employs Multimodal Metaphor Analysis to investigate how Malafaia uses metaphors during a training sermon for new pastors and church workers. Through microanalyses of portions of the sermon, we show how Malafaia employs metaphors at both speech and gesture levels to construct an “us versus them” narrative around a supposed confrontation between Neo-Pentecostal believers and followers of an African-Brazilian religion. Through a combination of spoken and gestured orientational metaphors (i.e. Good is Up; Bad is Down), he then emphasizes a negative affective stance toward followers of African-Brazilian religions, associating them with the Devil. The Multimodal Metaphor Analysis performed highlights how, as an epistemic agent, Malafaia instills in his congregation a particularly antagonistic, even bellicose, attitude towards believers of another faith. Results suggest that power structures can be reinforced through gesture use.
