Live coding, or real-time algorithmic performance, is a rich medium for engaging novices in informal creative STEM learning. However, despite inclusive and open-source communities, disabled practitioners are underrepresented in live coding, and prior work highlights numerous accessibility barriers. To understand the perspectives of Blind and Low Vision (BLV) live coders, we formed FiLOrk (Fil Laptop Orchestra) with five BLV teens. Across two semesters, FiLOrk performed three original works, each guided by a core concept and improvisational structure for manipulating code and maintaining shared awareness. We interviewed four musicians to understand how they felt about the learning environment and how their creative identities formed individually and in relation to one another. We reflect on FiLOrk’s outcomes and propose strategies for future live coding ensembles to meaningfully include novices with and without disabilities.

Live coding is a performance art in which media, commonly music and video, are generated and manipulated with open-source tools. While live coding initially centered upon local meet-ups, many individuals and loosely organized groups now learn, collaborate, and share work online. This expansion of interest may offer new, informal opportunities to learn computer programming, but little is known about how releveant online forums operate. To better understand this online space, we gathered and analyzed content from 414 threads in two popular live coding forums, Tidal Club and in_thread. We found that while both communities include discussions about using and extending core programming tools, significantly more posts on Tidal Club pertain to installation difficulties, while in_thread possesses a wider culture of sharing creative artifacts. We reflect on these findings and offer suggestions for future work in understanding the motivations behind why live coders post code, how they receive help, and the extent to which informal learning in live coding forums aligns with formal computer programming and music teaching.

The Fil Laptop Orchestra, aka FiLOrk, is an electronic music ensemble that performs through live coding, i.e. collaboratively editing in a shared document to generate sound. In forming FiLOrk, we set out to use music performance and tactile graphics to address a lack of accessible tools and curricula for teaching Blind and Visually Impaired (BVI) people to code. In this poster, researchers and the high school members of FiLOrk, who use screen readers and magnification, report their experiences composing and performing two original works that incorporate assistive technologies. We discuss a shift from instruction to open-ended experimentation, and we highlight technical and pedagogical opportunities for making live coding more accessible.

Understanding how youth make sense of machine learning and how learning about machine learning can be supported in and out of school is more relevant than ever before as young people interact with machine learning powered applications everyday; while connecting with friends, listening to music, playing games, or attending school. In this symposium, we present different perspectives on understanding how learners make sense of machine learning in their everyday lives, how sensemaking of machine learning can be supported in and out of school through the construction of applications, and how youth critically evaluate machine learning powered systems. We discuss how sensemaking of machine learning applications involves the development and integration of conceptual, creative, and critical understandings that are increasingly important to prepare youth to participate in the world.

While music notation can be represented across modalities, it is typically only available visually (in standard print) resulting in access barriers for Blind and Visually Impaired (BVI) musicians. Automated conversion tools and common formats such as MusicXML have enabled new workflows where a print score is transcribed (copied) and made into large print and/or braille music. However, many musicians are unable to acquire music in a format legible to them as transcriptions require time, expertise, and specialized software to produce. Drawing upon the experiences and suggestions of 11 adult BVI musicians with varying vision ability and musical backgrounds, we outline a path to improving access to music notation. To this end, we describe opportunities for utilizing automation and crowd workers, make recommendations for customizing music in print, braille, and audio, and identify open challenges and research directions for creating, storing, and sharing music across formats.

Performing arts computing environments have received little attention in the educational sphere; yet, they offer opportunities for learners to validate their efforts, ideas, and skills through showcasing their work in a public-facing performance. In this work, we explore an out-of-school dance and computing educational program run by the organization, STEM From Dance. The organizational mission is to create an equitable learning experience for young women of color to engage with computing while exposing them to STEM careers. Through an analysis of eleven interviews with youth participants, instructors, and the executive director, we examine how the social, cultural, and political dimensions of the learning environment facilitate identity work in computing and dance. Our findings point to three primary activities used by the organization to promote equity: (1) providing psychological safety through a supportive community environment, (2) meaningfully engaging with learners’ social and cultural context through creative work with constructionist artifacts, and (3) actively promoting identity work as women of color in computing and STEM through both artifact work and community events. Applying the constructs of identity and psychological safety we explore the tensions and synergies of designing for equity in this performing arts and computing learning environment. We demonstrate how the seemingly contradictory elements of a high-stakes performance within a novice learning environment provides unique opportunities for supporting young women of color in computing, making them non-negotiable in the organization’s efforts to promote equity and inclusion. Our work illustrates how attending closely to the sociocultural dimensions in a constructionist learning environment provides lenses for navigating equity, identity work, and support for inclusive computing.

Artificial intelligence (AI) and machine learning (ML) systems are ubiquitous across many fields ranging from medicine (e.g., tumor detection), to natural language processing (e.g., digital home assistants, auto-translate tools), and personalization (e.g., social media recommendations). The rise of these systems has also seen a rise in the ethical challenges resulting from how some of these systems are designed and implemented. Most problematic is their propagation and amplification of problems such as racism and sexism [4, 6], among others. Further, these challenges exist within a computing discipline that is already burdened by exclusive, marginalizing cultures and practices that lead to low participation by women and Black, Indigenous, and People of Color (BIPOC) [3]. Our project, AI + Dance, attends to these dimensions of inequity within AI/ML education through developing our understanding of how we can equip learners to recognize and rectify issues of AI and ML systems within an inclusive and culturally sustaining experience. We explore this in collaboration with STEM From Dance1 (SFD), a non-profit organization that supports girls of color in creative production with dance, CS, and STEM. In our prior work with SFD, we developed danceON, an open-access creative coding environment that enables learners to create code to engage authentically with dance and body motion [5]. With danceON, learners can code animations that can bind and respond to body positions and be statically and dynamically positioned in space. The system provides two ways to explore AI and ML. First, it integrates a pose detection machine learning model (PoseNet [8]) that consists of body points (i.e., Left Ear, Nose, etc.) and confidence scores for these key points, which are all accessible within the code. Second, danceON enables learners to import a model trained in Google’s Teachable Machine [2] to access the probabilistic classifier to manipulate and trigger animations. While we have explored the use of danceON to enable learners to build culturally relevant artistic artifacts [5], we have yet to explore how we can teach AI and ML concepts and ethics through the use of the system. We investigate two research questions through the co-design of culturally sustaining AI/ML resources with teachers: RQ1) How can we authentically build on learners’ identities, cultural knowledge, and practices with dance as they build, explore, and critique ML and AI models and systems? RQ2) How can we leverage learners’ creative, embodied experiences with ML and AI to facilitate reflection and critique of CS and AI/ML within their communities and society more broadly? We focus on co-design as a method that facilitates sustainability through centering teachers’ values, ownership, and authentic contexts in the realization, implementation, and evaluation of learning designs [7]. Through co-designing curricular resources, we will develop a set of modules aligned with the AI4K12 guidelines [1] that builds our understanding of creating culturally sustaining educational resources to teach about AI and ethical design in ways that leverage the culturally situated, collaborative, and embodied nature of dance.

Technologies for notating music pose usage barriers to blind and visually impaired musicians requiring many to overcome a significant learning curve and/or rely on complicated tool chains with limited screen reader support. To address a need for accessible music notation software, we present SoundCells, a browser-based system designed to make music notation easy, intuitive, and accessible to screen reader users, and output music in audio, print, and braille formats. We share findings from a co-design process, in which two experienced musicians used SoundCells for two months guided by four remote meetings, and from a Design Probe, in which five other musicians tried SoundCells with a screen reader and reflected on its usability and accessibility in the context of their current practices. Finally, we discuss design recommendations relevant to a broader ecosystem of creative technologies, including how text-editing and multi-modal output capabilities could be extended and improved, how SoundCells' current design facilitated remote collaboration between sighted researchers and blind musicians, and future opportunities for learning and sharing music on the web.

Music creation technologies often use highly visual representations of music and mouse/touch screen gestures for manipulating it. While graphic user interfaces can make music creation more understandable and approachable, they pose access barriers to blind and visually impaired people who operate computers with keyboards and use assistive technologies like screen readers. I believe that vision ability should not be a prerequisite to expressive music creation with technology. In my dissertation, I intend to address the accessibility barriers prevalent in music technology design across three phases of research: First, an interview study conducted in 2020 with experienced blind and visually impaired music makers depicts the the current state of accessibility and workflows within commercially available tools [18]. Second, findings from that study led to the ongoing development of a browser-based music notation tool SoundCells. A technical paper describing its design along with the results and recommendations gathered from co-design and testing with 7 blind musicians will be presented at Web4All, 2022. Finally, I propose forming an electronic music ensemble with blind and visually impaired music makers who will collectively write and perform original music using SoundCells and other technologies. Much of this research is carried out in partnership with students and teachers at the Filomen M. D'Agostino Greenberg (FMDG) School [9], a community music school that teaches blind and visually impaired musicians of all ages and skill levels.

In this hands-on workshop participants will experience the curricula from three NSF funded projects, which engage youth in creating art with and about AI technologies while exploring related ethical concerns. danceON is a web-based creative coding environment that engages learners in creating multimedia dance performances. Besides writing reactive code that generates animations to augment dance performances, students also use the system to explore the boundaries and biases of AI. DAILy is a curriculum focused on developing AI literacy among middle school students through the integration of technical concepts and processes, ethical and societal implications, and career futures in AI. Participants will be focusing on the AI + art modules of DAILy. Imagine AI develops project-based curricula to support youth in exploring critical ethical issues related to AI. Students read short stories featuring youth at the center of AI ethical dilemmas, build and manipulate AI systems, and create digital media to express ethical stances. Participants will leave the workshop with multiple AI teaching strategies that blend technical learning with social purpose and creative expression.

In this hands-on workshop participants will experience the curricula from three NSF funded projects, which engage youth in creating art with and about AI technologies while exploring related ethical concerns.

Technologies for notating music pose usage barriers to blind and visually impaired musicians requiring many to overcome a significant learning curve and/or rely on complicated tool chains with limited screen reader support. To address a need for accessible music notation software, we present SoundCells, a browser-based system designed to make music notation easy, intuitive, and accessible to screen reader users, and output music in audio, print, and braille formats. We share findings from a co-design process, in which two experienced musicians used SoundCells for two months guided by four remote meetings, and from a Design Probe, in which five other musicians tried SoundCells with a screen reader and reflected on its usability and accessibility in the context of their current practices. Finally, we discuss design recommendations relevant to a broader ecosystem of creative technologies, including how text-editing and multi-modal output capabilities could be extended and improved, how SoundCells' current design facilitated remote collaboration between sighted researchers and blind musicians, and future opportunities for learning and sharing music on the web.

Commercial technologies for notating music pose usage barriers to blind and visually impaired (BVI) musicians because they use graphic user interfaces and only produce visual, print scores. However, more research to date has studied how to make existing scores available in braille or large print rather than understand the needs and workflows of BVI musicians who notate new music. To address this gap, we conducted a six-week remote study in which six BVI musicians with wide-ranging backgrounds wrote original music culminating in a live performance. To create their scores, participants used SoundCells, a product of ongoing co-design and testing with BVI musicians that uses text to generate audio, print, and braille music. Across three interviews, participants offered diverse and nuanced views of how text input could facilitate creative expression. We uncovered how vision ability, music experience, and assistive technology preference affected how music was accessed and traversed. From this research, we provide design recommendations for improving SoundCells’ input and output systems, discuss how visual cues embedded in SoundCells’ syntax make learning and remembering harder for people who can’t view it, and reflect on how our chosen methods resulted in high engagement.

Block-based coding environments enable novices to write code that bypasses the syntactic complexities of text. However, we see a lack of effective block-based tools that balance programming with expressive music making. We introduce Toneblocks, a prototype web application intended to be intuitive and engaging for novice users with interests in computer programming and music. Toneblocks is designed to lower the barrier of entry while increasing the ceiling of expression for advanced users. In Toneblocks, users produce musical loops ranging from static sequences to generative systems, and can manipulate their properties live. Pilot usability tests conducted with two participants provide evidence that the current prototype is easy to use and can produce complex musical output. An evaluation offers potential future improvements including user-defined variables and functions, and rhythmic variability.

Dance provides unique opportunities for embodied interdisciplinary learning experiences that can be personally and culturally relevant. danceON is a system that supports learners to leverage their body movement as they engage in artistic practices across data science, computing, and dance. The technology includes a Domain Specific Language (DSL) with declarative syntax and reactive behavior, a media player with pose detection and classification, and a web-based IDE. danceON provides a low-floor allowing users to bind virtual shapes to body positions in under three lines of code, while also enabling complex, dynamic animations that users can design working with conditionals and past position data. We developed danceON to support distance learning and deployed it in two consecutive cohorts of a remote, two-week summer camp for young women of color. We present our findings from an analysis of the experience and the resulting computational performances. The work identifies implications for how design can support learners’ expression across culturally relevant themes and examines challenges from the lens of usability of the computing language and technology.

Many blind musicians and composers read and write music using braille. Yet, braille music is not as widely available as print (visual) music, sighted collaborators and educators do not read braille music, and workflows and toolchains for converting between print and braille music are complex. In this research, we present Sound Cells, a music notation system that simultaneously outputs visual and braille notation, and provides audio feedback as a user writes music with text. We share findings from a Design Probe in which two experienced blind musicians notated music using Sound Cells and reflected on it in the context of their current notation practices. Finally, we highlight music navigation and outputted score customization as opportunities for further study.

Both music creation tools and novice coding environments often use highly visual interfaces to aid in writing, editing, and navigation. Such designs fail to include blind users and may be totally inaccessible. This work sets out to develop a novel domain specific language, interface, and curriculum in partnership with students and teachers at the Filomen M. D'Agostino Greenberg (FMDG) School, a community music school that reaches blind and visually impaired musicians of all ages and skill levels. The project will consist of an iterative co-design phase and an implementation/evaluation phase in which a small number of students at the school will use the technology during a music composition course to create original works.