Augmentation allows rapid reconfiguration of passive physical interfaces to improve accessibility, support independent living through domestic automation, and more. However, its potential is largely unrealized for novice users due to several key barriers. First, users rarely identify latent interaction problems within their built environments. Second, they often lack the knowledge to clearly express design intent. Third, many innovative solutions remain in research prototypes, limiting access. We introduce EUREXA, an agentic AI system to share the spirit of discovery (“Eureka!”). EUREXA supports end-users through a diagnose–discover–describe workflow: from input with varying ambiguity and complexity, it surfaces latent interaction challenges, presents reconfiguration opportunities through augmentations, and produces interpretable designs. Its novelty is a dual search across public augmentation repositories and research articles, enabling reusable designs even when no design libraries or parametric tools exist. EUREXA transforms non-parametric models into parametric ones or directly generates fully explainable designs. To evaluate EUREXA across varied user inputs, complexities, and clarity levels, we define ambiguity metrics, conduct a user study, and report critical factors for advancing generative AI to help end-users readily augment physical interfaces through fabrication.

Light’s interaction with object surfaces through anisotropic reflection–where reflected light varies with viewing angles–offers significant potential for enhancing visual capabilities and assisting informed decision-making. Such ubiquitous light transfer phenomenon supports directional information encoding in sensing and dynamic display applications. We present LumosX, a set of techniques for encoding and decoding information through light intensity changes using 3D-printed optical anisotropic properties. By optimizing directional reflection and brightness contrasts through off-the-shelf materials and precise control over processing parameters (e.g., extrusion volume, raster angles, layer height, nozzle positioning), we enable cost-effective fabrication of visually enhanced objects. Our method supports modular assembly for highly curved regular surfaces and direct printing on top of relatively flat curved surfaces, enabling flexible information encoding for diverse applications. We showcase LumosX’s effectiveness through various indoor and smart urban sensing scenarios, demonstrating significant improvements in both human interaction and autonomous machine perception.

Imbuing sensing and interactivity into everyday objects has long been sought after within the HCI community to facilitate richer and more immersive user experiences. However, conventional methods rely on costly hardware, such as embedded sensor tags, or passive visual markers that lack digital capabilities to sense user context. We present LuxAct, an interaction-powered visual communication system that enables everyday objects to encode their information and user interaction data into sequences of RGB-colored light. These sequences are decoded by Point of View (POV) cameras on AR headsets or smart glasses to derive meaningful information from interactions. LuxAct mechanisms are self-powered and ultra-low-cost, leveraging striking and plucking on piezoelectric generators to harvest energy from user interactions. Through strategic pattern design, our system transforms visual channels into carriers of both object identification and sensory data, supporting applications with rich sensing needs. We demonstrate a wide range of use cases, including interactive controls, sensate storage, smart water hose, medicine reminders, fingertip probes and beyond, offering a practical alternative for digitizing passive objects to enable ubiquitous sensing in AR-enhanced environments.

Transparent materials transmit light without significant scattering or absorption due to total internal reflection. Transparent channels in 3D printed objects follow this theory, functioning similarly to optical fibers by transmitting input light. While prior work enabled basic interactions in custom optical sensors like push and display interactions using photopolymers, complex channeling remains challenging, particularly in desktop Fused Deposition Modeling (FDM) due to the inherent printing discontinuities. We present a framework that enables low-cost desktop fabrication of optical interactive devices using Voronoi-based segmentation of objects for in-place FDM printing of optical channels. It allows uninterrupted light transmission where the embedded channels are aligned in the XY plane for uninterrupted printing. We further explore the use of FDM optical objects in routing and channeling strategies to support applications in displays, sensing, and embodied interactions.

Kerfing is a well-known method in subtractive manufacturing used to create flexible surfaces from stiff planar materials. In this work, we study 3D-printed kerfing to enable freeform movement in rigid polymer materials suitable for FDM (Fused Deposition Modeling). Designers and end users can leverage kerf structures due to their ability to bend in single, double, or multiple axes. With the accessibility of consumer-grade 3D printers, kerf structures offer an easy approach to fabricating freeform structures and compliant mechanisms for interactive primitives and applications like grippers. Building on the principles and mechanics of traditional subtractive kerf structures, we propose that 3D-printed kerf structures can be modified by varying the shape, cell density, and alignment of cells, which influence their deformation and load-bearing capabilities. Additionally, we examine how the unique advantages of 3D printing can further augment the capabilities of kerfing, such as through multi-material printing (e.g., applying a thin TPU layer over PLA at contact points of a kerf-structured gripper for improved friction) or bilayer structures (e.g., using a one-to-one proportion of PLA and TPU material overlaid over one another in the kerf structure for a wearable sensor band). We validate our approach through various applications and the interaction spaces created by kerf designs, such as tangible user interfaces with tunable haptic feedback and robotic grippers.

Sign language serves individuals with hearing impairments as a crucial communication mode operating through visual-manual means. While there has been established theory and agreement about embodiment in multiple fields, only limited research has deeply engaged to lower access to the physical body for spatial perception and engagement. Embodied robots are often cost-prohibitive, and existing open-source robot fabrication packages are limited in their ability to fully address communication nuances, typically running only on predefined programs. Reprogramming for broader bodily interactions, such as gestures in various domains (e.g., construction), is nearly impossible unless expertise precedes. We introduce FABRIC, an end-to-end toolkit for fabricating and programming bodily language for unique human-robot interactions. The toolkit includes a fully 3D-printable robot, designed for consumer-grade FDM machinery, that learns from demonstration (LfD) to capture and translate users’ bodily expressions through its upper torso (arms and hands) movements. A visual programming interface enables appending or sequencing demonstrations from various sources, i.e., videos, cameras, and expandable word/phrase/sentence libraries.

Instance detection (InsDet) aims to localize specific object instances within a novel scene imagery based on given visual references. Technically, it requires proposal detection to identify all possible object instances, followed by instance-level matching to pinpoint the ones of interest. Its open-world nature supports its broad applications from robotics to AR/VR but also presents significant challenges: methods must generalize to unknown testing data distributions because (1) the testing scene imagery is unseen during training, and (2) there are domain gaps between visual references and detected proposals. Existing methods tackle these challenges by synthesizing diverse training examples or utilizing off-the-shelf foundation models (FMs). However, they only partially capitalize the available open-world information. In contrast, we approach InsDet from an Open-World perspective, introducing our method IDOW. We find that, while pretrained FMs yield high recall in instance detection, they are not specifically optimized for instance-level feature matching. Therefore, we adapt pretrained FMs for improved instance-level matching using open-world data. Our approach incorporates metric learning along with novel data augmentations, which sample distractors as negative examples and synthesize novel-view instances to enrich the visual references. Extensive experiments demonstrate that our method significantly outperforms prior works, achieving >10 AP over previous results on two recently released challenging benchmark datasets in both conventional and novel instance detection settings.

In our increasingly diverse society, everyday physical interfaces often present barriers, impacting individuals across various contexts. This oversight, from small cabinet knobs to identical wall switches that can pose different contextual challenges, highlights an imperative need for solutions. Leveraging low-cost 3D-printed augmentations such as knob magnifiers and tactile labels seems promising, yet the process of discovering unrecognized barriers remains challenging because disability is context-dependent. We introduce AccessLens, an end-to-end system designed to identify inaccessible interfaces in daily objects, and recommend 3D-printable augmentations for accessibility enhancement. Our approach involves training a detector using the novel AccessDB dataset designed to automatically recognize 21 distinct Inaccessibility Classes (e.g., bar-small and round-rotate) within 6 common object categories (e.g., handle and knob). AccessMeta serves as a robust way to build a comprehensive dictionary linking these accessibility classes to open-source 3D augmentation designs. Experiments demonstrate our detector's performance in detecting inaccessible objects.

Reconfigurable physical interfaces empower users to swiftly adapt to tailored design requirements or preferences. Shape-changing interfaces enable such reconfigurability, avoiding the cost of refabrication or part replacements. Nonetheless, reconfigurable interfaces are often bulky, expensive, or inaccessible. We propose a reversible shape-changing mechanism that enables reconfigurable 3D printed structures via translations and rotations of parts. We investigate fabrication techniques that enable reconfiguration using magnets and the thermoplasticity of heated polymer. Proposed interfaces achieve tunable haptic feedback and adjustment of different user affordances by reconfiguring input motions. The design space is demonstrated through applications in rehabilitation, embodied communication, accessibility, safety, and gaming.

Chiral kerf structures are formed by arranging chiral and coiled unit cells which allows for multi-dimensional and multi-scale shape configurations under mechanical loadings. In this study, we investigate how the mechanical properties of materials and microstructural topologies interact to control the flexibility, toughness, and load bearing of 3D-printed chiral kerf structures. We explore chiral kerf structures with two different kerf patterns, i.e., square and hexagon, and three coiling densities. We consider three materials, namely brittle Polylactic Acid (PLA), compliant thermoplastic polyurethane (TPU), and a ductile composite made by alternating PLA and TPU, referred to as a programmable composite. The chiral kerf structures undergo two deformation mechanisms when subjected to mechanical loadings. The first one results from reconfigurations of kerf cells such as uncoiling, rotation of cells, and cell packing, and the second mechanism arises from nonlinear and inelastic material responses. The use of brittle material limits cell reconfigurations before material failure, reducing the overall flexibility and toughness of kerf structures. While the compliant material enables full cell reconfigurations, it results in low load bearing. The use of PLA:TPU composite allows for cell reconfigurations and inelastic material response, enhancing flexibility and toughness while maintaining a relatively high load bearing. We demonstrate that stress distribution in kerf structures can be controlled by using multiple materials or coil densities. This strategy can delay failure and improve the toughness and load-bearing capabilities of kerf structures.

The widespread consumer-grade 3D printers and learning resources online enable novices to self-train in remote settings. While troubleshooting plays an essential part of 3D printing, the process remains challenging for many remote novices even with the help of well-developed online sources, such as online troubleshooting archives and online community help. We conducted a formative study with 76 active 3D printing users to learn how remote novices leverage online resources in troubleshooting and their challenges. We found that remote novices cannot fully utilize online resources. For example, the online archives statically provide general information, making it hard to search and relate their unique cases with existing descriptions. Online communities can potentially ease their struggles by providing more targeted suggestions, but a helper who can provide custom help is rather scarce, making it hard to obtain timely assistance. We propose 3DPFIX, an interactive 3D troubleshooting system powered by the pipeline to facilitate Human-AI Collaboration, designed to improve novices' 3D printing experiences and thus help them easily accumulate their domain knowledge. We built 3DPFIX that supports automated diagnosis and solution-seeking. 3DPFIX was built upon shared dialogues about failure cases from Q&A discourses accumulated in online communities. We leverage social annotations (i.e., comments) to build an annotated failure image dataset for AI classifiers and extract a solution pool. Our summative study revealed that using 3DPFIX helped participants spend significantly less effort in diagnosing failures and finding a more accurate solution than relying on their common practice. We also found that 3DPFIX users learn about 3D printing domain-specific knowledge. We discuss the implications of leveraging community-driven data in developing future Human-AI Collaboration designs.

Unmaking is a counterpart to making and creating new things that has emerged as a concept of interest in diverse parts of the HCI community. Unmaking has been posed as an ally to sustainability, encouraging designers to foreground issues relating to reuse, repair, obsolescence, degradation, and decay early in their design process. As a follow-up to the 2022 Unmaking@CHI workshop, this workshop will bring together researchers and practitioners interested in unmaking as it relates to sustainability and will focus primarily on exploring the role of unmaking in material practices, drawing upon the growing body of unmaking theory to explore future research opportunities for designing physical things with sustainable materials that are transient, degradable, and intentionally unmake-able. In addition to considering the pragmatics of what and how to unmake, we seek to articulate the relationships among unmaking and other related emerging themes and sustainable material practices – including biodegradation, designing with more-than-human agencies, reuse, and repair – and propose guidelines for designing for the unmaking of physical artifacts that are sustainable, equitable, and respectful of all entities involved.

Within the domain of fabrication, the recent strides in Fused Deposition Modeling (FDM) have sparked growing interest in its sustainability. In this work, we analyze the contemporary life cycle of polymers consumed in FDM, a common and accessible fabrication technique. Then we outline the points of design intervention to reduce wasted polymers in fabrication. Specifically, we discuss the design intervention of Filament Wiring, a set of hybrid craft techniques to promote sustainable prototyping and robust applications by highlighting left-over filaments. Our techniques aim to enhance the understanding of filaments as a unique material for hybrid fabrication, fostering creativity. Through our computational design system, end users can generate 3D printable frames, for exploring the possibilities of filament-based fabrication beyond 3D printing. We hope to provoke thought about filament as its own form of material, having capabilities to be made, unmade, and remade repeatedly into various artifacts. With this outlook, we discuss future research avenues, and urge makers and practitioners to value material in any form, quantity, or stage of its life cycle.

The majority of errors in making processes can be tracked back to errors in dimensional specifications. While technical aspects of measurement, such as precision and speed have been extensively studied in metrology, the user aspects of measurement received significantly less attention. While little research exists that specifically addresses the user aspects of handling dimensions, various systems have been built that embed new interactive modalities, processes, and techniques which significantly impact how users deal with dimensions or conduct measurements. However, these features are mostly hidden in larger system contributions. To uncover and articulate these techniques, we conducted a holistic literature survey on measurement practices in crafting techniques and systems for rapid prototyping. Based on this survey, we contribute 10 measurement patterns, which describe reusable elements and solutions for common difficulties when dealing with dimensions throughout workflows for making physical artifacts.

This study presents the use of a 3D printing method to create kerf structures that can be formed into complex geometries. Kerfing is a subtractive manufacturing method to create flexible surfaces out of stiff planar materials such as metal or wood sheets by removing portions of the materials. The kerf structures are characterized by the kerf pattern, such as square interlocked Archimedean spiral and hexagon spiral domain, cell size, and cut density. By controlling the kerf pattern, spatial density, cell size, and material, the local properties of the structure can be controlled and optimized to achieve the desired local flexibility while minimizing the stresses developed in the kerf structure. Since subtractive manufacturing limits the patterns and materials that can be considered in kerf structures, FDM 3D printing is explored to fabricate kerf structures using polymers, such as Polylactic acid (PLA) and Thermoplastic polyurethane (TPU), where it is possible to vary microstructural topology and materials within the kerf structures. 3D printing enables the combination of the two different polymers and tuning printing factors to create multifunctional kerf structures. The multifunctional kerf structures can then be actuated using non-mechanical stimulations, such as thermal, to shape them into complex geometries.

The rising incidence of fires calls for advanced training methodologies that surpass the limitations of traditional firefighter training, both in scale and scope. Virtual Reality (VR) emerges as a potent solution, offering a wide range of realistic scenarios and cost-effective, safe training environments. This paper presents a novel VR-based training platform tailored for firefighters, which leverages Unity 3D and state-of-the-art fire simulation techniques to deliver high-fidelity experiences that closely mimic real-world dynamics. Trainees engage in an immersive VR setting where they experience full autonomy and multisensory feedback, heightening the educational impact through a procedural fire spreading mechanism that emulates actual fire behavior. Our system excels in providing a comprehensive framework, modular design for customizable scenarios, and integration of varied training modules to prepare trainees for an array of firefighting emergencies. Future work aims to enhance realism through advanced features such as Flashover and Backdraft simulation, real-time environmental controls for trainers, team-based exercises with human-agent interaction, etc. The paper concludes by emphasizing the platform’s alignment with the set design standards for VR-based firefighter training and outlines prospective user testing with professional firefighters to further refine the VR experience.

Instance detection (InsDet) is a long-lasting problem in robotics and computer vision, aiming to detect object instances (predefined by some visual examples) in a cluttered scene. Despite its practical significance, its advancement is overshadowed by Object Detection, which aims to detect objects belonging to some predefined classes. One major reason is that current InsDet datasets are too small in scale by today's standards. For example, the popular InsDet dataset GMU (published in 2016) has only 23 instances, far less than COCO (80 classes), a well-known object detection dataset published in 2014. We are motivated to introduce a new InsDet dataset and protocol. First, we define a realistic setup for InsDet: training data consists of multi-view instance captures, along with diverse scene images allowing synthesizing training images by pasting instance images on them with free box annotations. Second, we release a real-world database, which contains multi-view capture of 100 object instances, and high-resolution (6k x 8k) testing images. Third, we extensively study baseline methods for InsDet on our dataset, analyze their performance and suggest future work. Somewhat surprisingly, using the off-the-shelf class-agnostic segmentation model (Segment Anything Model, SAM) and the self-supervised feature representation DINOv2 performs the best, achieving >10 AP better than end-to-end trained InsDet models that repurpose object detectors (e.g., FasterRCNN and RetinaNet).

Polylactic acid (PLA) filament is widely used for desktop 3D printing purposes due to its exceptional mechanical properties such as high strength; however, its brittleness restricts its use for producing flexible objects. Thermoplastic polyurethane (TPU) filament which is also widely used for desktop 3D printing, on the other hand, is flexible and commonly used in printing compliant objects with relatively low load-bearing performance. This study investigates the ability to tune the mechanical properties of specimens that are printed using programmable filaments composed of PLA and TPU filaments with different volume ratios of PLA and TPU. Two types of PLA and TPU filament arrangements, i.e., series and parallel, are considered. The PLA:TPU programmable filaments are used to print dogbone specimens for tensile testing. In printing the dogbone specimens, the raster angle is varied, i.e., 0, 45, and 90° with respect to the transverse direction of the specimen. To examine their mechanical behaviors based on different PLA and TPU filament arrangements, compositions, and raster angles, tensile tests are conducted on both programmable filaments and dogbone specimens. This study demonstrates the ability to tune the mechanical properties of printed objects by designing programmable filaments and varying raster angles during printing.

The increase of distributed embedded systems has enabled pervasive sensing, actuation, and information displays across buildings and surrounding environments, yet also entreats huge cost expenditure for energy and human labor for maintenance. Our daily interactions, from opening a window to closing a drawer to twisting a doorknob, are great potential sources of energy but are often neglected. Existing commercial devices to harvest energy from these ambient sources are unaffordable, and DIY solutions are left with inaccessibility for non-experts preventing fully imbuing daily innovations in end-users. We present E3D, an end-to-end fabrication toolkit to customize self-powered smart devices at low cost. We contribute to a taxonomy of everyday kinetic activities that are potential sources of energy, a library of parametric mechanisms to harvest energy from manual operations of kinetic objects, and a holistic design system for end-user developers to capture design requirements by demonstrations then customize augmentation devices to harvest energy that meets unique lifestyle.

By varying the arrangements of microstructures and constituents in bodies allows for effectively tuning the mechanical performance of composite structures, i.e., increasing stiffness and toughness, maximizing energy dissipation, etc. We explore the use of programmable filaments consisting of soft and hard constituents in tuning the mechanical properties of materials used for kerf structures. Kerf structures consist of connected microstructural topology with spatially controlled flexibility enabling for easy macroscopic and microscopic shape reconfigurations. Using additive manufacturing (AM), we first fabricate programmable filaments out of Polylactic acid (PLA) and Thermoplastic polyurethane (TPU) polymers which we will then use to print kerf structures. We demonstrate the use of kerf structures with programmable filaments to enhance toughness and energy dissipation, which is driven by microstructural shape reconfigurations of kerf topology and hierarchical failure of the programmable filaments.