Development, Deployment, and Evaluation of DyMand: Wearable AI for In-Situ Detection and Capture of Couples’ Dyadic Interactions in Chronic Disease Management

Dyadic interactions of couples are of interest as they provide insight into relationship quality and chronic disease management. Existing sensing systems primarily infer social structure within groups retrospectively, or capture couples’ interactions at random or scheduled times, which could miss meaningful interpersonal interactions between partners. In this work, we developed a smartwatch-based algorithm that detects interaction moments between partners and enables interaction-aware data capture in everyday settings. It uses the Bluetooth signal strength between two smartwatches, each worn by one partner, and a voice activity detection machine-learning model that we trained, to infer that the partners are interacting and then trigger data collection. We operationalized this sensing approach by developing, deploying, and evaluating DyMand, an open-source smartwatch and smartphone system that integrates multimodal sensor and self-report data collection in situ. We deployed the DyMand system in a 7-day field study and collected and analyzed data about social support and diabetes discussions from 13 (N=26) Swiss-based heterosexual couples managing diabetes mellitus type 2 of one partner. Our system evaluation showed that 77.6% of algorithm-triggered recordings contained partners’ conversation moments compared to 43.8% for scheduled triggers, and that DyMand was easy to use. Preliminary insights into disease management behavior revealed that 5.2% of couples’ conversation moments contained social support, and partners discussed diabetes management 2.4% of the time based on the audio data analyses. In contrast, the self-report data showed that patients received support in 67.7% of couples’ interactions and caregivers provided support in 71.8% of couples’ interactions. DyMand enables insights into dyadic behavior in-situ, and could be used by social, clinical, or health psychology researchers to understand the social dynamics of couples that are managing chronic diseases in everyday life to inform the development and delivery of behavioral interventions.