Patterns of User Engagement in the BRAVE Study (Preprint)

BACKGROUND Many American Indian and Alaska Native (AI/AN or Native) communities express concern about high rates of suicide and poor mental health. Technology-based health interventions that nurture resilience, coping skills, connectedness and help-seeking skills may be an effective strategy for promoting health and wellbeing for AI/AN youth. OBJECTIVE This study explored system data from the BRAVE intervention to determine patterns of user engagement. METHODS The BRAVE study included 1,030 AI/AN teens and young adults nationwide (15-24 years old). The message series in the BRAVE and STEM study arms included 3–5 text messages per week, featuring 1 role model video and 1 image per week. Messages were sent out via Mobile Commons, a mobile messaging provider that supports text, picture, and video SMS. This study utilized two sets of data. The first dataset included 23,004 records of messages sent teens and young adults and 874 records of messages clicked with time stamp and content. The second dataset was created by aggregating the clicks in the first dataset to calculate a total number of clicks for each user. RESULTS Of the 509 participants in the original BRAVE analysis, 270 had sufficient data to analyze user engagement, with at least one trackable click on a study text message. Of these, 68% were female (n = 184), 19% were male (n = 50), and 13% selected another gender category (n = 36). The average participant was 20.6 years-old, with a minimum and maximum of 15 and 26 years. Most participants had relatively low engagement measured by the number of clicks (median 2, mean 3.4), while others clicked message content as many as 49 times. Users engaged most frequently with the YouTube-based content (viewing one of 7 role model videos), with 64.8% of total clicks coming from the role model videos, and earlier episodes seeing the highest number of clicks. Most baseline psychosocial measures were not significantly associated with the number of links clicked. However, help-seeking behavior was highly significant (P<.001), with a rate ratio of 0.82 (0.73, 0.92), indicating that each one-unit increase in help-seeking score at baseline was associated with an 18% decrease in the expected number of study clicks. CONCLUSIONS This is the first study to set initial standards for assessing user engagement in an mHealth intervention. Our work underscores the feasibility of exploring the impact of engagement on intended outcomes, allowing for more precise exploration of the dose-response relationship that may be realized through these low-touch interventions that offer promising potential for reaching high numbers of program participants. CLINICALTRIAL 1384639