An investigation of mHealth juggernauts: efficacy and conflicts of interest for Headspace and Calm (Preprint)

BACKGROUND Although there are thousands of mental health (MH) apps, two apps, Headspace and Calm, are responsible for the majority of downloads and active users. These two mindfulness and meditation apps have reached tens of millions of active users. To guide consumers, clinicians, and researchers, we performed a systematic review of randomized controlled trials (RCTs) of Headspace and Calm. OBJECTIVE Our study aimed to evaluate intervention efficacy, risk of bias and conflict of interest in the evidence base for Headspace and Calm, the two most popular MH apps. METHODS To identify studies, we searched academic databases (Google Scholar, MEDLINE, and PsycINFO) and the websites of Headspace and Calm. For each study, we coded a) study characteristics (e.g., participants, sample size, outcome measures), b) intervention characteristics (e.g., free vs. paid version of the app, intended frequency of app usage), c) all study outcome results, d) Cochrane risk of bias variables, and e) conflict of interest variables (e.g. presence or absence of a pre-registration, presence or absence of a conflict-of-interest statement involving the company). RESULTS We identified 14 RCTs of Headspace and 1 RCT of Calm. 93% (13/14) of Headspace studies and 100% of Calm studies (1/1) recruited participants from a non-clinical population. Studies commonly measured mindfulness, well-being, stress, depressive symptoms, and anxiety symptoms. Headspace use improved depression in 75% of studies that evaluated it as an outcome. Findings were mixed for mindfulness, well-being, stress, and anxiety, but at least 40% of studies showed improvement for each of these outcomes respectively. Studies were generally underpowered to detect “small” or “medium” effect sizes. 50% (7/14) of Headspace studies and 0% (0/1) of Calm studies reported a conflict of interest that involved Headspace or Calm. The most common conflict of interest was the app company providing premium app access for free for participants, and notably, 14.28% (2/14) of Headspace studies had Headspace employee involvement in study design, execution, and data analysis. Only 36% (5/14) of Headspace studies were pre-registered, and the one Calm study was not pre-registered. CONCLUSIONS The empirical research on Headspace appears to be promising, whereas there is an absence of randomized trials on Calm. Special attention should be paid to study findings in the context of potential conflicts of interest related to for-profit companies. The determination of apps as high quality should involve judgment of effectiveness and an in-depth evaluation of investigator conflict of interest.