Users’ Responses to a Machine-Learning Decision Support Model: A Randomized Controlled Trial for Prostate-Specific Antigen Screening

Abstract Background: Although a shared decision-making (SDM) process integrates patient values and evidence-based medicine, patients’ anxiety and decision conflicts remain. Thus, we propose a new decision-making model integrating a machine-learning algorithm to investigate its feasibility for reducing anxiety, decision conflicts, and increasing satisfaction after making a decision.Methods: We enrolled participants willing to undergo the SDM process for a prostate-specific antigen (PSA) blood test and obtained data including age, PSA knowledge, if they have a friend with prostate cancer, perceptive risk of prostate cancer, International Prostate Symptom Score and Importance for Physiological and Psychological Impact in PSA Testing scores, personal values, and their final decisions, including “Accept” PSA blood test or “Not now,” to build the dataset for training the following machine-learning models: multilayer perceptron neural network, random forest (RF), extreme gradient boosting, support vector machine, and deep learning neural network. Uniform parameter tuning and model comparison were implemented. The best model was used for a randomized controlled trial (RCT), in which we measured the effects of personalized suggestions generated by the machine-learning model on anxiety, decision satisfaction, and decision conflicts.Results: RF was the best algorithm for building models with our dataset from 507 subjects (mean AUC: 0.8801, mean ACC: 0.8313, Max ACC: 0.8933). Therefore, we used the RF model for RCT with 185 and 182 subjects in the machine-learning suggestion group (MLSG) and control group (CG), respectively. The MLSG patients were calmer, more content, and less worrisome than those in the CG. They also experienced higher decision satisfaction and less decision conflict, including more decision support, advice, assurance of decision, ease of decision-making, and adherence to decision. Moreover, participants who were suggested “Accept” by the model were more likely to make “Accept” their final decision than the CG participants (50.75% vs 24.18%, χ2 = 16.07, p < 0.000). The “Not now” suggestion followed a similar trend.Conclusions: A highly accurate machine-learning model was constructed using our methods. Personalized suggestions generated from this model yielded increased satisfaction and reduced anxiety and decision conflict. Patients tended to take machine-learning suggestions as their final decision.Trial name: Shared Decision Making: Decision Tree and Artificial Neural Network Assisted Decision Aid for PSA ScreeningTrial registration: ChiCTR, ChiCTR2000034126. Registered 25 June 2020 – Retrospectively registered, http://www.chictr.org.cn/ChiCTR2000034126