Oncologist participation in pilot testing a crowdsourcing platform to build a survivorship care risk model.

e13568 Background: Crowdsourcing as a means of codifying the knowledge of many individuals can be an effective way to build models where data do not exist to inform these models. Risk for health complications during cancer survivorship is one area where there are limited large datasets that track detailed outcomes. While many survivorship care expert groups such as the National Comprehensive Cancer Network (NCCN) and Children’s Oncology Group (COG) Survivorship Guidelines Panels recognize the need for a stratified system of assigning survivorship care, criteria for assigning risk to individuals at the end of active cancer treatment are not well agreed upon. To build a model for risk of health complications during survivorship care, we created an online crowdsourcing platform called Follow-up Interactive Long-Term Expert Ranking (FILTER). With FILTER we invite oncologists from a range of backgrounds to determine which case, among two cases, is higher risk. This process is repeated many times by many oncologists to achieve a consensus rating for risk for each case. The purpose of this preliminary study was to understand the characteristics of oncologists who were willing to participate in pilot testing FILTER and the number of cases they were able to adjudicate. Methods: We released the FILTER application in November 2021 to members of the NCCN Survivorship Guidelines Panel, COG Survivorship Guidelines Panel, and Vanderbilt-Ingram Cancer Center oncologists. Through the registration process, we collected institution and specialty information about each expert to determine whether a diverse range of experts were contributing their experience to the risk model. Results: Out of over 100 oncologists who were invited to participate in pilot testing of FILTER, 29 users have signed up for an account and adjudicated at least one matchup. These experts came from 13 institutions and have adjudicated a total of 1665 matchups generating risk scores for 78 synthetic cases. The median number of matchups adjudicated per expert was 45. There were 15 medical oncologists, 3 radiation oncologists, a surgical oncologist, and 10 pediatric oncologists who participated in pilot testing. Conclusions: We succeeded in recruiting a limited number of experts to date from a diverse range of oncology specialties, institutions, and backgrounds to pilot test FILTER. However, recruitment has been inadequate to meet the needs of the project. We therefore will seek to recruit more oncologists to refine these scores and will use meetings of oncology groups such as ASCO to achieve this goal. With enough adjudications by oncologists, a reliable ranking of cases by risk score will allow us to create a model to inform a survivorship risk calculator. This calculator, which will be made publicly available once finalized and validated, will provide another resource to help triage cancer survivors to appropriate survivorship care needs.