Crowdsourcing data collection of the retail tobacco environment: case study comparing data from crowdsourced workers to trained data collectors.

Objective To assess whether crowdsourcing is a viable option for conducting surveillance of point of sale (POS) tobacco marketing practices. Methods We posted jobs to an online crowdsourcing platform to audit 194 Florida licensed tobacco retailers over a 3-week period. During the same period, trained data collectors conducted audits at the same retail locations. Data were collected on cigarette advertising, cigarette promotions and product availability (electronic cigarettes, snus and dissolvables). We compared data collected by crowdsourced workers and trained staff and computed frequencies, percent agreement and inter-rater reliability. Photographs of e-cigarettes and exterior cigarette advertisements submitted by crowdsourced workers were used to validate responses. Results Inter-rater reliability between crowdsourced and trained data collectors was moderate to high for coding exterior cigarette advertisements, product availability and some tobacco promotions, but poor to fair when coding presence of sales and interior cigarette advertisements. Photos submitted by crowdsourced workers confirmed e-cigarette availability that was missed by trained data collectors in three stores. Conclusions Crowdsourcing may be a promising form of data collection for some POS tobacco measures. Future studies should examine the cost-effectiveness of crowdsourcing compared with traditional trained data collectors and assess which POS measures are most amenable to crowdsourcing.