Accuracy of AI-generated Captions With Collaborative Manual Corrections in Real-Time

Automatic Speech Recognition (ASR) is a cost-efficient and scalable tool to automate real-time captioning. Even though its overall quality has improved rapidly, generated transcripts can be inaccurate. While manual correction helps to increase transcription accuracy, this causes new real-time challenges, especially for live-streaming. Crowd-sourcing can make the high workload more manageable by distributing the work across multiple individuals. In this paper, we developed a prototype that enables humans to collaboratively correct AI-generated captions in real-time. We conducted an experiment with 40 participants to measure the accuracy of the created and manually corrected captions. The results show that manual corrections improved the overall text accuracy according to multiple metrics as well as overall qualitative analysis.