RT-S: SURFACE RICH TRANSCRIPTION SCORING, METHODOLOGY, AND INITIAL RESULTS

In this paper we present a methodoly for the scoring of punctuation annotated texts, as well as a preliminary system to perform the task. We modify SCLITE's scoring method- ology to support scoring of punctuation. Using this method- ology, we show that the error rate of an initial automatic sys- tem is comparable to annotator inconsistency. However, the use of multiple references allows us to differentiate between human inconsistencies and system errors.