TEL'M: Test and Evaluation of Language Models
CoRR(2024)
Abstract
Language Models have demonstrated remarkable capabilities on some tasks while
failing dramatically on others. The situation has generated considerable
interest in understanding and comparing the capabilities of various Language
Models (LMs) but those efforts have been largely ad hoc with results that are
often little more than anecdotal. This is in stark contrast with testing and
evaluation processes used in healthcare, radar signal processing, and other
defense areas. In this paper, we describe Test and Evaluation of Language
Models (TEL'M) as a principled approach for assessing the value of current and
future LMs focused on high-value commercial, government and national security
applications. We believe that this methodology could be applied to other
Artificial Intelligence (AI) technologies as part of the larger goal of
"industrializing" AI.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined