An energy-based comparative analysis of common approaches to text classification in the Legal domain
CoRR(2023)
摘要
Most Machine Learning research evaluates the best solutions in terms of
performance. However, in the race for the best performing model, many important
aspects are often overlooked when, on the contrary, they should be carefully
considered. In fact, sometimes the gaps in performance between different
approaches are neglectable, whereas factors such as production costs, energy
consumption, and carbon footprint must take into consideration. Large Language
Models (LLMs) are extensively adopted to address NLP problems in academia and
industry. In this work, we present a detailed quantitative comparison of LLM
and traditional approaches (e.g. SVM) on the LexGLUE benchmark, which takes
into account both performance (standard indices) and alternative metrics such
as timing, power consumption and cost, in a word: the carbon-footprint. In our
analysis, we considered the prototyping phase (model selection by
training-validation-test iterations) and in-production phases separately, since
they follow different implementation procedures and also require different
resources. The results indicate that very often, the simplest algorithms
achieve performance very close to that of large LLMs but with very low power
consumption and lower resource demands. The results obtained could suggest
companies to include additional evaluations in the choice of Machine Learning
(ML) solutions.
更多查看译文
关键词
classification,legal domain,text
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要