The Right Tool for the Job: Matching Model and Instance Complexities

ACL, pp. 6640-6651, 2020.

Cited by: 12|Views97
EI
Weibo:
Experiments with BERT-large on five text classification and natural language inference datasets yield substantially faster inference compared to the standard approach, up to 80% faster while maintaining similar performance

Abstract:

As NLP models become larger, executing a trained model requires significant computational resources incurring monetary and environmental costs. To better respect a given inference budget, we propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" from neural network ...More

Code:

Data:

0
Your rating :
0

 

Tags
Comments