The Right Tool for the Job: Matching Model and Instance Complexities
ACL, pp. 6640-6651, 2020.
Experiments with BERT-large on five text classification and natural language inference datasets yield substantially faster inference compared to the standard approach, up to 80% faster while maintaining similar performance
As NLP models become larger, executing a trained model requires significant computational resources incurring monetary and environmental costs. To better respect a given inference budget, we propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" from neural network ...More
PPT (Upload PPT)