HOW TO DESIGN A SUSTAINABLY ACCURATE AND FAST MODEL FOR TEXT CLASSIFICATION TASKS A PREPRINT MoyanMei Course5 AI Labs Toronto, Canada moyan.mei@course5i.com September 10, 2020 ABSTRACT Empirical studies have shown that large-scale pre-trained language models such as BERT (Bidirec- tional Encoder Representations from Transformers) bring significant improvements. However, they are often computationally expensive in many practical scenarios, as such heavy models are not easily implemented with limited resources. Improving the efficiency of the model while maintaining its performance is then a key challenge. In this paper, I provide two "sustainable" solutions for Course5’s text classification task. "Sustainable" is defined here as (1) easy to plug into any PLM (2) lower computational complexi

Join for free to read