Enzyme Classification Using Transformer-Based Sequence Embeddings

Enzyme function classification plays a critical role in understanding biological processes, drug discovery, and protein annotation. This paper presents a computational pipeline that leverages ESM2, a transformer-based protein language model to generate contextual embeddings from raw amino acid sequences. We explore strategies to address class imbalance and evaluate the embeddings on two supervised learning architectures: a MLP and a deeper custom neural network. Our observations demonstrate that the MLP model with oversampling achieves the best performance, achieving a test accuracy of 93.5% and macro F1-score of 91% outperforming deeper architectures and class-weighted loss. Our findings suggest that even embeddings generated from a lightweight transformer combined with effective imbalance handling techniques can provide an efficient solution for enzyme function classification.