MI-PEFT: Mixture-of-Experts Integrated Parameter-Efficient Fine-Tuning Protein Language Models Improves Acidophilic Proteins Classification
Honghan Shen
Abstract
Acidophilic proteins that remain stable and functional under highly acidic conditions,
are important for industrial biocatalysis, acid-related bioprocessing, and the discovery of acidstable enzymes. However, their identification relies heavily on time-consuming experimental
screening methods. With the rapid growth of protein sequence databases, the need for
computational identification methods that are both accurate and efficient has become stronger.
The emergence of protein language models (PLMs) has significantly improved the sequence
representation of downstream biological prediction tasks. This paper proposes MI-PEFT, a
mixture-of-experts integrated parameter-efficient fine-tuning framework. Built on the ESM C600M backbone, the framework incorporates LoRA-based PEFT methods and a
DeepSeekMoE-based classification head to resolve the limitations of PEFT and significantly
improve computational efficiency. Notably, this task is characterized by a significant class
imbalance in the dataset, making high specificity particularly challenging. The experimental
results demonstrate that MI-PEFT on PLMs, especially C
3A, serves as an efficient tool for
identifying acidophilic proteins and a constrained pathway that helps resolve class-imbalance
by preserving the pretrained representations.