About MoSE
Intrinsic disorder regions (IDRs) are prevalent in proteins and play critical roles in a wide range of biological processes. Precise identification of these regions is crucial for advancing our understanding of protein structure and biological function. Extracting informative residue-level features is a fundamental prerequisite for developing effective computational models to predict intrinsically disordered regions (IDRs). Previous studies have typically employed a variety of amino acid features as semantic cues to represent protein sequences. However, different feature extraction strategies often yield distinct semantic representations for the same residues within a protein sequence, thereby influencing the contextual semantic modelling of the entire sequence. Effectively capturing the dynamic semantic relationships between residues and their contextual variations, and appropriately integrating these semantics into predictive models, remains a significant challenge. In this study, we propose a novel framework, Mixture of Semantic Experts (MoSE), that adaptively models protein sequence semantics using context-aware cues, thereby improving the prediction of intrinsic disorder regions (IDRs). MoSE leverages the Mixture of Experts (MoE) paradigm to dynamically capture semantic variation and consists of two main components: Multi-Semantic Experts and a Multi-Semantic Router. The Multi-Semantic Experts are designed to specialize in learning diverse semantic representations of protein sequences, enabling sample-adaptive retention of local contextual information. The Multi-Semantic Router, guided by a multi-feature fusion gate, determines which expert should be activated at each residue, thereby automatically selecting the most relevant semantic information in a context-sensitive manner. Comprehensive evaluation on five independent benchmark datasets reveals that MoSE not only improves the semantic representation of amino acid features but also outperforms competing models in predicting intrinsically disordered regions.
Overview of the MoSE framework. (a) Residue Feature Extraction Module: this stage involves generating residue-level feature vectors using diverse extraction strategies (e.g., PSSM, ESM-2, DR-BERT, etc.). These features serve as inputs to different expert networks in the subsequent module. (b) Mixture of Semantic Experts Module: this component consists of two submodules. The first is the Multi-Semantic Experts, where different semantic expert networks encode residue-level representations derived from various feature types. The second is the Multi-Semantic Router, which is guided by a multi-feature fusion gate to determine which expert should be activated at each residue position, thereby automatically selecting the most contextually relevant semantic information. (c) Output Layer: the final output layer is utilized to perform residue-level identification of IDRs.