By capturing local and remote dependencies, deep learning architectures such as CNN and Transformers have significantly advanced biological sequence modeling. However, their application in biological environments is limited by high computational requirements and the need for large data sets. CNNs effectively detect local sequence patterns with subsubsubstantially scaling, while Transformers utilizes self to model global interactions, but requires secondary scaling, making them computationally expensive. Hybrid models (such as magnetic fields) integrate CNNs and transformers to balance local and international environmental modeling, but they still face scalability issues. Large-scale transformer models including Alphafold2 and ESM3 have made breakthroughs in protein structure prediction and sequence function models. However, their dependence on broad parameter scaling limits their efficiency in biological systems where data availability is often limited. This highlights the need for more computationally efficient methods to accurately model sequence-to-function relationships.
To overcome these challenges, uptoxicity (interactions between intrasequence mutations) provides a structured mathematical framework for biological sequence modeling. Polylinear polynomials can represent these interactions, providing a principled way to understand sequence functional relationships. The state space model (SSM) is naturally consistent with this polynomial structure, using hidden dimensions to approximate the epistaxis effect. Unlike transformers, SSM uses fast Fourier transform (FFT) convolution to effectively model global dependencies while maintaining subquadratic scaling. In addition, through adaptive feature selection, integrating closed-form depth convolution can enhance local feature extraction and expressiveness. This hybrid approach can balance computational efficiency with interpretability, making it a promising alternative to transformer-based architectures for biological sequence modeling.
Researchers at institutions including MIT, Harvard University and Carnegie Mellon introduced secondary sequence modeling architectures designed for biological applications. Lyra integrates SSM to capture remote dependencies with expected closed convolution for local feature extraction, enabling efficient O(n log n) scaling. It effectively simulates synonymous interactions and achieves state-of-the-art performance in over 100 biological tasks, including protein fitness prediction, RNA functional analysis and CRISPR guide design. Lyra has much fewer parameters, as small as 120,000 times smaller than existing models, while reasoning is 64.18 times faster, democratizing access to advanced biological sequence modeling.
Lyra consists of two key components: the expected closed convolution (PGC) block and a state space layer with deep convolution (S4D). The model has approximately 55,000 parameters, including two PGC blocks for capturing local dependencies, followed by S4D layers for modeling long-distance interactions. The PGC inputs the sequence by projecting it to the intermediate dimension, applying depth 1D convolution and linear projection, and multiplying elements by recombining features. S4D uses diagonal state space model to compute convolution kernels using matrices A, B, and C, effectively capture sequence-range dependencies through weighted exponential terms, and enhances Lyra’s ability to effectively model biological data.
Lyra is a sequence modeling architecture designed to effectively capture local and long-term dependencies in biological sequences. It integrates PGC for local modeling and diagonal S4D for global interactions. Lyra uses polynomial expression to approximate complex epithelial interactions, and transformer-based models outperform transformer-based models in tasks such as protein fitness landscape prediction and depth mutation scanning. It achieves the latest accuracy in a variety of protein and nucleic acid modeling applications, including barrier prediction, mutation impact analysis and RNA-dependent RNA polymerase detection, while maintaining significantly smaller parameter counts and lower computational costs than existing large models.
In summary, Lyra introduces a sub-subsid structure for biological sequence modeling, using SSM to effectively approximate multilinear polynomial functions. This allows excellent modeling of epithelial interactions while significantly reducing computational requirements. By integrating PGCs for local feature extraction, Lyra achieves state-of-the-art performance in over 100 biological tasks, including protein fitness prediction, RNA analysis, and CRISPR guide design. It performs better than large base models, with much fewer parameters and faster inference, and requires only one or two GPUs to train in a few hours. Lyra’s efficiency democratizes access to advanced biomodeling through treatment, pathogen monitoring and biomanufacturing applications.
Check Paper. All credits for this study are to the researchers on the project. Also, please keep an eye on us twitter And don’t forget to join us 85k+ ml reddit.
Lyra Post: Calculation of valid subquadratic architecture for biological sequence modeling first appeared on Marktechpost.
