Mamba (deep learning)

Mamba is a neural network architecture for sequences, introduced by Albert Gu and Tri Dao in a paper posted on December 1, 2023. It is a selective state space model: the network carries a compact state along the sequence, one step at a time, and decides at each step how much of the new input to write into that state and how much of the old state to keep. The cost of processing a sequence grows in proportion to its length, where the cost of the attention layers of a transformer grows with the square of it.^[1] The design has been applied to language, audio, genomics and, in 2025, to the current signals of solid-state nanopores.

A transformer reads a sequence by letting every position look at every earlier position. The result is powerful, but the work grows with the square of the length and the memory kept for generation grows with the length itself. The older alternative is the recurrent network, which passes a state forward and so keeps its cost proportional to length, at the price of being hard to train in parallel and of forgetting.

State space models come from control theory. A hidden state h evolves under the input x and is read out as y, and after discretization with a time step Δ the model is the recurrence

with $\overline{A}=\exp (\Delta A)$. When A, B, C and Δ are constants the model is linear and time-invariant, and the whole operation can be computed as one long convolution, which is fast to train. In 2021 Gu, Goel and Ré found a structure for the matrix A that made such models stable and efficient; their S4 was the first model to solve the Path-X task of the Long Range Arena benchmark, which is a sequence of 16,384 steps.^[2]

A time-invariant model applies the same dynamics to every input, so it cannot decide that one token matters and the next does not. Mamba removes the restriction by making Δ, B and C functions of the current input. The model can now close its state to filler and open it to a token worth keeping. The price is that the convolution shortcut is lost, since the dynamics change at every step. The authors replaced it with a parallel scan written to keep the expanded state in the fast on-chip memory of the GPU, and not write it to the slower main memory, which they call a hardware-aware algorithm.^[1]

The block that results has an input projection, a short convolution, the selective state space layer and a gate. It has no attention layer and no feed-forward layer, and it is stacked to make the network. The paper trained models of up to about 3 billion parameters and reported roughly five times the generation throughput of a transformer, linear scaling with length, and results on language, audio and genomics; its 3-billion-parameter model outperformed transformers of the same size and matched those twice its size.^[1]

Dao and Gu showed in 2024 that state space models and attention are two views of one family of structured matrices, and built Mamba-2 on the connection, with a core layer they reported as 2 to 8 times faster than Mamba's.^[3] Other groups mixed the two kinds of layer. AI21 Labs' Jamba, of March 2024, interleaves transformer and Mamba layers with a mixture of experts, and the released model has 52 billion parameters in all, about 12 billion of them active for any token, with a context window of up to 256,000 tokens.^[4]

A current trace is a long sequence whose events vary in length, which suits a model that carries a state. A conference paper of 2025 from Southeast University in Nanjing paired a Mamba layer with an attention layer in each encoder to classify the events of a solid-state nanopore into eight classes and reported 97.1 percent accuracy. The paper did not name the method; the founder of Molecular Reality Corporation did (see Mambapore).

Kent Kemmish introduced the paper to members of EQBio-OS in March 2026 and described the architecture as one "which combines the best features of both Recursive[sic] Neural Networks and Transformers".^[5] The recurrent half of the description is accurate: Mamba generates one token at a time as a recurrence, and trains on whole sequences in parallel, as a transformer does.

The company's plans give the architecture a place. In its critique of its own proposals, published under the name of Keltar, the essay on The Nanopore World Model proposes that a Mamba state-space model carry the running belief, the compact summary of what a device has seen so far. The same plans combine Mamba, attention and a Conformer in one backbone.^[6] See Machine learning at Molecular Reality.

• Machine learning on nanopore signals
• Joint embedding predictive architecture
• World model