Mamba (deep learning)

Mamba is a neural network architecture for sequences, introduced by Albert Gu and Tri Dao in a paper posted on December 1, 2023. It is a selective state space model: the network carries a compact state along the sequence, one step at a time, and decides at each step how much of the new input to write into that state and how much of the old state to keep. The cost of processing a sequence grows in proportion to its length, where the cost of the attention layers of a transformer grows with the square of it.^[1] Researchers have since used it on language, audio and genomics and, in 2025, on the current signals of solid-state nanopores.

A transformer reads a sequence by letting every position look at every earlier position. That is powerful and expensive: the work grows with the square of the length, and the memory kept for generation grows with the length itself. A recurrent network passes a state forward instead and so costs only in proportion to length, but it trains poorly in parallel and forgets.

State space models come from control theory. A hidden state h evolves under the input x and is read out as y, and after discretization with a time step Δ the model is the recurrence

with $\overline{A}=\exp (\Delta A)$. When A, B, C and Δ are constants the model is linear and time-invariant, and the whole operation can be computed as one long convolution, which is fast to train. In 2021 Gu, Goel and Ré found a structure for the matrix A that made such models stable and efficient; their S4 was the first model to solve the Path-X task of the Long Range Arena benchmark, which is a sequence of 16,384 steps.^[2]

A time-invariant model applies the same dynamics to every input, so it cannot decide that one token matters and the next does not. Mamba removes the restriction by making Δ, B and C functions of the current input. The model can now close its state to filler and open it to a token worth keeping. The price is that the convolution shortcut is lost, since the dynamics change at every step. The authors replaced it with a parallel scan written to keep the expanded state in the fast on-chip memory of the GPU, and not write it to the slower main memory, which they call a hardware-aware algorithm.^[1]

The block that results has an input projection, a short convolution, the selective state space layer and a gate, and the network is a stack of them with no attention and no feed-forward layers between. The authors trained models of up to about 3 billion parameters. They reported roughly five times the generation throughput of a transformer and linear scaling with length, with results on language, audio and genomics; the 3-billion-parameter model outperformed transformers of the same size and matched those twice its size.^[1]

Dao and Gu showed in 2024 that state space models and attention are two views of one family of structured matrices, and built Mamba-2 on the connection, with a core layer they reported as 2 to 8 times faster than Mamba's.^[3] AI21 Labs' Jamba, of March 2024, mixes the two kinds of layer: it interleaves transformer and Mamba layers with a mixture of experts, and the released model has 52 billion parameters in all, about 12 billion of them active for any token, with a context window of up to 256,000 tokens.^[4]

A current trace is a long sequence whose events vary in length, which suits a model that carries a state. A conference paper of 2025 from Southeast University in Nanjing paired a Mamba layer with an attention layer in each encoder to classify the events of a solid-state nanopore into eight classes and reported 97.1 percent accuracy.^[5] The paper did not name the method; the founder of Molecular Reality Corporation did (see Mambapore).

Jade O'Neill, the company's AI lead, found the paper in March 2026. Kent Kemmish introduced it to members of EQBio-OS and described the architecture as one "which combines the best features of both Recursive[sic] Neural Networks and Transformers".^[6] The recurrent half of the description is accurate: Mamba generates one token at a time as a recurrence, and trains on whole sequences in parallel, as a transformer does.

The essay on The Nanopore World Model, the company's critique of its own proposals, published under the name of Keltar, proposes that a Mamba state-space model carry the running belief, the compact summary of what a device has seen so far. Other plans put Mamba, attention and a Conformer in a single backbone.^[7] See Machine learning at Molecular Reality.

• Machine learning on nanopore signals
• Joint embedding predictive architecture
• World model