Standing
The Science paper reports a generated fluorescent protein at 58 percent sequence identity to known fluorescent proteins, which the authors frame as an estimate equivalent to a large evolutionary distance rather than a literal simulation.
Protein language model · EvolutionaryScale · 2025
ESM3
A generative masked model over three parallel tracks. A geometric tokenizer turns local backbone environments into discrete codes; a shared transformer predicts logits for every track; iterative unmasking fills the positions you left open.
In plain language
One model that reads and writes three things at once: the sequence, the shape, and what the protein does. You hand it a partly filled form and it completes the parts you left blank.
One way to picture it
A crossword where the across clues are amino acids, the down clues are geometry, and the theme is the function. Fill in any of them and the rest gets easier.
Commonly misread as
The headline results come from the largest model. The openly available checkpoint is the small one, and it will not reproduce them.
How it is put together
Select a component to read itMultimodal tokens
Per-track logits
Try it
Complete a partly masked proteinfrom esm.models.esm3 import ESM3
from esm.sdk.api import ESMProtein, GenerationConfig
model = ESM3.from_pretrained("esm3_sm_open_v1")
# underscores are the positions you want the model to decide
prompt = ESMProtein(sequence="MKTAYIAKQRQISFVK___________VLDRHDL")
result = model.generate(
prompt,
GenerationConfig(track="sequence", num_steps=8, temperature=0.7),
)
print(result.sequence)These snippets have not been executed here. Versions move; check the model card before trusting a line of it.
Distinctions
- Structure is a token track, not an output head bolted onto a sequence model.
- A prompt can mix modalities: fix a motif, name a function, leave the rest masked.
- Generation order is chosen by the model rather than running left to right.
Limits
Results reported for the largest model should not be attributed to the small open checkpoint. The evolutionary-distance framing is an estimate, and the open weights are the smallest member of the family.
Weights and code
Checked against the registry, not from memory| Repository | Size | Licence | Note |
|---|---|---|---|
| biohub/esm3-sm-open-v1 | 1.4B | Non-commercial community licence | Formerly published under EvolutionaryScale |
Sources
Each number above comes from one of theseHayes et al. 2025, Science. Simulating 500 million years of evolution with a language modelPeer reviewedESM3 open weightsModel card
Same task, other answers
Protein language modelESM-2
Read enough proteins and structure falls out of the reading.
A bidirectional transformer trained to reconstruct masked amino acids from unaligned sequences.
Explore the architectureESM C
The representation line, scaled further.
A transformer encoder with pre-layer-norm, rotary embeddings and SwiGLU activations, trained on sequences from UniRef, MGnify and the Joint Genome Institute clustered at 70 percent identity.
Read the entryProtT5 and ProtBERT
The encoders that were there first.
A family of transformers trained on UniRef and BFD with the objectives of their text counterparts.
Read the entry