Standing
A widely downloaded structure-aware encoder, trained on predicted structures from the AlphaFold database as well as experimental ones.
Protein language model · Westlake · 2024
SaProt
A masked language model over a structure-aware vocabulary. Each position carries an amino acid combined with a Foldseek structural alphabet symbol, so sequence and local geometry share one token.
In plain language
A language model whose alphabet holds two things per position: which amino acid it is, and roughly what shape the backbone takes there.
One way to picture it
Writing each letter with an accent that says how the chain bends at that point.
Commonly misread as
It needs a structure at input time. If that structure is predicted, its errors go straight into the tokens.
Distinctions
- The structural alphabet comes from Foldseek, so the model needs a structure at input time.
- It sits between a sequence-only encoder and a multimodal generative model.
- Two checkpoint lines exist, one trained on predicted structures and one on the Protein Data Bank.
Limits
Requires a structure, predicted or experimental, which is a heavier input requirement than ESM-2. Errors in a predicted structure enter the tokens directly.
Weights and code
Checked against the registry, not from memory| Repository | Size | Licence | Note |
|---|---|---|---|
| westlake-repl/SaProt_650M_AF2 | 650M | MIT | — |
| westlake-repl/SaProt_650M_PDB | 650M | MIT | — |
Sources
Each number above comes from one of theseSame task, other answers
Protein language modelESM-2
Read enough proteins and structure falls out of the reading.
A bidirectional transformer trained to reconstruct masked amino acids from unaligned sequences.
Explore the architectureESM3
Sequence, structure and function as one masked prediction.
A generative masked model over three parallel tracks.
Explore the architectureESM C
The representation line, scaled further.
A transformer encoder with pre-layer-norm, rotary embeddings and SwiGLU activations, trained on sequences from UniRef, MGnify and the Joint Genome Institute clustered at 70 percent identity.
Read the entry