.png)
Two independent teams โ UT Austin, and a Microsoft / NVIDIA / Profluent / Caltech / Duke / Oxford collaboration (FLIP2) โ reached the same conclusion: protein language models rarely beat simple baselines when real experimental data exists, and their rankings collapse to near-random when pushing toward genuinely new function.
Evidence continues to accumulate against the protein-LLM hype. Two rigorous studies from recent weeks caught the attention of our team. Separate groups, same conclusion.
โ
One from UT Austin (Ellington and Wilke labs), one from Microsoft / NVIDIA / Profluent / Caltech / Duke University / University of Oxford (FLIP2).
โ
What they found:
- ๐๐๐ญ๐ ๐ข๐ฌ ๐ค๐ข๐ง๐ . ๐๐ฉ๐ฆ๐ฏ ๐บ๐ฐ๐ถ ๐ฉ๐ข๐ท๐ฆ ๐ณ๐ฆ๐ข๐ญ ๐ฆ๐น๐ฑ๐ฆ๐ณ๐ช๐ฎ๐ฆ๐ฏ๐ต๐ข๐ญ ๐ฅ๐ข๐ต๐ข, expensive models rarely earn their cost. FLIP2 showed plain linear regression often matched or beat fine-tuned protein language models. A fine-tuned pLM was the best option on fewer than half the tasks tested.
- ๐๐๐ซ๐จ-๐ฌ๐ก๐จ๐ญ ๐ข๐ฌ ๐ ๐๐ข๐ฅ๐ญ๐๐ซ, ๐ง๐จ๐ญ ๐ ๐ซ๐๐ง๐ค๐๐ซ. ๐๐ฉ๐ฆ๐ฏ ๐บ๐ฐ๐ถ ๐ฅ๐ฐ๐ฏ'๐ต ๐ฉ๐ข๐ท๐ฆ ๐ฆ๐น๐ฑ๐ฆ๐ณ๐ช๐ฎ๐ฆ๐ฏ๐ต๐ข๐ญ ๐ฅ๐ข๐ต๐ข, the models can flag a broken protein, but inside the working set, when your goal is to find the best variants, their ranking drops to roughly random. (This is an obvious consequence when you understand the training data being unlabled.)
- ๐๐ฎ๐ฌ๐ก ๐ญ๐จ๐ฐ๐๐ซ๐ ๐ ๐๐ง๐ฎ๐ข๐ง๐๐ฅ๐ฒ ๐ง๐๐ฐ ๐๐ฎ๐ง๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐ญ๐ก๐๐ฒ ๐๐ซ๐๐๐ค. On mutations meant to redirect a protein to a new substrate or new chemistry, top models came out slightly negatively correlated with measured results. In other words, they point you at the wrong variants. These models have only ever seen what evolution already built.
- ๐๐ก๐๐ฒ ๐ฆ๐จ๐ฌ๐ญ๐ฅ๐ฒ ๐๐ ๐ซ๐๐ ๐ฐ๐ข๐ญ๐ก ๐๐๐๐ก ๐จ๐ญ๐ก๐๐ซ, ๐ง๐จ๐ญ ๐ฐ๐ข๐ญ๐ก ๐ซ๐๐๐ฅ๐ข๐ญ๐ฒ. Predictions correlate more across models than with ground truth, because they all read the same natural sequence data.
โ
The conclusion: the bottleneck is not the algorithm. It is high-quality labeled data. Both papers more or less end there.
โ
None of this is new. It gets shown again and again in rigorous works in the field. It just rarely seems to reach the ecosystem beyond the real scientists, where "foundation model for biology" keeps being hyped while real breakthroughs in data generation barely register a ripple.
โ
The best case scenario I hope for: the ecosystem catches up to what rigorous science is finding, gets a hold of the mania, and directs capital toward real innovations rather than the next hype company.
โ
The worst case I fear: things go on unchanged until five years from now, when the hyped claims don't materialize, and capital loses confidence, -- not just in the hype companies, but in the entire field of proteins, enzymes, and biotech. We've seen that in synthetic biology thanks to the disservice of a few irresponsible hype players.
โ
Regardless of what happens, ENZIDIA will keep building the way we believe is right and responsible. I just hope there isnโt this much waste of resource and talent when there are pressing challenges to be solved for humanity.
โ
Discussion credit to: Bruce Wittmann, Adam Meyer
โ
