The 2% Trap: What Pharma Can Learn from Machine Learning About Ontology, Noise, and Asymmetric Learning
A recent post by Yun-Ta Tsai, Sr. Staff Engineer at Tesla AI, cut through the hype with unusual clarity (Elon Musk quickly replied “So true”):
“Many people think any given ML project is 99% training.
In reality, it’s 50% evaluation, 40% data cleaning, 8% integration, and 2% training.
The first two set the noise floor for learning. No ML magic matters; the model cannot lower the noise floor, as that’s the optimal bound of Shannon encoding of your data.
Thus, not a single day goes by without me thinking about ontology. Even the old labels have to be constantly reviewed.”
Link to the original post:
Most people, he argued, assume that success in ML is mostly about training. In reality, the greater share of the work lies elsewhere: in evaluation, in data cleaning, in integration, and in the ontology used to define the problem in the first place. Training matters, of course. But it does not rescue a noisy system. If the labels are crude, the assumptions unstable, or the structure of the problem poorly framed, the model can only learn within those limits.
That same logic applies, with surprising force, to drug development.
In pharma, we tend to become captivated by the visible and dramatic 2%: the pivotal Phase III readout, the elegance of the mechanism, the brilliance of the launch strategy. These are the moments that attract the most attention, the most money, and the most organisational energy. But the eventual ceiling on a programme is usually set much earlier, and much more quietly.
It is set by the quality of the ontology.
By that I mean the categories, assumptions, labels, and frames through which a team first understands an asset. Is this “a diabetes drug”, or something broader? Is this patient group non-responsive, or merely poorly segmented? Is this endpoint non-viable, or simply too blunt to capture what matters? These questions sound abstract, but they are not. They shape what gets tested, what gets noticed, what gets discarded, and what never even comes into view.
Get those early frames wrong, and you create a permanent noise floor. From that point on, no amount of late-stage brilliance can fully compensate. Bigger trials, better statistics, sharper execution - all of these may improve the handling of the signal, but they do not fundamentally change its quality. A poorly framed programme may still advance, but it does so under a ceiling it set for itself long before anyone noticed.
This is one reason so many pharmaceutical pipelines underperform. Not because the science is weak, or the teams unintelligent, or the trials simply unlucky. More often, they underperform because the learning architecture around the asset is too narrow, too linear, and too committed to early assumptions.
That is where asymmetric learning matters.
Give the same molecule to two capable teams and one will often create far more value than the other. Not necessarily because it predicts better, but because it learns better: faster, earlier, and with greater willingness to revisit what the asset might actually be. The stronger team does not merely execute the plan more efficiently. It improves the plan by improving the quality of the questions being asked.
That learning has to happen across at least three domains at once.
First, there is the biological question: what can the molecule actually do, beyond the first and most obvious hypothesis? Second, there is the clinical and commercial question: where might that effect matter most, and to whom? Third, there is the regulatory and payer question: what kind of evidence will ultimately be meaningful, credible, and fundable in the real world?
The teams that pull ahead are usually the ones that build better loops between those domains. They explore in parallel rather than in sequence. They revisit assumptions instead of defending them. They treat early development not simply as a process of proving a thesis, but as a process of refining one.
There are obvious examples. GLP-1 agonists did not become transformative because the molecules changed. What changed was the way their value was interpreted: side effects became benefits, narrower labels gave way to broader opportunity, and a therapeutic class once seen in relatively modest terms was re-understood at a much larger scale. In immuno-oncology, first movers were often overtaken by teams that became better at identifying the right patients, the right endpoints, and the right development logic. In both cases, the advantage came not only from science, but from superior learning.
The pattern recurs whenever organisations resist the temptation to lock too early into a single story about what an asset is for. Biomarker-led expansion, parallel development pathways, and deliberate exploration of edge-case responders all create forms of optionality that rigid pipelines tend to suppress. They may look untidy in the short term. In the long term, they are often the source of the best strategic surprises.
If that is right, then one of the central tasks in drug development is not simply to advance compounds. It is to improve the quality of the decisions made about them, early enough for that improvement to matter.
That requires a few practical shifts.
It means revisiting ontology as a routine discipline rather than an occasional moment of reflection. Teams should repeatedly ask which categories they are treating as fixed that may in fact be provisional: patient groups, endpoints, mechanisms, comparators, even the definition of success itself.
It means rewarding evaluation and interpretation, not only progression. Clean signal, sharper segmentation, and more discriminating early experiments are not administrative preliminaries to “real” development work. They are the work.
And it means treating learning velocity as a serious operating metric. Not simply: how quickly did the programme move? But: how quickly did the organisation improve its understanding of what the programme was?
The deeper point is that late-stage excellence cannot compensate fully for early conceptual poverty. Once a programme is built on noisy categories, crude labels, or unexamined assumptions, it becomes very expensive to discover that fact. By then, most organisations do not really learn; they simply scale their earlier misunderstanding.
Tsai’s observation, then, is not just about machine learning. It is a useful reminder that in any complex system, the glamour of execution can distract from the discipline of framing. Pharma has become highly skilled at the visible end of development. What it still undervalues, too often, is the quieter work of defining the problem well enough that the rest of the system has a chance to learn.
That is why asymmetric learning matters. It is not a slogan for moving faster or taking more risk. It is the disciplined habit of building a better signal before committing to scale.
And in drug development, that may be the difference between a programme that progresses and a programme that genuinely becomes more intelligent as it moves.