For healthtech founders and funders, scientific claims and conclusions are more than policy — business models depend upon the lucid appraisal of clinical problems, evaluating inadequacies in current standards of care, a clear understanding of disease pathways, and designing superior interventions.
At each step along this value chain, founders stand on the shoulders of the scientists that preceded them to obtain reliable evidence. When they promote their own innovations, credibility is a critical prerequisite. But where does credibility come from?
A 2012 study selecting 50 common cookbook ingredients found that 80% had publications linking their consumption to cancer risk; according to some reports, tomatoes, lemons, and celery all cause cancer. The to-and-fro of nutrition science is emblematic of a larger dynamic related to fickle research findings across disciplines. Because investigators seeking to build upon seminal studies struggle to reproduce the original findings, researchers have deemed the problem a reproducibility crisis.
Simulations have found that up to 85% of published findings could not be replicated. In turn, tens of billions of dollars are wasted and countless patient lives are adversely impacted annually due to unreliable research.
The Intersection Between Reproducibility and Healthcare Venture Capital
Historically, academic research and healthcare VC have had considerable overlap, but in recent years, this co-dependence has increased as researchers are looking more and more for financial support. Government research funding has seen a steady decline, with private sources now supporting almost 60% of the spend. Biomedical VC has been portrayed as a critical source of risk capital for early-stage research and a key engine for its translation at later stages.
Investors are increasingly looking to biomedical experts for support through knowledge: traditional healthcare entities such as universities and provider systems play an essential role in deal sourcing, diligence, and decision-making. This is particularly true as nontraditional and crossover investors flock to a sector where funding “mega rounds” are commonplace.
Simultaneously, VC involvement in healthcare is increasing. Population health, ancillary and subacute care, and health insurance are all being funded with venture capital (through companies such as CityBlock Health, Devoted Health, and Oscar). Digital health and healthtech at companies like 23AndMe, Ottobock, and Hims are being driven at a breakneck pace, and record-setting efforts to develop innovations premised on various forms of artificial intelligence, like those at Zebra Medical, have also been catalyzed by VC.
Yet, even as hoped-for health innovations rely on bulletproof research and development, the reproducibility crisis may be making “Eureka!” moments less likely. A 2016 survey of 885 VCs found distinctive considerations in healthcare compared to other sectors: namely, significantly higher weights placed on “product” as the most important business model factor to success—and “industry experience” as the most important managerial factor.
Taken together, these suggest that the success of healthcare companies is closely tied to an ability to translate and generalize their underlying scientific assets —which means that unreliable assets can threaten the viability of healthcare startups. For example, replication studies by Amgen and Bayer indicated that 89% and 64% respectively of preclinical candidate molecules derived from the published scientific literature could not be validated. Other healthcare domains, such as artificial intelligence, have been similarly effected.
“Innovation fatigue” at the level of frontline medical staff, resulting from the repeated failure of new technologies to make clinically meaningful impacts, piggybacks on this to cause a feedback loop in which subsequent technologies face compounding hurdles to successful implementation.
In the wake of these challenges, whispers of an inflating bubble in healthcare VC abound.
Approaches Healthcare Startups Can Take to Ensure Validity
These are all reasons to doubt the veracity of scientific novelties at face value, but evaluating research in a rigorous manner in parallel to product design, marketing, and fundraising can be redemptive.
Many healthcare-focused VCs have recommended that academic expertise and evidence generation become central roles in startup corporate strategy. “Without physician innovators as part of the creative team, many potential innovations will die in the cradle,” warns Andreessen Horowitz. A number of minimally-resourced intensive strategies (relying predominantly on founder/funder time and not requiring additional investments) exist which can support startups vetting and generating evidence.
Confirming the validity of study elements (such as key metrics or underlying datasets/reagents) can make sure that observed outcomes are consistent across real-world settings. Scrutinizing methods, controls, results, and statistical protocols promotes the certainty of study findings, as opposed to being artifacts of manipulative and non-transparent publication strategies. Assessing whether experiments were repeated—and contrasting these follow-on findings with the index study—can also help suss-out their long-term reproducibility.
Additionally, startups can generate their own evidence in-house.
Empirically, healthcare startups haven’t embraced this kind of research-centric approach to prove out their innovations to date. The generation of objective, company-relevant evidence has not been a requirement for VC: a recent review found that numerous healthcare unicorns had published no research at all, and the majority had no impactful evidence directly supporting the benefits of their innovations.
But since peer review is the major medium through which science is vetted and deemed valuable, healthcare startups could prove the worth of their groundbreaking innovations through disciplined experimentation. A variety of study paradigms exist through which startups can promote the scientific value of their innovations in some capacity without compromising IP, such as through academic partnerships.
As said by Stanford professor and entrepreneur Steve Blank: “there are no facts in your office, so get the hell outside.”
Reinterpreting Reproducibility as a Business Model
Additionally, baking robust evidence into business models should be a pillar, not a perk, of high-performing biomedical startups. Deferring evidence appraisal and/or generation until “strictly necessary” may well be myopic.
While indexing more heavily on evidence bears upfront costs, it simultaneously offers long-term promise for startups seeking to convince clinicians, provider systems, and payors to take up their goods/services — as well as incentive for the regulators who must approve them. These stakeholders are acutely sensitive to the need for impacts of innovation to be documented, validated, replicable, and generalizable.
The healthcare VC ecosystem would be well-served by cultivating a culture of reproducibility. Turning a blind eye to the quality of underlying healthcare assets has consequences that are measurable in morbidity and mortality beyond money alone.
Comment