OHE Critique of CBO’S Pharmaceutical Investment Model Provides Warning for Policymakers on Reliability of Estimates

US policymakers have been considering reforms to reduce drug spending, including allowing the government to directly set prices for branded medicines. Such policies would reduce global pharmaceutical revenues, leading to a reduction in pharmaceutical R&D expenditure and ultimately to lower levels of innovation.

To support policymakers in evaluating the impact of such proposals, the Congressional Budget Office (CBO) has developed models to quantify the loss in biopharmaceutical industry revenues and the reduction in the number of new drugs expected to reach the US market.

Most recently, the CBO developed a simulation model of the R&D investment decision-making process that can be used to evaluate any policy which alters expected pharmaceutical industry returns (e.g., any policy reducing drug prices) or R&D costs. Based upon this simulation model, the CBO has published estimates of the innovation impacts of drug price setting provisions of the Build Back Better Act (BBBA).

The CBO estimates that the BBBA would result in only one fewer drug being launched in the first decade (2022-31), four fewer in the second decade (2032-41), and five fewer in decade three and each subsequent decade. Although CBO does not explicitly quantify the expected impact on pharmaceutical industry revenues, based on our calculations, these estimates imply that the BBBA would reduce industry revenues by 2.6%.

A new OHE report evaluates the CBO’s simulation model of new drug development to assess whether the methodology provides policymakers with an accurate estimation of the impact of lowering US drug prices on future innovation. We conclude that while the model is novel and has intellectual merit, it is too limited to guide policymaking.

Moreover, the estimated impacts are similar in magnitude to those in CBO’s literature-based evaluation of H.R. 3, which for several reasons is likely to underestimate the true loss in innovation resulting from the policy. This raises the question of to what extent the CBO simulation model underestimates the innovation impacts of drug pricing policies. Due to the lack of transparency – an important point in itself – this question is hard to answer definitively, which further reduces confidence in the model as a tool for policymakers.

Summary of the limitations of the CBO simulation analysis

• The model makes the unrealistic assumption that investment decisions are based on a single product rather than a given company’s entire drug development portfolio.
• The model applies an oversimplified decision rule assuming that any positive expected return on investment is sufficient to incentivise clinical development. It assumes that any product that makes any profit will be launched.
• The model does not reflect that pharmaceutical companies and their investment decisions differ in relevant characteristics such as size and development costs.
• Preclinical development is assumed to be wholly unrelated to expected costs and returns of new drug development.[1]
• Signals about a drug candidate’s likelihood of success are unrealistically assumed to be independent across phases.
• There is an unexplained increase in the assumed baseline number of new molecular entities (NMEs) compared to the CBO’s first analysis.

We conclude that the CBO simulation model suffers from several serious flaws and fails to adequately represent the reality of biopharmaceutical investing. The simulation model may be of academic interest, but it cannot be reliably used to inform policymaking, at least not without significant qualifying information, including a better accounting of the uncertainty around the point estimates.

While policies that reduce drug spending by allowing the government to set prices may initially sound attractive, a focus on poorly targeted short-term savings could have major adverse consequences for patients in the future by causing an immediate decline in R&D spending, resulting in fewer new drugs coming onto the market. Importantly, the CBO model cannot project the type of innovation that would be lost, the populations impacted, or the impact of policy change on population health.  Policymakers should clearly understand that the CBO’s estimates are subject to considerable uncertainty and should exercise caution in relying on these findings for evaluating the potential impact of real-world policy changes.

Endnote:

[1] In the “Title XIII Notes” tab of their latest analysis entitled “Estimated Budgetary Effects of Title XIII, Committee on Ways and Means, H.R. 5376, the Build Back Better Act”, CBO claim to have made some technical changes to the model in their white paper. These include the modelling of effects of preclinical decisions about development, effects of greater costs of capital for small companies, and effects of accelerated approvals for some drugs. However, they have provided no further information on what has been updated or how this impacts their model.