Open Source Modelling: From Ideas to Action

OHE’s Chris Sampson is currently Chair of the ISPOR Open Source Models Special Interest Group (SIG). The SIG recently published their first journal article in Value in Health and will be delivering a Forum at this week’s ISPOR conference. In this blog post, Chris provides some background to the work and an invitation to get involved.

OHE’s Chris Sampson is Chair of the ISPOR Open Source Models Special Interest Group (SIG). The SIG recently published its first journal article in Value in Health and will be delivering a Forum at this week’s ISPOR conference. In this blog post, Chris provides some background to the work and an invitation to get involved.

Cost-effectiveness modelling has become a central part of health technology assessment (HTA). It now makes up a large proportion of the work conducted by people who call themselves health economists. Despite the importance and influence of this kind of research, it is often seen as a black box by decision-makers. What’s more, researchers often have to reinvent models that other researchers have already developed to tackle similar decision problems.

Transparency has long been recognised as a cornerstone of science. Transparent research practices enable replication, validation, and scrutiny, all of which contribute to knowledge generation. When it comes to health economics and outcomes research – and decision modelling specifically – there are numerous ways in which our research might be more or less transparent. A few years ago, I and some colleagues outlined the role of different manifestations of transparency and how they relate to one another. One example that we discussed is open source modelling. Open source modelling is about making the underlying analysis script freely available to researchers, who might borrow and adapt the methods in part or in full for their own purposes.

The potential benefits of open source modelling have been discussed at length at numerous meetings in recent years. And on the whole, researchers have written favourably about the prospects of open source modelling. It’s becoming a popular idea. Yet, open source cost-effectiveness models are still few and far between. Though researchers report their intent to adopt open source modelling practices, we don’t see it happening.

This is why we created the ISPOR Open Source Models (OSM) Special Interest Group (SIG). The OSM SIG was the brainchild of Renée Arnold, with whom I developed the SIG and became the first Chair-Elect (and now Chair). One of the first things we wanted to do was to build an understanding of why open source modelling isn’t more widespread, despite indications of support from the research community. For this purpose, we surveyed ISPOR members.

The results of our survey were recently published in Value in Health. Across 230 respondents, both academia and industry were well represented. Unsurprisingly, the support for open source modelling was reiterated by this sample, but we were also able to find out more about the perceived barriers to open source modelling. For instance, legal concerns and secure data sharing were cited as key barriers. And what might cause a shift in practice? Many respondents took advantage of the free-text part of our survey to highlight the potential value of HTA agencies mandating or otherwise supporting open source practices. We can still learn more about stakeholders’ perspectives on open source modelling – especially from decision-makers – but our study could inform the development of new initiatives that might incentivise the adoption of open source practices.

This week sees the return of an in-person ISPOR conference in the US, the first since 2019. Several members of the OHE team will be there, and the OSM SIG has organised a Forum session titled Open Source In Precision Medicine: The Perfect Fit? Our goal with this session is to get real. We’ll discuss applied examples of open source modelling in precision medicine and consider whether open source practices are especially valuable in this context. Sometimes, open source modelling is seen as complicated and technical, while decision modelling in precision medicine is often necessarily so. Does this make them perfect bedfellows? If you’re attending ISPOR in person, please come and join our discussion.

Even if you aren’t attending the ISPOR conference, there are other opportunities to engage with this topic and with the SIG. Next month, we’ll be hosting our latest (virtual) journal club meeting. Our guest will be Koen Degeling, who will be discussing an open source discrete event simulation for colorectal cancer using real-world data. To attend this journal club and keep up to date with future activities, join our SIG.

Related research

Pouwels, X.G.L.V., Sampson, C.J., Arnold, R.J.G., On behalf of the Open Source Models Special Interest Group, 2022. Opportunities and Barriers to the Development and Use of Open Source Health Economic Models: A Survey. Value in Health, 25(4), pp.473–479. 10.1016/j.jval.2021.10.001.

Sampson, C.J., Arnold, R., Bryan, S., Clarke, P., Ekins, S., Hatswell, A., Hawkins, N., Langham, S., Marshall, D., Sadatsafavi, M., Sullivan, W., Wilson, E.C.F. and Wrightson, T., 2019. Transparency in Decision Modelling: What, Why, Who and How? PharmacoEconomics, 37(11), pp.1355–1369. 10.1007/s40273-019-00819-z.