Summary
Once a new drug has been developed, it must be reviewed by both the regulators (for the decision on marketing authorisation) and health technology assessment (HTA) bodies (for the decision on access to patients). The regulators draw on data, mostly from clinical trials, to determine if a drug is safe and works well enough to be authorised for use in patients. For their part, HTA organisations assess more broadly the value of a new drug for a given healthcare system. For this, they need real-world evidence (RWE) to support the assessment of ‘relative effectiveness’: that is the extent to which a treatment does more good than harm when compared to one or more alternative treatments when provided under normal healthcare circumstances.
However, while there are clear, detailed guidelines on how to carry out clinical trials and what is required to achieve marketing authorisation, there is little guidance on how to generate RWE and integrate this into drug development. This is a serious issue: even if a drug is approved as safe and effective by the regulators, insufficient evidence supporting relative effectiveness may delay, or restrict (partially or entirely) patient access to new treatments. The challenge, therefore, is to incorporate relative effectiveness research into the earlier stages of drug research and development. IMI’s GETREAL project set out to bridge that gap.
The project broke ground in that it brought together companies, healthcare decision-makers, patient representatives and other stakeholders to begin to work towards building a consensus on best practices in the use of RWE in drug development, as well as regulatory and reimbursement decision-making. The project developed tools, recommendations and guidance for incorporating RWE earlier in the drug development process.
Getting to grips with RWE: the Navigator tool
One of the main project achievements was the launch of RWE Navigator. This online resource helps users from a broad range of research and healthcare backgrounds get to grips with the need for and the generation of RWE, and understand how this can be used to inform decision-making.
The RWE Navigator is designed to help users from industry define and implement a robust RWE generation plan; HTAs and regulators to better understand the analytical models used; and patients to understand concepts related to RWE and the challenges faced when using or generating RWE. It directs users to the TOOLBOX, which in turn guides them on how to define the attributes of the medicine that may drive a difference in the efficacy seen in a clinical trial setting, compared to the effectiveness when used in routine healthcare (often referred to as the efficacy-effectiveness gap). It also informs users on the specific types of analyses or study designs that can be used to generate or model RWE, including their pros and cons, and includes a comprehensive directory of resources to support users in implementing their RWE generation plans.
Pragmatic trials and the PRAGMAGIC tool
Pragmatic trials offer an opportunity to generate high-quality evidence on relative effectiveness prior to and (soon) after drug launch. However, many people are still reluctant to conduct such studies as they are very challenging to implement and there is some uncertainty regarding the acceptability of pragmatic trial data in decision-making.
In order to aid researchers in pragmatic clinical trial design, the GETREAL project developed an innovative online tool called PRAGMAGIC. It is a decision support tool designed to help users understand how the choices that they make when designing a pragmatic trial can impact the operational feasibility of the study, and how reflective the data generated will be of the effectiveness of the medicine in routine healthcare. The tool draws on guidance and recommendations accumulated through methods reviews, case studies and stakeholder feedback. The project hopes that the new tool will increase the understanding of the value and feasibility of conducting pragmatic trials in the future.
Demystifying relative effectiveness assessments
Both network meta-analysis (NMA) and multi-criteria decision analysis (MCDA) are useful methods for assessing the relative effectiveness of medicines by using data generated from randomised controlled trials. These methods are gaining acceptance among industry, academia, HTA agencies, and regulatory bodies. However, continued efforts are required to embed these approaches in decision-making. To this end, GETREAL developed the ADDIS tool, which provides users access to clinical trial data and the software required to apply NMA and MCDA to the data. Additionally, GETREAL improved the NMA and MCDA methods further by developing a methodology to include real-world data.
Webinars, courses and other achievements
Other important project achievements include:
- An interactive online course entitled ‘Real-world evidence in medicine development’. Under the supervision of leading academics from the project, the course provides students with an understanding of current techniques, opportunities and challenges for the use of real-world evidence in medicine development.
- RWE policy recommendations written by the project’s Policy Expert Group, setting out seven key themes that require further attention to advance the use of RWE and RWD in drug development.
- A network of regulators, HTA organisations, companies, academics, healthcare professionals, patients and other societal stakeholders. This network is important because the challenges faced by the pharmaceutical industry, regulators and other healthcare decision makers are all interlinked.
GETREAL also hosted a series of webinars to support the dissemination of information and organised many cross-sector workshops and events attracting participants from academia, industry, regulatory bodies and patient organisations from across Europe.
For the benefit of industry, academia and SMEs
All of the partners in the project benefitted from the network which was created and collaboration with different stakeholders. This was a unique opportunity for the different stakeholders to share experiences and perspectives in RWE generation and use in decision-making, and all of them will benefit from a greater consensus on the issue.
As the funding for methodology development can be scarce, the academic partners also benefited from IMI funding, which enabled them to drive the development of innovative RWE analytical approaches, published in high-impact journals.
Last but not least, the SME in the project was able to demonstrate their leadership in this field, and had the opportunity to work with leading academics and major pharmaceutical companies who ultimately are their key customers. Through publications and meeting attendance, they were also able to increase their company’s visibility.
What’s next?
GETREAL project partners applied for additional funding through IMI2 - Call 11 for the exploitation of project results. The follow-up project, the GetReal Initiative, started in June 2018 and built on the success of GETREAL. In particular, the follow-up project aimed to to continue building the research community, drive forward some of the project’s policy recommendations, and take forward the global debate regarding the opportunities for and the barriers to the use of RWE in healthcare decision-making.
Read the interview with the project coordinator.
Achievements & News
February 2023
IMI projects are presenting results on remote clinical trials, patient information, health data, and real world evidence at the upcoming...
A new non-profit organisation called the GetReal Institute, an outgrowth of the IMI-funded GetReal Initiative project, has been set up to drive the adoption of real-world evidence (RWE) in healthcare decision-making in Europe. ###The Netherlands-based institute will offer an independent, neutral meeting point where regulatory agencies, health technology assessment (HTA) bodies, patient organisations, partners from academia and industry and data partners can work on reducing barriers to the secondary use of data sources for healthcare decision-making.
The institute will tackle challenges related to data discoverability, quality, and accessibility, and issues relating to bridging the gap between RWE and conventional randomised controlled trial approaches. It will act as an ‘incubator and design lab’ where scientific and operational uncertainties relating to generating, evaluating and using RWE can be worked out, while also offering education and training resources through the GetReal Academy.
Any organisation involved in the generation, evaluation, or use of RWE may join the GetReal Institute. It will operate as a not-for-profit member association, and will be supported through a combination of membership fees, revenue from the GetReal Academy, and other programmes and grants.
Find out more
- Read the article in full
Registration is open for an interactive online course developed by IMI’s GETREAL project on the current techniques, opportunities and challenges for the use of real-world evidence in medicine development. The course, which is aimed at people from pharmaceutical companies, regulatory authorities, health technology assessment bodies, patients’ organisations, consultancy companies and academia, will run in September and October. ###It comprises 5 learning units, each of which will require around 8 to 10 hours of student input. The units will cover the following topics: overview of the medicine development landscape; real-world evidence generation; real-world evidence synthesis; decision-making and weighing of evidence; and demonstrating relative effectiveness to decision-makers. People who complete the course successfully will receive a certificate. Previous course participants have described it as ‘comprehensive’ and ‘a great way to increase my knowledge’.
Incorporating data from real-life clinical settings into drug development represents a serious challenge for pharmaceutical companies, regulators, and health authorities alike. By bringing together all key stakeholder groups to share their insights and know-how, IMI's GETREAL project developed new tools and resources for incorporating real-life data into drug development. These could increase confidence in new medicines and help to get them to patients more quickly. ###In an interview with the IMI Programme Office, project coordinator, Elaine Irving of GlaxoSmithKline explains the main project achievements, and why they wouldn’t have been possible without IMI. ‘We enabled open and transparent interaction with all the key stakeholders: health technology assessment bodies, regulators, patients, industry scientists, academia, and really unearthed all the different perspectives, which enabled us to work together on building solutions,’ she said. ‘It’s been a great collaboration, a great group. I stepped into this role only for the last year which was daunting because there was still a lot to be done. But in this last year we really focused and we really delivered. I think the team is pretty unique, they are all very driven and aligned on the need for the project and that energy has come through from start to finish.’
- Read the full interview
- Read the project factsheet
A potential drawback of clinical trials of new drugs is that they do not always reflect how well these will work in the 'real world'. IMI's GetReal project broke ground in that it brought together companies, healthcare decision makers, patient representatives and other stakeholders to generate a consensus on best practice in the use of real-world evidence in drug development, as well as regulatory and reimbursement decision-making. ###It is hoped that the alternative evidence-generating strategies developed will deliver more focused research in pharmaceutical R&D and allow healthcare decision makers to be more confident when giving patients access to new treatments. ‘Uncertainty slows things down with regard to getting new medicines to market,' says Chris Chinn, GetReal’s deputy project coordinator. ‘Social and economic benefits should follow if we have been successful in helping individual research teams work with the healthcare decision makers to define evidence-based studies that ensure effective medicines have a smoother path to patients, freeing up research capital for other projects and ensuring patients start to benefit sooner.’ Read the full story
Following the successful first edition of the interactive online course ‘Real-world evidence (RWE) in medicine development’, one of the deliverables of the GetReal project, a new edition of the course will start on 16 April. ### During the six-week course, students will learn about current techniques, opportunities and challenges for the use of real-world evidence (RWE) in medicine development. The course will be supervised by leading academics who were involved in the IMI GetReal project and is ideal for anyone who wishes to become familiar with the key issues in the areas of real-world data and to broaden their expertise. The course is particularly relevant for anyone working for pharmaceutical companies, regulatory authorities, HTA bodies, patients’ organisations, consultancy companies and academic institutions. As such, it will facilitate discussion, interaction and mutual learning across all professionals involved in medicine development. Participants who successfully complete the course assignments will receive a certificate at the end of the course. For more information and to enrol, visit the course page of the project website.
Launched in October 2013, GetReal was set up to explore how how robust new methods of real-world evidence (RWE) collection and synthesis could be adopted earlier in pharmaceutical R&D and the healthcare decision making process. The project, which brought together experts from diverse backgrounds, has delivered a range of tools and resources to help in this, including tools on the use of RWE and pragmatic clinical trials, as well as an online course on RWE. ### Now, the project’s Policy Expert Group has set out seven key themes that it believes require further attention to advance the use of RWE and real world data (RWD) in drug development.
1. Integrity, quality, access and privacy protection of RWD sources
2. Guidance on RWE study design, evidence synthesis and interpretation in decision making
3. Standards for decision makers’ use of RWE in decision making
4. RWE training and education
5. Broader involvement of stakeholders, especially patients and healthcare professionals, in RWE generation and use of RWD
6. Emphasis on a joint (regulatory/HTA/payer) scientific advice process
7. Construction of a standing forum and linking with ongoing initiatives
IMI’s GetReal project has launched an innovative tool to aid in pragmatic clinical trial design. Pragmatic trials aim to capture the true effect of a treatment strategy in the real world. As such, they provide a great opportunity to generate real-world evidence in the early stages of drug development. However, pragmatic trials are still in their early days and designing them is far from easy.### GetReal designed the PragMagic tool in collaboration with gaming company IJsfontein. ‘In a user-friendly (and dare we say fun) way, PragMagic gives insight into the possible consequences of more pragmatic trial design choices and operational challenges on generalisability, risk of bias, precision, stakeholder & ethical acceptability, cost and duration,’ explained Mira Zuidgeest of the University Medical Center Utrecht. The project hopes that the new tool will increase the value and feasibility of pragmatic trials by informing stakeholders when designing and assessing pragmatic clinical trials. ‘The tool helps to balance the aim to be pragmatic with the feasibility of the trial and other implications of design choices such as validity of the trial results,’ said Dr Zuidgeest. For more information on the tool, visit the PragMagic website.
IMI’s GetReal project will run the first edition of an interactive online course entitled ‘Real-world evidence in medicine development’ from 2nd October to 12 November 2017. Under the supervision of leading academics from the project, the course will provide students with an introduction to the topics covered by GetReal, and present the current techniques, opportunities and challenges for the use of real-world evidence in medicine development. Participants are expected to come from pharmaceutical companies, regulatory authorities, health technology assessment (HTA) bodies, patients’ organisations, consultancy companies and academia. As such, the course will foster discussion, interaction and mutual learning across all professionals involved in medicine development.###
The course will focus on the following topics:
- overview of the medicine development landscape
- real-world evidence generation
- real-world evidence synthesis
- decision-making and weighing of evidence
- demonstrating relative effectiveness to decision-makers
Participants who successfully complete the assignments will be awarded a certificate.
In March, the GetReal project launched the Real-World Evidence (RWE) Navigator. This is an online resource designed to help users from a broad range of research and healthcare backgrounds get to grips with RWE. RWE refers to evidence of relative effectiveness - how well medicines work in the real world, as opposed to the controlled experimental settings of clinical trials.### Data to generate RWE can come from a range of sources, including patient registries, medical records, pragmatic trials, and other observational studies. The overall aim of IMI’s GetReal project was to explore how new, better ways of generating and using RWE can be incorporated earlier on in drug development and healthcare decision-making. The RWE Navigator is an educational resource to help users find out more about potential issues in demonstrating the relative effectiveness of new medicines (referred to as ‘effectiveness issues’). It also guides users to specific types of analyses or study designs that use RWE to support the development of medicines. Finally, it is a comprehensive directory of resources that support the use of RWE in medicines development. The RWE Navigator has been designed for a wide variety of users. For example, pharmaceutical companies may find it useful to increase awareness about the potential use of RWE among their medicine development teams, or patients may use it to understand concepts related to RWE and the challenges faced when of using or generating RWE.
- To find out more about the RWE Navigator, watch the launch webinar. For specific enquiries about the RWE Navigator, contact navigator.admin@nice.org.uk
Before it draws to a close at the end of this year, IMI’s GetReal project will hold a closing event to showcase its results and discuss its future legacy. The event, titled ‘Delivering tools for real-world evidence development’, will take place on 24 November 2016 at the Royal Flemish Academy of Belgium for Science and Arts in Brussels.### Registrations are on a first-come, first-served basis. This event comes five months after a big stakeholder conference in London which aimed to gather perspectives on the potential value of GetReal in decision-making within industry, regulatory authorities, and patient groups. The report from that event is now available online. Launched in October 2013, GetReal is a three-year project which aims to show how robust new methods for real-world evidence collection and synthesis could be adopted earlier in pharmaceutical research and the healthcare decision-making process. It brings together 29 organisations representing pharmaceutical companies, academia, regulators, patient organisations and SMEs.
Information on drug development is often filled with jargon. Now IMI’s GetReal and ADAPT SMART projects have released new/updated glossaries to facilitate discussions in their respective subject areas.###
GetReal is investigating how to best incorporate ‘real world evidence’ into drug development. Its updated glossary, which was the subject of a consultation, also includes a more detailed discussion on questions such as the difference between ‘efficacy’ and ‘effectiveness’.
ADAPT SMART was set up (among other things) to coordinate IMI activities in the area of MAPPs (medicines adaptive pathways to patients). Many of the items in its glossary are drawn from the GetReal glossary. The project welcomes feedback on the glossary.
IMI’s GetReal project will hold a number of events this year on the themes of relative effectiveness and how it can best be integrated into drug development. Registration is now open for the following events.###
18 April: Introduction to the concept of drivers of effectiveness - The webinar will introduce the concept of drivers of effectiveness and will provide an overview of the methods which can be used to identify them early during the drug development plan.
10 May: Synthesis and integration of real-world evidence in network meta-analyses and outcome prediction - The focus of this webinar will be on the use of network meta-analysis for evidence synthesis and on the extension of its conclusions in real-world settings, using predictive modelling.
Further webinars are planned for later in the year. Visit the GetReal website for the latest updates.
IMI’s GetReal consortium has run the first five workshops bringing together stakeholders to discuss the use of real-world evidence for demonstrating the effectiveness of new drugs. Workshop participants have included so far: patients, clinicians, academic specialists, clinical trialists, pharmaceutical company representatives, regulators, health technology assessment (HTA) bodies and payers.### Collaboration between these diverse stakeholders is paramount for the success of this project and this is the first time many of the participants have worked together in a collaborative environment. By demonstrating that participation is both safe and beneficial, GetReal has successfully created a new platform for interlocutors to engage and share new ideas.
Each workshop includes up to 40 participants and the outputs will inform the delivery of a framework which will outline the potential uses of real-world evidence at different stages of medicine development. The workshops are part of case studies in the following disease areas:
- Multiple sclerosis
- Non-small cell lung cancer
- Rheumatoid arthritis
- Chronic obstructive pulmonary disease
- Metastatic melanoma
Launched in October 2013, GetReal is a three-year project which aims to show how robust new methods for real-world evidence collection and synthesis could be adopted earlier in pharmaceutical research and the healthcare decision-making process.
Participants
Show participants on mapEFPIA companies
- Amgen, Brussels, Belgium
- Astrazeneca AB, Södertälje, Sweden
- Bayer Pharma AG, Berlin, Germany
- Boehringer Ingelheim Internationalgmbh, Ingelheim, Germany
- Bristol-Myers Squibb Company Corp, Princeton, NJ, United States
- Eli Lilly And Company LTD, Basingstoke, United Kingdom
- F. Hoffmann-La Roche AG, Basel, Switzerland
- Glaxosmithkline Research & Development Limited, Brentford, Middlesex, United Kingdom
- Janssen Pharmaceutica Nv, Beerse, Belgium
- Merck Kommanditgesellschaft Auf Aktien, Darmstadt, Germany
- Merck Sharp & Dohme Corp, Whitehouse Station, New Jersey, United States
- Novartis Pharma AG, Basel, Switzerland
- Novo Nordisk A/S, Bagsvaerd, Denmark
- Sanofi-Aventis Recherche & Developpement, Chilly Mazarin, France
- Takeda Development Centre Europe LTD, London, United Kingdom
Universities, research organisations, public bodies, non-profit groups
- Academisch Ziekenhuis Groningen, Groningen, Netherlands
- European Organisation For Research And Treatment Of Cancer Aisbl, Brussels, Belgium
- National Institute For Health And Care Excellence, Manchester, United Kingdom
- Panepistimio Ioanninon, Ioannina, Greece
- The European Medicines Agency, Canary Wharf, London, United Kingdom
- The University Of Manchester, Manchester, United Kingdom
- Universitaet Bern, Bern, Switzerland
- Universitair Medisch Centrum Utrecht, Utrecht, Netherlands
- University Of Leicester, Leicester, United Kingdom
Small and medium-sized enterprises (SMEs)
- L.A. Sante, Epidemiologie, Evaluation Et Recherche SARL, Paris, France
Non EFPIA companies
- Haute Autorite De Sante, Saint-Denis la Plaine , France
- International Alliance Of Patient'Organizations, London, United Kingdom
- Zorginstituut Nederland, Diemen, Netherlands
| Participants | |
|---|---|
| Name | EU funding in € |
| Academisch Ziekenhuis Groningen | 615 260 |
| European Organisation For Research And Treatment Of Cancer Aisbl | 269 500 |
| Haute Autorite De Sante | 269 170 |
| International Alliance Of Patient'Organizations | 125 128 |
| L.A. Sante, Epidemiologie, Evaluation Et Recherche SARL | 860 242 |
| National Institute For Health And Care Excellence | 1 058 742 |
| Panepistimio Ioanninon | 97 050 |
| The European Medicines Agency | 117 501 |
| The University Of Manchester | 225 786 |
| Universitaet Bern | 1 430 140 |
| Universitair Medisch Centrum Utrecht | 2 270 934 |
| University Of Leicester | 464 447 |
| Zorginstituut Nederland | 196 100 |
| Total Cost | 8 000 000 |
Elaine Irving
Glaxosmithkline Research And Development LTD
United Kingdom
Diederick Grobbee
Universitair Medisch Centrum Utrecht