IDEAL2020 in Amsterdam, the Netherlands: Innovation and Evaluation in Surgery

Save the date: 11th-12th June 2020

Information on the conference #IDEAL2019 is now available on our conference webpage:

IDEAL Conference 2019

Details on how to register will be available early July.

BMJ Surgery, Interventions, & Health Technologies

BMJ Surgery, Interventions, & Health Technologies is an open access journal that publishes original research on complex healthcare interventions. Intended to encourage higher standards of science and more rapid dissemination of new knowledge in the field, it provides a high-impact international platform for both early and later stage clinical studies as well as long-term assessment and surveillance studies that meet high standards of design and reporting. It actively promotes the adoption of an IDEAL-based model of an evaluation framework and encourages the submission of studies at progressive stages, with a focus on device-based surgical and invasive treatments.

Read the Editorial: Lighting a candle by Art Sedrakyan and Peter McCulloch

There has been enough cursing of the darkness engendered by inadequate evaluation of new health technology and surgical and interventional treatments. We can do better.

To submit an article read the Instructions for authors and follow the link to the manuscript submission site.

The Current Issue of the Annals of Surgery features four papers about IDEAL

February 2019 – Volume 269 – Issue 2 

The Update to the IDEAL Framework and Recommendations:

This paper incorporates additional relevant methodological principles that have developed in the surgical community since the original IDEAL papers in 2009.

Announcing a new BMJ Journal 

We are delighted to announce that Prof Peter McCulloch (Chair of the IDEAL Collaboration) (below left) and Prof Art Sedrakyan (Vice-Chair) (below right) have been invited to become joint Editors in Chief of a new journal in the BMJ stable.  The journal will be called BMJ Surgery, Innovation and Health Technology, and we expect to produce the first issue early in 2019.

Scope of the Journal: The focus will be on evaluation of innovation in surgery, other invasive therapeutic techniques (such as interventional radiology, endoscopy, cardiology etc) and therapeutic medical devices. The Journal will be serving both surgical and technology communities (IDEAL and MDEpiNet community/societies). We will concentrate to a significant extent on early stage evaluation, and on registry-based studies, but will consider work at all stages in the life cycle of a therapy, from first-in-human use to long term study.  We will welcome studies in the formats prescribed by the IDEAL and IDEAL-D Recommendations, and our Instructions for Authors will include clear guidance on how to report such studies.  We intend to have a significant section dealing with reports from regulatory surveillance systems, registries and epidemiological studies using real world data and specific guidance will be available for any unique article types.

Our aim is to develop a thriving Journal which provides a platform for valid, appropriate studies of new complex therapeutic interventions using designs and reporting methods which allow maximum transparency.  To achieve this we will seek to ensure that the Impact Factor of the journal, which will be calculated after two years of publication, is high enough to ensure that it becomes the journal of choice for the type of studies we will specialise in.  We are therefore especially keen to hear from any Collaboration members who have unpublished work which they believe is liable to be newsworthy and/or strongly cited, and would urge you to submit to the new journal.  In the first two years of the journal we intend to experiment with our Editorial policy over formats and may be willing to discuss Supplements and compendium articles as well as reviews on highly topical issues.  An Editorial platform is currently under construction, and  an initial Editorial panel of referees has been appointed.  We will post a further announcement when the journal goes live for formal submissions, but would be glad to hear from you now if you would like to discuss possible submissions informally or volunteer to be a peer reviewer.

Peter McCulloch and Art Sedrakyan,

On Behalf of the IDEAL Collaboration and MDEpiNet

Email: peter.mcculloch@nds.ox.ac.uk

Hirst A, Philippou Y, Blazeby J, Campbell B, Campbell M, Feinberg J, Rovers M,  Blencowe N, Pennell C, Quinn T, Rogers W, Cook J, Kolias AG, Agha R, Dahm P, Sedrakyan A, McCulloch P. No Surgical Innovation Without Evaluation: Evolution  and Further Development of the IDEAL Framework and Recommendations.  Ann Surg.  2018 Apr 24. doi: 10.1097/SLA.0000000000002794. [Epub ahead of print] PubMed  PMID: 29697448.

A writing group from the IDEAL Collaboration including surgeons, research trialists, methodologists, statisticians, ethicists, and experts in device evaluation and HTA have developed an update to the original IDEAL Framework papers published in the Lancet in 2009.

OBJECTIVE: To update, clarify, and extend IDEAL concepts and recommendations.

BACKGROUND: New surgical procedures, devices, and other complex interventions need robust evaluation for safety, efficacy, and effectiveness. Unlike new medicines, there is no internationally agreed evaluation pathway for generating and analyzing data throughout the life cycle of surgical innovations. The IDEAL Framework and  Recommendations were designed to provide this pathway and they have been used increasingly since their introduction in 2009. Based on a Delphi survey, expert workshop and major discussions during IDEAL conferences held in Oxford (2016) and New York (2017), this article updates and extends the IDEAL Recommendations, identifies areas for future research, and discusses the ethical problems faced by investigators at each IDEAL stage.

METHODS: The IDEAL Framework describes 5 stages of evolution for new surgical therapeutic interventions-Idea, Development, Exploration, Assessment, and Long-term Study. This comprehensive update proposes several modifications.

1. A “Pre-IDEAL” stage describing preclinical studies has been added.
2. We discuss potential adaptations to expand the scope of IDEAL (originally designed for surgical procedures) to accommodate therapeutic devices, through an IDEAL-D variant.
3. We explicitly recognise the value of comprehensive data collection through registries at all stages in the Framework.
4. We examine the ethical issues that arise at each stage of IDEAL and underpin the recommendations.

The Recommendations for each stage are reviewed, clarified and additional detail added. We use a PICO format to outline the key features of each stage, outline reporting guidance for each stage and describe how to identify when an endpoint to a stage has been reached.

CONCLUSIONS: The intention of this article is to widen the practical use of IDEAL by clarifying the rationale for and practical details of the Recommendations. Additional research based on the experience of implementing these Recommendations is needed to further improve them.

We look forward to developing these Recommendations further by working with surgical researchers developing innovative surgical therapies and medical devices.

Peter McCulloch, Chair of IDEAL and co-authors examined progress in surgical research and their findings are now published in The Lancet Online First

Joshua Feinberg, surgeon, researcher and member of the IDEAL Collaboration at Maimonides Medical Center, Brooklyn, New York describes the research undertaken to examine how surgical research has improved in the last two decades and whether the IDEAL Framework and Recommendations have contributed.

“The quality of clinical research in surgery has long attracted criticism. High-quality randomised trials have proved difficult to undertake in surgery, and many surgical treatments have therefore been adopted without adequate supporting evidence of efficacy and safety. To address this, we conducted a review of the progress of methodology in clinical studies of surgery, and whether it is moving towards compliance with the IDEAL Recommendations.

To evaluate progress, we examined the surgical literature, focusing specifically on studies of surgical outcomes, in two periods 10 years apart, one before and one after the publications of the IDEAL Recommendations (2000-2004 and 2010-2014). Compliance with the following IDEAL recommendations were compared between the two time periods:

• Use of standardised terminology
• Definition and description of procedure
• Prospective data collection
• Explanation of modifications during early studies (2a)
• Previous analysis of learning curves in pre-RCT studies
• Use of quality control measures
• Use of qualitative research to define RCT questions
• Use of prior prospective cohort study to prepare for RCTs
• Mention of pilot of feasibility studies to prepare for RCT
• Masking reported in RCTs

The main findings of the study include:

IMPROVEMENT in the use of standard outcome measures, adoption of Consolidated Standards of Reporting Trials (CONSORT) standards, and assessment of the quality of surgery and of learning curves.

NO PROGRESS in the use of qualitative research or reporting of modifications during procedure development.

When discussing visions for the future, the paper concludes

“Surgical research is getting better, although it still has a long way to go…The IDEAL Framework and Recommendations have probably only contributed in a minor way to the improvements seen so far, but their influence is growing, they are useful as a tool to measure progress, and they represent a serious attempt to create a new framework for surgical research methodology.”

We are pleased to announce that the China IDEAL Centre was opened in Chengdu, Schizuan province, on 28th October 2017.

The Director, Professor Xin Sun, is also Director of the Chinese Cochrane Centre, and of a large statistics and clinical epidemiology department at the West China Hospital in Chengdu. Funded by the Chinese government through the West China hospital and university, the centre will educate clinicians about IDEAL and research methodology and help groups to develop their clinical research using IDEAL principles.

At the opening event of the Centre Professor Xin Sun and the Emeritus Director, Professor Louping Yi, emphasised the value of IDEAL for the evaluation of new medical devices and procedures in the Chinese health system.  They referred to recent new guidance from the FDA on the use of “real world evidence” to evaluate devices, and stressed how IDEAL could provide a guiding template for interpreting FDA proposals.  This theme was supported by the representative of the Chinese FDA who spoke at the meeting, who explained how China aimed to develop a distinctive approach which is nevertheless compatible with US FDA guidance.

Professor Peter McCulloch (IDEAL Chair) presented two talks, one on the principles of IDEAL and the other on its use in device evaluation, and how it could assist regulators and healthcare purchasers.

The meeting was generously sponsored by Medtronic China, whose Vice-President for Clinical Affairs, Xioajing Chen, made a presentation on Medtronic’s plans for device surveillance.

We look forward to supporting and collaborating with IDEAL China. Future plans include development of a post-doctoral Fellowship in Oxford for one of the IDEAL China staff and co-operation on research projects. It was agreed that one important area where IDEAL needed to develop a clear position was the guidelines for policy on what types of evidence should be acceptable for regulation or purchasing in which stages of evolution of a device.  An annual IDEAL China conference is planned and it is hoped next year’s event will be able to report progress based on this year’s meeting.

We are delighted to welcome Dr Arsenio Paez to the IDEAL Collaboration in the role of Specialty Lead for Physiotherapy.

Arsenio is a physiotherapist with a pediatric private practice in New York, NY. He is a Professor in the Doctoral Program in Physical Therapy in the Dept of Physical Therapy, Movement, and Rehab Sciences at Northeastern University, Boston, MA, and former Lend Fellow in Neurodevelopmental Disabilities at the Children’s Hospital, Boston. 

Arsenio recently completed the MSc in Evidence-Based Health Care at the University of Oxford, and has applied to continue to a DPhil in Evidence-Health Care for 2018.

His current area of research is in neurodevelopmental outcomes in childhood epilepsy, innovative practice in physiotherapy and clinical trial methods in complex interventions.

Arsenio states ” I am very enthusiastic about IDEAL and what IDEAL-Physio can do to improve the quality of practice and research in physiotherapy. We have had a great surge in the amount of evidence being designed and produced in physiotherapy, with over 18,500 clinical trials and 5600 systematic reviews added to the body of evidence in the last decade alone. Ideal-Physio offers us an invaluable and practical framework to help guide the innovation of new practices and improve the quality of evidence. The unique, complex nature of our interventions requires a dynamic and practicable approach to innovation and evidence gathering, and I believe IDEAL-Physio draws on these to create something truly representative of the nature of complex interventions. It also has great potential to aid us in preparing the evidence-makers and innovators of the future, with very promising applications in academic settings and clinical education”.

Arensio has led work on extending IDEAL to physiotherapy (IDEAL-Physio) recently accepted for publication in the Physical Therapy Journal:

Beard D, Hamilton D, Davies L, Cook J, Hirst A, McCulloch P and Paez A. Evidence-based evaluation of practice and innovation in Physical Therapy using the IDEAL-Physio framework. Physical Therapy Journal (In Press) DOI 10.1093/ptj/pzx103 

(Link to follow when available)

“There is a clear need for surgical registry data to improve patient safety and help regulate surgical practices.The IDEAL collaborative, Department of Health (DOH), the National Institute for Health and Care Excellence (NICE), policymakers and commissioning groups have called for surgical registries that can collect prospective outcome and safety data, promote transparency as well as provide patients and the public with information on their care. However, developing and maintaining a registry faces considerable challenges with the majority of registries failing.

To help address this, we conducted a systematic review and narrative synthesis, learning from existing and previous registries, to identify the fundamentals to successful registry development. We found that the key factors for successful registry development include:  steering committee to lead and oversee the registry; clear registry objectives; planning for initial and long-term funding; strategic national collaborations among key stakeholders; dedicated registry management team; consensus meetings to agree registry dataset; established data processing systems; anticipating challenges; and implementing strategies to increase data completion. Patient involvement and awareness of legal factors should occur throughout the development process.

Our work provides robust knowledge that can be used to inform the successful development of any UK surgical registry. It also provides a methodological framework for international surgical registry development”

Mr Rishi Mandavia , NIHR Academic Clinical Fellow ENT Surgery and NICE Specialist Advisor, evidENT Team, Ear Institute, University College London

Link to the paper here: http://bmjopen.bmj.com/content/7/9/e017373.info