The IDEAL team Claudia Ashton, Allison Hirst, Claire Thomson and Peter McCulloch presented a workshop on using IDEAL to appraise the surgical literature at #EvidenceLive on 21st June 2017 http://evidencelive.org/
Following the workshop Peter McCulloch, Chair of the IDEAL Collaboration discussed some of the challenges of surgical research with Douglas Badenoch @Mental_Elf
As part of our 2017 Conference in New York, 4th-5th May 2017, we are holding a half day educational workshop. Please select this at the time of registration for the conference at www.idealconference2017.com
Early registration discount available until April 1st Registration fees include breakfast and lunch each day of the conference, and CME credits will be available for physicians. For further information, contact: information@ideal-conference2017.com or visit www.ideal-collaboration.net.
Workshop 4th May 2017
Critical appraisal using the IDEAL Framework (provisional timetable)
8.30—9.15AM (JH) History of EBM; Inherent difficulties of surgical evaluation; lack of formal education among trainees in US
9.15—10.00AM (JF) Group workshop – Systematic approach to the Critical Appraisal of a Surgical Paper
10.00—10.30AM Coffee/Tea
10.30—11.00AM (CP) IDEAL: Framework and How to utilize it
11.00AM – 11.30AM Group workshop – Identifying the IDEAL stage of innovation
11.30 – 12.00PM (PM) Using IDEAL: Progressing through the life-cycle of surgical Innovation
12.00—12.15PM (JF) IDEAL: Current status and future
12:15 – 12.30 (JF) Assessment of participants’ comprehension of IDEAL
Teaching Faculty
JH – Joel Horovitz, Vice Chair of Surgery, Maimonides Medical Center
JF – Joshua Feinberg, Maimonides Medical Center
CP – Christopher Pennell, Maimonides Medical Center
PM – Peter McCulloch, Chair IDEAL Collaboration, University of Oxford
This question was recently addressed and the findings published in the British Journal of Surgery
Two of the authors describe what they did and what they found;
Stephen J Chapman, Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK
Aneel Bhangu, Dept of ColorectalSurgery, University of Birmingham, Birmingham, UK
Innovation in surgery is important to ensure the treatment of surgical disease continues to advance. The role of industry is paramount in driving this forward, but must be matched by robust clinical evidence and dedicated regulation.
Whilst regulatory bodies in the U.K. work to ensure safe introduction of surgical devices, many are supported by minimal or no evidence of safety or effectiveness. We are acutely aware of this problem; new surgical devices are introduced every day yet these may be unfamiliar or unproven. Whilst regulatory bodies are often satisfied with evidence of “equivalence”, the true clinical impacts of these are unclear.
Our study, published in the British Journal of Surgery, demonstrated the extent of this problem (1). We systematically compiled a list of implantable devices used in gastrointestinal surgery and set out to find evidence to support their use. Alarmingly, only 1 in 10 of these devices was supported by evidence from high quality randomised controlled trials (RCT).
Whilst RCTs are perceived to slow the rate of innovation, dedicated networks of surgeons have proven their ability to evaluate surgical interventions quickly and effectively (2). Additionally, whilst RCTs are not always the most appropriate design to test novel interventions, they provide high quality evidence when undertaken by dedicated clinical and academic networks.
In line with proposals from the IDEAL Collaboration, we encourage a standardised, step-wise approach to the assessment of novel surgical technology. This should begin with first in man studies, progress through stages of randomised assessment, and culminate in long term surveillance in dedicated devices registries.
References:
Would you like to participate in the development of major new scientific studies to evaluate your work?
If so, the 2017 IDEAL MEETING in New York offers a unique opportunity. During the SURGICAL SPECIALITY THINK TANKS ON 4TH MAY IDEAL 2017 Provisional Agenda we’ll be seeking to initiate multi-centre prospective collaborative studies of one new cutting edge technique in each workshop. These will use the IDEAL 2b PROSPECTIVE COHORT format and the intention is that the initial study will form the basis for a subsequent major RCT. If you would like to get in on the ground floor of one of these collaborative studies by surgical innovators, register for the meeting and send your ideas to: information@ideal-conference2017.com
Registration is now open at www.ideal-conference2017.com – early registration discount available until April 1st 2017! Registration fees include breakfast and lunch each day of the conference, and CME credits will be available for physicians. For further information, contact: information@ideal-conference2017.com or visit www.ideal-collaboration.net
This is an opportunity to help plan important studies which will shape the direction of future practice in your speciality. The IDEAL Collaboration contains expert methodologists, statisticians and others who can help guide the development of research groups and reduce the burden of participation on clinicians to a minimum.
We look forward to meeting you and working together on these exciting projects.
Prof Peter McCulloch, IDEAL Chair, Oxford (left)
Prof Art Sedrakyan, IDEAL Vice-Chair, New York (right)
The provisional programme for our 2017 conference is now available. See the preview below and download the full pdf here: IDEAL Agenda 1 26_NYC
For further details and information on registration and abstract submission please see our IDEAL Conference 2017 webpage
Abstracts from our 2016 Conference are now available in the International Journal of Surgery
December 2016 Volume 36, Supplement 2, S133-S144
Access to innovative treatments and device support during IDEAL stages 2b-4 K. Hutchison
Advancing the cause of research registration: The first 500 registrations of the ResearchRegistry.com R. Agha, A.J. Fowler, C. Limb, Y. Al Omran, H. Sagoo, K. Koshy, D.J. Jafree, M.O. Anwar, Peter McCulloch, D.P. Orgill
Surgical trainee research collaboratives in the United Kingdom A.G. Kolias, A.A.B. Jamjoom, P.N.H. Phan, P.J.A. Hutchinson
The merits of decision modeling in the earliest stages of the IDEAL framework – The case of innovative bilateral DIEP flap surgery J. Grutters, J. Gerrits, L. Schultze Kool, D. Ulrich, M. Rovers, S. Hummelink
Evaluation of innovation in radiation oncology: R-IDEAL H. Verkooijen, L. Kerkmeijer, D. Fuller, Robbert Huddart, C. Faivre-Finn, M. Verheij, A. Sahgal, E. Hall, M. van Vulpen
Small simple trials: A strategy to study rare surgical condition J.G. Wright
Short-term cost-effectiveness of colonic stenting is lost at 90-day follow-up in patients with malignant large bowel obstruction H. Yeo, J. Abelson, J. Milsom, S. Sharma, A. Sedrakyan
3-Year outcomes and cost-savings of combined endoscopic laparoscopic surgery (CELS) for benign colon polyps M. Kiely, S. Sharma, A. Sedrakyan, J. Yoo, H. Yeo, J. Abelson, J. Milsom
Stereotactic body radiotherapy followed by surgery for unstable spinal metastases: Technical feasibility and safety study according to the IDEAL stages 1 and 2a. A. Versteeg, J. van der Velden, H. Verkooijen, W. Eppinga, N. Kasperts, S. Gerlich, C. Oner, M. van Vulpen, J.-J. Verlaan
Perioperative outcomes, health care costs and survival after robotic-assisted versus open radical cystectomy: A national comparative effectiveness study J. Hu, B. Chughtai, P. O’Malley, J. Halpern, J. Mao, D. Scherr, D. Hershman, J. Wright, A. Sedrakyan
Comparative effectiveness of cancer control and survival after robotic assisted versus open radical prostatectomy J. Hu, B. Chughtai, P. O’Malley, A. Isaacs, J. Wright, D. Hershman, A. Sedrakyan
Reconstruction of bladder defects with amniotic membrane – IDEAL-D Stage 0-1 D. Barski, H. Gerullis, A. Winter, I. Pintelon, J.-P. Timmermans, A. Ramon, M. Boros, G. Varga, T. Otto
Is more evidence always better? The value of adding decision analytical modeling to the IDEAL framework C. Tax, P.H.M. Govaert, M. Stommel, M.G.H. Besselink, H.G. Gooszen, J. Grutters, M.M. Rovers
Progressing through IDEAL: When is the right time to move from observational to randomised studies? – A case study of REBOA J.O. Jansen, M.K. Campbell
MiCollar – A novel iPhone application to analyse cervical spine motion restriction with different size and type of cervical orthoses R. Ingleton, C. Ashton, J. Bull, V.N. Vakhari
The X-Bolt dynamic hip plating system: Evaluating a novel surgical device for hip fracture surgery M. Fernandez, J. Achten, N. Parsons, X.L. Griffin, M.L. Costa
Radical cystectomy in epidural anaesthesia – Feasibility analysis using a new reporting method following the IDEAL recommendations H. Gerullis, T.H. Ecke, C. Bantel, A. Weyland, D. Barski, T. Jansen, J. Uphoff, F. Wawroschek, A. Winter
Applying IDEAL: Early stage surgical innovation of a novel bio-wrap-assisted vasectomy reversal technique A. Gudeloglu, J. Brahmbhatt, S. Parekattil
Matching trial design decisions to the needs of those you hope will use the results: The PRECIS-2 tool K. Loudon, M. Zwarenstein, F. Sullivan, P. Donnan, S. Treweek
The GASTROS Study: Standardising outcome reporting in gastric cancer surgery research Bilal Alkhaffaf, Anne-Marie Glenny, Jane Blazeby, Paula Williamson, Iain Bruce
Support for reporting guidelines in surgical journals needs improvement: A systematic review R. Agha, I. Barai, S. Rajmohan, S.Y. Lee, M. Anwar, A.J. Fowler, D. Orgill, D. Altman
Compliance of systematic reviews in plastic surgery with the PRISMA statement: A systematic review S.Y. Lee, H. Sagoo, K. Whitehurst, G. Wellstead, A.J. Fowler, R. Agha, D. Orgill
A systematic review of the methodological and reporting quality of case series in surgery R. Agha, A.J. Fowler, S.Y. Lee, B. Gundogan, K. Whitehurst, H. Sagoo, K.J.L. Jeong, D. Altman, D. Orgill
An assessment of the compliance of systematic review articles published in craniofacial surgery with the PRISMA statement guidelines: A systematic review T.E. Pidgeon, G. Wellstead, H. Sagoo, D.J. Jafree, A.J. Fowler, R. Agha
The use of study registration and protocols in plastic surgery research: A systematic review T.E. Pidgeon, C. Limb, R. Agha, K. Whitehurst, C. Chandrakumar, G. Wellstead, A.J. Fowler, D. Orgill
Twist-drill craniostomy with hollow screws for evacuation of chronic subdural haematoma A.G. Kolias, A. Chari, S.J. Bond, P.J.A. Hutchinson
Intra-operative hyperspectral imaging for brain tumour detection and delineation: Current progress on the HELICoid project S. Kabwama, D. Bulters, H. Bulstrode, H. Fabelo, S. Ortega, G.M. Callico, B. Stanciulescu, R. Kiran, D. Ravi, A. Szolna, J.F. Piñeiro
Developing a patient and public involvement intervention to enhance recruitment and retention in UK surgical trials (PIRRIST) J.C. Crocker, S. Rees, L. Locock, S. Petit-Zeman, A. Chant, S. Treweek, J.A. Cook, N. Farrar, K. Woolfall, J. Bostock, L. Bowman, R. Bulbulia
Osseointegrated joint replacement connected to a lower limb prosthesis: An IDEAL stage 1 proof of concept study with 7 cases A. Khemka, S. Lord, M. Al Muderis
The East Grinstead Consent Collaborative (EGCC) online consent creator J. Dhanda, M. Dungarwalla
Development of an online platform for registration and outcome measurement of urogynecological implants according to IDEAL-system D. Barski, H. Gerullis, T. Ecke, R. Joukhadar, J. Kranz, R. Tahbaz, F. Queissert, L. Schneidewind, S. Mühlstädt, M. Grabbert, N. Huppertz, A.E. Pelzer, U. Klinge, M. Boros, W. Bader, F. Puppe, T. Otto
Bringing laparoscopy to your own home M. Vella-Baldacchino, M. Vella Baldacchino, M. Schembri, R. Bugeja
Autologous plasma coating improves the biocompatibility of mesh implants. On the IDEAL way from bench to bedside H. Gerullis, D. Barski, T.H. Ecke, C. Eimer, M. Boros, B. Klosterhalfen, A. Ramon, T. Otto
Percutaneous nephrolithotomy (PCNL) under local infiltrative anesthesia with and without stand-by anesthesia – Complication rates and clinical outcome for a method at the exploration stage according to IDEAL T.H. Ecke, G. Weingart, C. Lange, S. Hallmann, F. Wawroschek, D. Barski, J. Ruttloff, H. Gerullis
The landscape of surgical innovation in robotic microsurgery A. Gudeloglu, B. Ozdemir, S.R. Fleischman, P. Dahm
Awareness of the ideal recommendations among the members of the Robotic-Assisted Microsurgery and Endoscopic Society (RAMSES) A. Gudeloglu, S.R. Fleischman, P. Dahm
Project outlineSurgical interventions can have profound effects on people’s lives, both in terms of benefits and (potentially) in terms of risks or complications. It is therefore essential that surgical interventions should be evaluated appropriately and accurately to inform clinical decisions.
Although randomized controlled trials (RCT) provide gold standard evidence, surgical RCTs are expensive, challenging (given pragmatic, ethical, and other considerations), and hence scarce in the scientific literature. In addition, RCTs tend to use restrictive eligibility criteria therefore limiting the external validity (generalizability) of their findings to relatively healthy populations.
Conversely, routinely collected data (i.e. anonymized data from national registries, audits, and computerized NHS records) provide a unique opportunity to evaluate surgical interventions in potentially any NHS user, widening our ability to gather information on the performance of surgical and medical alternatives in different patient groups. However, given the observational nature of such studies, confounding (by indication) is a major challenge that needs to be accounted for to avoid biased results. Methods exist to minimize such confounding and other biases, which are widely used in pharmaco-epidemiology (drug safety studies), but these have been seldom tested for the study of surgical treatments.
The project will focus on two clinical scenarios for the risk-benefit evaluation of 1.two different surgical alternatives (unicompartmental vs total knee replacement) for the treatment of knee arthritis, and 2.a surgical procedure (bariatric surgery) compared to no surgery for the management of severe obesity. A number of different methodological approaches will be examined and compared (including feature selection methods to identify predictive factors) and propensity score, disease risk scores, instrumental variables, and case-only designs to study the risks and benefits of these treatments. These findings will be supplemented with simulation studies to investigate the performance and robustness of the different design parameters and statistical methods proposed in different scenarios, and to offer solutions to the challenges encountered while analyzing big data.
The project will use routinely collected data from the UK Clinical Practice Research Datalink, Hospital Episode Statistics (HES), and National Joint Registry, as well as bespoke (created ad-hoc) simulated datasets.
The DPhil will be jointly supervised by Associate Prof Prieto-Alhambra, Dr M Sanni Ali, and Dr Sara Khalid, (all members of the NDORMS ‘Musculoskeletal Pharmaco- and Device epidemiology’ research group), and Prof Gary Collins, Professor of Medical Statistics and Deputy Director of the Oxford Centre for Statistics in Medicine (CSM).
Associate Prof Daniel Prieto-Alhambra has published extensively in the field of pharmaco-epidemiology, and is recognised internationally as an authority on use of routine data for musculoskeletal pharmaco- and device epidemiology.
Dr M Sanni Ali is a Senior Research Associate in Pharmaco-epidemiology. He has extensive expertise in the use, validation and development of pharmaco-epidemiological methods, both for the analysis of routinely collected data as well as in simulated datasets.
Dr Sara Khalid is lead analyst for the Health Services Delivery and Musculoskeletal Pharmaco-epidemiology group. She has an Oxford DPhil in Engineering Science, and has an excellent track record and experience in the use of big data methods including machine learning and similar methods.
Prof Gary Collins‘ research interests are focused on methodological aspects surrounding the development and validation of multivariable prediction (prognostic) models and has published widely in this area. He has a particular focus on the role that big data has in evaluating prediction models.
Current DPhil Students within the pharmaco-epidemiology group: 3
The Botnar Research Centre plays host to the University of Oxford’s Institute of Musculoskeletal Sciences and Centre for Statistics in Medicine.
Training will be provided in relevant related research methodology, including the handling and analysis of large datasets, and advanced statistical techniques. Attendance at formal training courses will be encouraged, and will include the “Real world epidemiology Oxford summer school” and the “Advanced musculoskeletal epidemiology UK-RIME summer school”.
In addition, courses from the Oxford Learning Institute and the Oxford University Computer Sciences on key skills for the completion of a successful DPhil thesis will be available. Additional on the job training opportunities will arise, and the supervisors will encourage the student to pursue such opportunities.
A core curriculum of lectures organized departmentally will be taken in the first term to provide a solid foundation in a broad range of subjects including epidemiology, health economics, and data analysis.
Students will attend weekly seminars within the department and those relevant in the wider University.
Students will be expected to present data regularly to the department, the research group and to attend external conferences to present their research globally.
Associate Prof D Prieto-Alhambra: Daniel.prietoalhambra@ndorms.ox.ac.uk
To address a lack of guidance for authors of papers reporting case reports and case series in surgical research Riaz Agha, Department of Plastic Surgery, Guy’s and St. Thomas’ NHS Foundation Trust, London, UK and colleagues have developed 2 consensus based guidelines.
For Case Reports
To cite the guidelines:
Agha RA, Fowler AJ, Saetta A, Barai I, Rajmohan S, Orgill DP, for the SCARE Group. The SCARE Statement: Consensus-based surgical case report guidelines. International Journal of Surgery 2016;34:180-186.
Download the paper and checklist from the SCARE Statement website
Preferred reporting of case series in surgery (PROCESS)
To cite the guidelines:
Agha RA, Fowler AJ, Rajmohan S, Barai I, Orgill DP and the PROCESS Group. The PROCESS Statement: Preferred Reporting of Case Series in Surgery. International Journal of Surgery 2016; (article in press).
Download the paper and checklist from the PROCESS Statement website
“Laurent Lantieri and colleagues report the prospective results of a programme of facial allotransplantation including long-term follow-up of seven transplanted patients (IDEAL Stage 2a) and they ask readers “should we continue?”. As J Rodrigo Diaz-Siso and Eduardo D Rodriguez discuss in their accompanying Comment, the promising outcomes reported in this small case series mean that the focus should now shift to outcome reporting and comparison with the standard(s) of care. This transition would be described as a move from the development and exploration stages to the assessment stage of evaluation by Peter McCulloch and the Balliol Collaboration in the Idea Development Exploration Assessment Long-term study (IDEAL) recommendations for surgical innovation.”
“ A workshop on the future of surgery held at the UK Royal College of Surgeons, London on Sept 20, explored the reasons why the transition from development (2a) and exploration (2b) to assessment (3) of surgical innovations now required by the technique of facial allotransplantation so often proves difficult. The personal and professional investment of individual surgeons in innovations seems to play a key part. In comparison to drug company research, individual surgeons play a greater part in the evaluation of techniques or devices, effectively functioning as gatekeepers for innovation. Recruiting patients to stage 1 or 2 trials requires a familiarity with and a belief in the innovation, such that surgeons might begin to feel equipoise no longer exists, or could continue to iteratively refine the technique, sometimes unnecessarily.”
“Failure to make a timely transition to innovation assessment has many implications: an ethical failure of responsibility to trial participants, a high likelihood of research waste, the risk that a superior technique is ignored, and the possibility that rapid uptake of an attractive new technique—recommended by an eminent surgeon, for example—is such that a randomised comparison is no longer possible.”
“In these circumstances emphasising uncertainty rather than equipoise, which suggests the clinician has equal levels of doubt about the efficacy of the two approaches, can be a way to get even the most polarised surgeons to collaborate on a trial”, said Andrew Cook, lead on surgery and interventional procedures for the UK National Institute of Health Research Health Technology Assessment programme at the Royal College meeting.
Weill Cornell Medical College
Art Sedrakyan
July 15, 2014
“This theme of improving the quality of surgery is also picked up by Art Sedraykan (Vice-Chair, IDEAL Collaboration) and colleagues, who discuss the importance of surgical registries. Rigorous registries are clearly vital for the transparent, long-term study of all new innovations but Sedraykan and colleagues go further, suggesting that almost all surgical patients should be entered into registries. Major national all-inclusive registries provide high-quality feedback to surgeons on their own outcomes, and have a proven role in facilitating the improvement they suggest.”
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(16)31402-7/abstract
(Emphasis and specific addition of IDEAL Stages added by IDEAL)
IDEAL-D: a rational framework for evaluating and regulating the use of medical devices
Article below reproduced with permission from Art Sedrakyan, Weill Cornell Medical College, NY
Every medicine we take has been extensively trialled to prove it is safe and it works, but the same standards do not necessarily apply to medical devices like joint replacements, surgical mesh or pacemakers. Now, a group of IDEAL Collaboration researchers led by Oxford University and Weill Cornell Medicine has proposed a new framework to make sure that such devices get better scrutiny.
Dr. Art Sedrakyan
Photo credit: Carlos Rene Perez
Writing in the June 9 issue of the BMJ, the researchers compare the existing approaches to evaluation of medical devices in both the United States and the European Union — where manufacturers have to prove only that a device is safe to get approval to sell it — with an integrated evaluation pathway for demonstrating that devices are both safe and effective.
“The current systems mean that a device is either unapproved — pre-market stage — or approved — post-market stage,” said Professor Peter McCulloch from Nuffield Dept of Surgical Sciences, University of Oxford (photo right). “Once approval is given, incentives for further study or evaluation become much weaker. Yet, with electronic or mechanical devices implanted in the body, long-term issues are possible and should be assessed.”
“An example is the failure of artificial metal-on-metal hip joints,” added Dr. Art Sedrakyan, a professor of healthcare policy and research at Weill Cornell Medicine. “Pre-clinical tests were enough to get these an approval by demonstrating technical performance, but were never likely to identify the issues of extensive wear, dislocations or pseudo-tumors as these devices were used by patients in the real world. That is why we are proposing a framework that tracks devices throughout the product life cycle, from first design through long-term use.”
The two professors and their co-authors propose using a framework (IDEAL) developed initially for introducing new surgical techniques. IDEAL stands for Idea, Development, Exploration, Assessment and Long term study. Starting with the use of a new technique in just one patient, it can be used to build up evidence of how safe and effective the technique is as it is used by more surgeons with more patients.
Critical to the use of IDEAL for device regulation is the concept that, to get away from a yes-no system, regulators could gradually widen approval by linking it to participation in the appropriate IDEAL study type. This would encourage early stage clinical research, as well as support for registries of devices, allowing performance issues to be tracked. The writers say that this would make evaluation of new devices faster and better, allowing more learning from previous errors, and ultimately allowing products to be available sooner, with better scientific evidence.
“While medical devices need better regulation, wholesale adoption of the approval framework for medicines, including randomised controlled trials, is unlikely to be appropriate,” Dr. Sedrakyan said. “The graduated approach of the IDEAL framework is better placed to support incremental improvements to existing devices.”
“Regulators would get better, safer evidence faster,” McCulloch added, “while patients would get better, safer devices faster, helping to improve health outcomes for a wide range of conditions.”