Systematic Review of key features of successful surgical registries

A breast implant“There is a clear need for surgical registry data to improve patient safety and help regulate surgical practices.The IDEAL collaborative, Department of Health (DOH), the National Institute for Health and Care Excellence (NICE), policymakers and commissioning groups have called for surgical registries that can collect prospective outcome and safety data, promote transparency as well as provide patients and the public with information on their care. However, developing and maintaining a registry faces considerable challenges with the majority of registries failing.

To help address this, we conducted a systematic review and narrative synthesis, learning from existing and previous registries, to identify the fundamentals to successful registry development. We found that the key factors for successful registry development include:  steering committee to lead and oversee the registry; clear registry objectives; planning for initial and long-term funding; strategic national collaborations among key stakeholders; dedicated registry management team; consensus meetings to agree registry dataset; established data processing systems; anticipating challenges; and implementing strategies to increase data completion. Patient involvement and awareness of legal factors should occur throughout the development process.

Our work provides robust knowledge that can be used to inform the successful development of any UK surgical registry. It also provides a methodological framework for international surgical registry development”

Mr Rishi Mandavia , NIHR Academic Clinical Fellow ENT Surgery and NICE Specialist Advisor, evidENT Team, Ear Institute, University College London

Mandavia R, Knight A, Phillips J, et al  What are the essential features of a successful surgical registry? a systematic review.

Link to the paper here:


New reporting guideline STROCSS extends STROBE for surgery-specific research studies


“For many years I saw articles compliant with the STROBE guidelines but still missing large amounts of key information relevant to surgeons.  Such key information centred around the intervention itself is paramount to understanding what’s been done in a study, critically appraising and replicating it. Readers need complete, clear and transparent reporting and we hope the STROCSS guideline goes some way towards this.”

Dr Riaz Agha, Managing and Executive Editor, International Journal of Surgery

The full guideline and checklist for reporting surgical cohort, cross-sectional and case-control studies is available on the STROCSS website:

Agha RA, Borrelli MR, Vella-Baldacchino M, Thavayogan R, Orgill DP; STROCSS Group. The STROCSS statement: Strengthening the Reporting of Cohort Studies in Surgery. Int J Surg. 2017;46: 198–202

The STROCSS checklist can be downloaded here: the_strocss_statement_checklist

Peter McCulloch talks about IDEAL at Evidence Live 2017

The IDEAL team Claudia Ashton, Allison Hirst, Claire Thomson and Peter McCulloch  presented a workshop on using IDEAL to appraise the surgical literature at #EvidenceLive on 21st June 2017

Following the workshop Peter McCulloch, Chair of the IDEAL Collaboration discussed some of the challenges of surgical research with Douglas Badenoch @Mental_Elf




Educational Workshop 4th May 2017, NY: Critical appraisal using IDEAL

As part of our 2017 Conference in New York, 4th-5th May 2017, we are holding a half day educational workshop. Please select this at the time of registration for the conference at

Early registration discount available until April 1st   Registration fees include breakfast and lunch each day of the conference, and CME credits will be available for physicians. For further information, contact: or visit

Workshop 4th May 2017

Critical appraisal using the IDEAL Framework (provisional timetable)

8.30—9.15AM (JH)  History of EBM; Inherent difficulties of surgical evaluation; lack of formal education among trainees in US

9.15—10.00AM (JF)  Group workshop – Systematic approach to the Critical Appraisal of a Surgical Paper

  • Short lecture explaining the approach to appraising a surgical paper using GATE framework
  • Using an RCT – group(s) will appraise the paper

10.00—10.30AM               Coffee/Tea

10.30—11.00AM (CP)  IDEAL: Framework and How to utilize it

  • Describe the purpose of IDEAL and why it is necessary
  • Explain each stage and the relevant questions at each stage

11.00AM – 11.30AM  Group workshop – Identifying the IDEAL stage of innovation

  • Group given a surgical article and asked to identify the IDEAL stage described in the paper

11.30 – 12.00PM (PM)  Using IDEAL: Progressing through the life-cycle of surgical Innovation

  • As a large group, we take a surgical procedure through the IDEAL stages of innovation and display how each stage would be designed

12.00—12.15PM (JF)  IDEAL: Current status and future

  • Update since conference last year
  • Applications to devices and other sectors

12:15 – 12.30 (JF)  Assessment of participants’ comprehension of IDEAL

  • Short questionnaire asking participants about Framework & Recommendations
  • Will provide handouts on IDEAL and critical appraisal

Teaching Faculty

JH – Joel Horovitz, Vice Chair of Surgery, Maimonides Medical Center

JF – Joshua Feinberg, Maimonides Medical Center

CP – Christopher Pennell, Maimonides Medical Center

PM – Peter McCulloch, Chair IDEAL Collaboration, University of Oxford

Where is the evidence for novel surgical devices?

This question was recently addressed and the findings published in the British Journal of Surgery

Two of the authors describe what they did and what they found;


Stephen J Chapman, Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, UK




Aneel Bhangu, Dept of ColorectalSurgery, University of Birmingham, Birmingham, UK


Innovation in surgery is important to ensure the treatment of surgical disease continues to advance. The role of industry is paramount in driving this forward, but must be matched by robust clinical evidence and dedicated regulation.

Whilst regulatory bodies in the U.K. work to ensure safe introduction of surgical devices, many are supported by minimal or no evidence of safety or effectiveness. We are acutely aware of this problem; new surgical devices are introduced every day yet these may be unfamiliar or unproven. Whilst regulatory bodies are often satisfied with evidence of “equivalence”, the true clinical impacts of these are unclear.

Our study, published in the British Journal of Surgery, demonstrated the extent of this problem (1). We systematically compiled a list of implantable devices used in gastrointestinal surgery and set out to find evidence to support their use. Alarmingly, only 1 in 10 of these devices was supported by evidence from high quality randomised controlled trials (RCT).

Whilst RCTs are perceived to slow the rate of innovation, dedicated networks of surgeons have proven their ability to evaluate surgical interventions quickly and effectively (2). Additionally, whilst RCTs are not always the most appropriate design to test novel interventions, they provide high quality evidence when undertaken by dedicated clinical and academic networks.

In line with proposals from the IDEAL Collaboration, we encourage a standardised, step-wise approach to the assessment of novel surgical technology. This should begin with first in man studies, progress through stages of randomised assessment, and culminate in long term surveillance in dedicated devices registries.


  1. Chapman SJ. Shelton B. Maruthappu M. Singh P. McCulloch P. Bhangu A. Cross-sectional observational study of the availability of evidence supporting novel implantable devices used in gastrointestinal surgery. Br J Surg 2017 [Epub ahead of print]. doi: 10.1002/bjs.10485
  2. Bhangu A. Kolias AG. Pinkney T. Hall NJ. Fitzgerald JE. Surgical research collaboratives in the UK. Lancet 2013;382:1091-2.

Opportunities to develop collaborative surgical projects at IDEAL 2017 Conference



Would you like to participate in the development of major new scientific studies to evaluate your work?

If so, the 2017 IDEAL MEETING in New York offers a unique opportunity.  During the SURGICAL SPECIALITY THINK TANKS ON 4TH MAY IDEAL 2017 Provisional Agenda  we’ll be seeking to initiate multi-centre prospective collaborative studies of one new cutting edge technique in each workshop.  These will use the IDEAL 2b PROSPECTIVE COHORT format and the intention is that the initial study will form the basis for a subsequent major RCT.  If you would like to get in on the ground floor of one of these collaborative studies by surgical innovators, register for the meeting and send your ideas to:

Registration is now open at – early registration discount available until April 1st 2017! Registration fees include breakfast and lunch each day of the conference, and CME credits will be available for physicians. For further information, contact: or visit

This is an opportunity to help plan important studies which will shape the direction of future practice in your speciality.  The IDEAL Collaboration contains expert methodologists, statisticians and others who can help guide the development of research groups and reduce the burden of participation on clinicians to a minimum. 

We look forward to meeting you and working together on these exciting projects.

Prof Peter McCulloch, IDEAL Chair, Oxford (left)


  Prof Art Sedrakyan, IDEAL    Vice-Chair, New York (right)

Abstracts from our 2016 Conference now published in International Journal of Surgery

Abstracts from our 2016 Conference are now available in  the International Journal of Surgery

December 2016 Volume 36, Supplement 2, S133-S144

Access to innovative treatments and device support during IDEAL stages 2b-4 K. Hutchison

Advancing the cause of research registration: The first 500 registrations of the R. Agha, A.J. Fowler, C. Limb, Y. Al Omran, H. Sagoo, K. Koshy, D.J. Jafree, M.O. Anwar, Peter McCulloch, D.P. Orgill

Surgical trainee research collaboratives in the United Kingdom A.G. Kolias, A.A.B. Jamjoom, P.N.H. Phan, P.J.A. Hutchinson

The merits of decision modeling in the earliest stages of the IDEAL framework – The case of innovative bilateral DIEP flap surgery J. Grutters, J. Gerrits, L. Schultze Kool, D. Ulrich, M. Rovers, S. Hummelink

Evaluation of innovation in radiation oncology: R-IDEAL H. Verkooijen, L. Kerkmeijer, D. Fuller, Robbert Huddart, C. Faivre-Finn, M. Verheij, A. Sahgal, E. Hall, M. van Vulpen

Small simple trials: A strategy to study rare surgical condition J.G. Wright

Short-term cost-effectiveness of colonic stenting is lost at 90-day follow-up in patients with malignant large bowel obstruction  H. Yeo, J. Abelson, J. Milsom, S. Sharma, A. Sedrakyan

3-Year outcomes and cost-savings of combined endoscopic laparoscopic surgery (CELS) for benign colon polyps M. Kiely, S. Sharma, A. Sedrakyan, J. Yoo, H. Yeo, J. Abelson, J. Milsom

Stereotactic body radiotherapy followed by surgery for unstable spinal metastases: Technical feasibility and safety study according to the IDEAL stages 1 and 2a.  A. Versteeg, J. van der Velden, H. Verkooijen, W. Eppinga, N. Kasperts, S. Gerlich, C. Oner, M. van Vulpen, J.-J. Verlaan

Perioperative outcomes, health care costs and survival after robotic-assisted versus open radical cystectomy: A national comparative effectiveness study  J. Hu, B. Chughtai, P. O’Malley, J. Halpern, J. Mao, D. Scherr, D. Hershman, J. Wright, A. Sedrakyan

Comparative effectiveness of cancer control and survival after robotic assisted versus open radical prostatectomy J. Hu, B. Chughtai, P. O’Malley, A. Isaacs, J. Wright, D. Hershman, A. Sedrakyan

Reconstruction of bladder defects with amniotic membrane – IDEAL-D Stage 0-1  D. Barski, H. Gerullis, A. Winter, I. Pintelon, J.-P. Timmermans, A. Ramon, M. Boros, G. Varga, T. Otto

Is more evidence always better? The value of adding decision analytical modeling to the IDEAL framework C. Tax, P.H.M. Govaert, M. Stommel, M.G.H. Besselink, H.G. Gooszen, J. Grutters, M.M. Rovers

Progressing through IDEAL: When is the right time to move from observational to randomised studies? – A case study of REBOA J.O. Jansen, M.K. Campbell

MiCollar – A novel iPhone application to analyse cervical spine motion restriction with different size and type of cervical orthoses R. Ingleton, C. Ashton, J. Bull, V.N. Vakhari

The X-Bolt dynamic hip plating system: Evaluating a novel surgical device for hip fracture surgery  M. Fernandez, J. Achten, N. Parsons, X.L. Griffin, M.L. Costa

Radical cystectomy in epidural anaesthesia – Feasibility analysis using a new reporting method following the IDEAL recommendations H. Gerullis, T.H. Ecke, C. Bantel, A. Weyland, D. Barski, T. Jansen, J. Uphoff, F. Wawroschek, A. Winter

Applying IDEAL: Early stage surgical innovation of a novel bio-wrap-assisted vasectomy reversal technique  A. Gudeloglu, J. Brahmbhatt, S. Parekattil

Matching trial design decisions to the needs of those you hope will use the results: The PRECIS-2 tool  K. Loudon, M. Zwarenstein, F. Sullivan, P. Donnan, S. Treweek

The GASTROS Study: Standardising outcome reporting in gastric cancer surgery research  Bilal Alkhaffaf, Anne-Marie Glenny, Jane Blazeby, Paula Williamson, Iain Bruce

Support for reporting guidelines in surgical journals needs improvement: A systematic review  R. Agha, I. Barai, S. Rajmohan, S.Y. Lee, M. Anwar, A.J. Fowler, D. Orgill, D. Altman

Compliance of systematic reviews in plastic surgery with the PRISMA statement: A systematic review  S.Y. Lee, H. Sagoo, K. Whitehurst, G. Wellstead, A.J. Fowler, R. Agha, D. Orgill

A systematic review of the methodological and reporting quality of case series in surgery  R. Agha, A.J. Fowler, S.Y. Lee, B. Gundogan, K. Whitehurst, H. Sagoo, K.J.L. Jeong, D. Altman, D. Orgill

An assessment of the compliance of systematic review articles published in craniofacial surgery with the PRISMA statement guidelines: A systematic review  T.E. Pidgeon, G. Wellstead, H. Sagoo, D.J. Jafree, A.J. Fowler, R. Agha

The use of study registration and protocols in plastic surgery research: A systematic review  T.E. Pidgeon, C. Limb, R. Agha, K. Whitehurst, C. Chandrakumar, G. Wellstead, A.J. Fowler, D. Orgill

Twist-drill craniostomy with hollow screws for evacuation of chronic subdural haematoma  A.G. Kolias, A. Chari, S.J. Bond, P.J.A. Hutchinson

Intra-operative hyperspectral imaging for brain tumour detection and delineation: Current progress on the HELICoid project S. Kabwama, D. Bulters, H. Bulstrode, H. Fabelo, S. Ortega, G.M. Callico, B. Stanciulescu, R. Kiran, D. Ravi, A. Szolna, J.F. Piñeiro

Developing a patient and public involvement intervention to enhance recruitment and retention in UK surgical trials (PIRRIST)  J.C. Crocker, S. Rees, L. Locock, S. Petit-Zeman, A. Chant, S. Treweek, J.A. Cook, N. Farrar, K. Woolfall, J. Bostock, L. Bowman, R. Bulbulia

Osseointegrated joint replacement connected to a lower limb prosthesis: An IDEAL stage 1 proof of concept study with 7 cases  A. Khemka, S. Lord, M. Al Muderis

The East Grinstead Consent Collaborative (EGCC) online consent creator J. Dhanda, M. Dungarwalla

Development of an online platform for registration and outcome measurement of urogynecological implants according to IDEAL-system D. Barski, H. Gerullis, T. Ecke, R. Joukhadar, J. Kranz, R. Tahbaz, F. Queissert, L. Schneidewind, S. Mühlstädt, M. Grabbert, N. Huppertz, A.E. Pelzer, U. Klinge, M. Boros, W. Bader, F. Puppe, T. Otto

Bringing laparoscopy to your own home M. Vella-Baldacchino, M. Vella Baldacchino, M. Schembri, R. Bugeja

Autologous plasma coating improves the biocompatibility of mesh implants. On the IDEAL way from bench to bedside  H. Gerullis, D. Barski, T.H. Ecke, C. Eimer, M. Boros, B. Klosterhalfen, A. Ramon, T. Otto

Percutaneous nephrolithotomy (PCNL) under local infiltrative anesthesia with and without stand-by anesthesia – Complication rates and clinical outcome for a method at the exploration stage according to IDEAL T.H. Ecke, G. Weingart, C. Lange, S. Hallmann, F. Wawroschek, D. Barski, J. Ruttloff, H. Gerullis

The landscape of surgical innovation in robotic microsurgery A. Gudeloglu, B. Ozdemir, S.R. Fleischman, P. Dahm

Awareness of the ideal recommendations among the members of the Robotic-Assisted Microsurgery and Endoscopic Society (RAMSES)  A. Gudeloglu, S.R. Fleischman, P. Dahm

DPhil place available at NDORMS, Oxford: Big data methods and evaluation of surgical interventions

primary-logoProject outline

Surgical interventions can have profound effects on people’s lives, both in terms of benefits and (potentially) in terms of risks or complications. It is therefore essential that surgical interventions should be evaluated appropriately and accurately to inform clinical decisions.

Although randomized controlled trials (RCT) provide gold standard evidence, surgical RCTs are expensive, challenging (given pragmatic, ethical, and other considerations), and hence scarce in the scientific literature. In addition, RCTs tend to use restrictive eligibility criteria therefore limiting the external validity (generalizability) of their findings to relatively healthy populations.

Conversely, routinely collected data (i.e. anonymized data from national registries, audits, and computerized NHS records) provide a unique opportunity to evaluate surgical interventions in potentially any NHS user, widening our ability to gather information on the performance of surgical and medical alternatives in different patient groups. However, given the observational nature of such studies, confounding (by indication) is a major challenge that needs to be accounted for to avoid biased results. Methods exist to minimize such confounding and other biases, which are widely used in pharmaco-epidemiology (drug safety studies), but these have been seldom tested for the study of surgical treatments.

The project will focus on two clinical scenarios for the risk-benefit evaluation of 1.two different surgical alternatives (unicompartmental vs total knee replacement) for the treatment of knee arthritis, and 2.a surgical procedure (bariatric surgery) compared to no surgery for the management of severe obesity. A number of different methodological approaches will be examined and compared (including feature selection methods to identify predictive factors) and propensity score, disease risk scores, instrumental variables, and case-only designs to study the risks and benefits of these treatments. These findings will be supplemented with simulation studies to investigate the performance and robustness of the different design parameters and statistical methods proposed in different scenarios, and to offer solutions to the challenges encountered while analyzing big data.

The project will use routinely collected data from the UK Clinical Practice Research Datalink, Hospital Episode Statistics (HES), and National Joint Registry, as well as bespoke (created ad-hoc) simulated datasets.

Research Group

The DPhil will be jointly supervised by Associate Prof Prieto-Alhambra, Dr M Sanni Ali, and Dr Sara Khalid, (all members of the NDORMS ‘Musculoskeletal Pharmaco- and Device epidemiology’ research group), and Prof Gary Collins, Professor of Medical Statistics and Deputy Director of the Oxford Centre for Statistics in Medicine (CSM).

Associate Prof Daniel Prieto-Alhambra has published extensively in the field of pharmaco-epidemiology, and is recognised internationally as an authority on use of routine data for musculoskeletal pharmaco- and device epidemiology.

Dr M Sanni Ali is a Senior Research Associate in Pharmaco-epidemiology. He has extensive expertise in the use, validation and development of pharmaco-epidemiological methods, both for the analysis of routinely collected data as well as in simulated datasets.

Dr Sara Khalid is lead analyst for the Health Services Delivery and Musculoskeletal Pharmaco-epidemiology group. She has an Oxford DPhil in Engineering Science, and has an excellent track record and experience in the use of big data methods including machine learning and similar methods.

Prof Gary Collins‘ research interests are focused on methodological aspects surrounding the development and validation of multivariable prediction (prognostic) models and has published widely in this area. He has a particular focus on the role that big data has in evaluating prediction models.

Current DPhil Students within the pharmaco-epidemiology group: 3


The Botnar Research Centre plays host to the University of Oxford’s Institute of Musculoskeletal Sciences and Centre for Statistics in Medicine.

Training will be provided in relevant related research methodology, including the handling and analysis of large datasets, and advanced statistical techniques. Attendance at formal training courses will be encouraged, and will include the “Real world epidemiology Oxford summer school” and the “Advanced musculoskeletal epidemiology UK-RIME summer school”.

In addition, courses from the Oxford Learning Institute and the Oxford University Computer Sciences on key skills for the completion of a successful DPhil thesis will be available. Additional on the job training opportunities will arise, and the supervisors will encourage the student to pursue such opportunities.

A core curriculum of lectures organized departmentally will be taken in the first term to provide a solid foundation in a broad range of subjects including epidemiology, health economics, and data analysis.

Students will attend weekly seminars within the department and those relevant in the wider University.

Students will be expected to present data regularly to the department, the research group and to attend external conferences to present their research globally.

Further information

Associate Prof D Prieto-Alhambra: