New reporting guidance for surgical case reports (SCARE) & case series (PROCESS)

To address a lack of guidance for authors of papers reporting case reports and case series in surgical research Riaz Agha, Department of Plastic Surgery, Guy’s and St. Thomas’ NHS Foundation Trust, London, UK and colleagues have developed 2 consensus based guidelines.

For Case Reports

  • CARE Guideline supports transparency and accuracy in publication of case-reports.
  • However, the CARE guidelines are not tailored to surgery.
  • Delphi-consensus exercise was used to develop Surgical CAse REport (SCARE) Guidelines.
  • Journals will be encouraged to endorse the SCARE guideline.

To cite the guidelines:
Agha RA, Fowler AJ, Saetta A, Barai I, Rajmohan S, Orgill DP, for the SCARE  Group. The SCARE Statement: Consensus-based surgical case report guidelines.  International Journal of Surgery 2016;34:180-186. 

Download the paper and checklist from the SCARE Statement website




Agha R, Fowler AJ, Saetta A, Barai I, Rajmohan S, Orgill DP on behalf of the SCARE Steering Group and the Academic Surgical Collaborative.  A Protocol for the Development of Reporting Criteria for Surgical Case Reports: the SCARE Statement.  International Journal of Surgery 2016;27:187-189.  PMID: 26828281.

Preferred reporting of case series in surgery (PROCESS) 

To cite the guidelines:
Agha RA, Fowler AJ, Rajmohan S, Barai I, Orgill DP and the PROCESS Group.  The PROCESS Statement: Preferred Reporting of Case Series in Surgery. International Journal of Surgery 2016; (article in press).

Download the paper and checklist from the PROCESS Statement website


  1. Agha R, Fowler AJ, Lee SY, Gundogan B, Whitehurst K, Sagoo H, Jeong KJL, Altman DG and Orgill DP. A Systematic Review Protocol for Reporting Deficiencies within Surgical Case Series.  BMJ Open 2015;5:e008007.  PMID:26438134.
  2. Agha R, Fowler A, Lee S, Gundogan B, Whitehurst B, Sagoo H, Jeong KJL, Altman D and Orgill D. A Systematic Review of the Methodological and Reporting Quality of Care Series in Surgery.  British Journal of Surgery 2016 (article in press). PMID: 27511619.


Special Surgical Theme issue of The Lancet features IDEAL


“Today’s surgical-themed issue of The Lancet, published ahead of the American College of Surgeons’ Clinical Congress in Washington, DC, USA, (Oct 16–20, 2016), contains three diverse surgical randomised trials (IDEAL Stage 3): early mobilisation compared with standard care for patients in the surgical ICU; lens extraction compared with laser peripheral iridotomy in atients with primary angle-closure glaucoma; and vertebroplasty compared with sham procedure in patients with osteoporotic fractures. All three are success stories of surgical research, supporting changes in practice and offering the possibility to improve the lives of thousands of patients every year.”

“Laurent Lantieri and colleagues report the prospective results of a programme of facial allotransplantation including long-term follow-up of seven transplanted patients (IDEAL Stage 2a) and they ask readers “should we continue?”. As J Rodrigo Diaz-Siso and Eduardo D Rodriguez discuss in their accompanying Comment, the promising outcomes reported in this small case series mean that the focus should now shift to outcome reporting and comparison with the standard(s) of care. This transition would be described as a move from the development and exploration stages to the assessment stage of evaluation by Peter McCulloch and the Balliol Collaboration in the Idea Development Exploration Assessment Long-term study (IDEAL) recommendations for surgical innovation.”

 A workshop on the future of surgery held at the UK Royal College of Surgeons, London on Sept 20, explored the reasons why the transition from development (2a) and exploration (2b) to assessment (3) of surgical innovations now required by the technique of facial allotransplantation so often proves difficult. The personal and professional investment of individual surgeons in innovations seems to play a key part. In comparison to drug company research, individual surgeons play a greater part in the evaluation of techniques or devices, effectively functioning as gatekeepers for innovation. Recruiting patients to stage 1 or 2 trials requires a familiarity with and a belief in the innovation, such that surgeons might begin to feel equipoise no longer exists, or could continue to iteratively refine the technique, sometimes unnecessarily.”

“Failure to make a timely transition to innovation assessment has many implications: an ethical failure of responsibility to trial participants, a high likelihood of research waste, the risk that a superior technique is ignored, and the possibility that rapid uptake of an attractive new technique—recommended by an eminent surgeon, for example—is such that a randomised comparison is no longer possible.”

ACook “In these circumstances emphasising uncertainty rather than equipoise, which suggests the clinician has equal levels of doubt about the efficacy of the two approaches, can be a way to get even the most polarised surgeons to collaborate on a trial”, said Andrew Cook, lead on surgery and interventional procedures for the UK National Institute of Health Research Health Technology Assessment programme at the Royal College meeting.

Weill Cornell Medical College Art Sedrakyan July 15, 2014

Weill Cornell Medical College
Art Sedrakyan
July 15, 2014

“This theme of improving the quality of surgery is also picked up by Art Sedraykan (Vice-Chair, IDEAL Collaboration) and colleagues, who discuss the importance of surgical registries. Rigorous registries are clearly vital for the transparent, long-term study of all new innovations but Sedraykan and colleagues go further, suggesting that almost all surgical patients should be entered into registries. Major national all-inclusive registries provide high-quality feedback to surgeons on their own outcomes, and have a proven role in facilitating the improvement they suggest.”

(Emphasis and specific addition of IDEAL Stages added by IDEAL)

Scientists Propose a Framework for More Comprehensive Assessment of Medical Device Safety and Efficacy – IDEAL-D (devices)

Sedrakyan Art, Campbell Bruce, Merino Jose G, Kuntz Richard, Hirst Allison, McCulloch Peter et al. IDEAL-D: a rational framework for evaluating and regulating the use of medical devices

Article below reproduced with permission from Art Sedrakyan, Weill Cornell Medical College, NY

Every medicine we take has been extensively trialled to prove it is safe and it works, but the same standards do not necessarily apply to medical devices like joint replacements, surgical mesh or pacemakers. Now, a group of IDEAL Collaboration researchers led by Oxford University and Weill Cornell Medicine has proposed a new framework to make sure that such devices get better scrutiny.

Weill Cornell Medical College Art Sedrakyan July 15, 2014Dr. Art Sedrakyan
Photo credit: Carlos Rene Perez

Writing in the June 9 issue of the BMJ, the researchers compare the existing approaches to evaluation of medical devices in both the United States and the European Union — where manufacturers have to prove only that a device is safe to get approval to sell it — with an integrated evaluation pathway for demonstrating that devices are both safe and effective.

Peter McCulloch“The current systems mean that a device is either unapproved — pre-market stage — or approved — post-market stage,” said Professor Peter McCulloch from Nuffield Dept of Surgical Sciences, University of Oxford (photo right). “Once approval is given, incentives for further study or evaluation become much weaker. Yet, with electronic or mechanical devices implanted in the body, long-term issues are possible and should be assessed.”

“An example is the failure of artificial metal-on-metal hip joints,” added Dr. Art Sedrakyan, a professor of healthcare policy and research at Weill Cornell Medicine. “Pre-clinical tests were enough to get these an approval by demonstrating technical performance, but were never likely to identify the issues of extensive wear, dislocations or pseudo-tumors as these devices were used by patients in the real world. That is why we are proposing a framework that tracks devices throughout the product life cycle, from first design through long-term use.”

The two professors and their co-authors propose using a framework (IDEAL) developed initially for introducing new surgical techniques. IDEAL stands for Idea, Development, Exploration, Assessment and Long term study. Starting with the use of a new technique in just one patient, it can be used to build up evidence of how safe and effective the technique is as it is used by more surgeons with more patients.

Critical to the use of IDEAL for device regulation is the concept that, to get away from a yes-no system, regulators could gradually widen approval by linking it to participation in the appropriate IDEAL study type. This would encourage early stage clinical research, as well as support for registries of devices, allowing performance issues to be tracked. The writers say that this would make evaluation of new devices faster and better, allowing more learning from previous errors, and ultimately allowing products to be available sooner, with better scientific evidence.

“While medical devices need better regulation, wholesale adoption of the approval framework for medicines, including randomised controlled trials, is unlikely to be appropriate,” Dr. Sedrakyan said. “The graduated approach of the IDEAL framework is better placed to support incremental improvements to existing devices.”

“Regulators would get better, safer evidence faster,” McCulloch added, “while patients would get better, safer devices faster, helping to improve health outcomes for a wide range of conditions.”

See also: Pennell CP, Hirst A, Sedrakyan A, McCulloch PG. Adapting the IDEAL Framework and Recommendations for medical device evaluation: A modified Delphi survey. Int J Surg. 2016 Apr;28:141-8. doi: 10.1016/j.ijsu.2016.01.082. Epub 2016 Feb 11.Review. PubMed PMID: 26876957.


2016 NCRI Future of Surgery Workshop 2 – 20th September 2016, RCS, London

“Technology Trials in Surgical Oncology”: What evidence is required prior to introduction of new technologies into surgical practice?”

Tues 20 Sep 2016: 10:00–16:15 
Royal College of Surgeons of England, 35-43 Lincoln’s Inn Fields, London, WC2A 3PE

More about this event

This event is being organised by the National Cancer Research Institute (NCRI)

Register now to shape the future of surgical trials at this multidisciplinary NCRI/RCS event, led by Mr Stephen Price. We welcome surgical consultants, trainees, patients, methodologists, trial unit staff and funders to this workshop which will provide guidance on introducing new technologies into surgical practice.

Key topics include:

  • The IDEAL Framework
  • Introducing and incentivising surgeons to evaluate new technologies
  • Regulation of medical devices
  • Parallel sessions:
    • Early Phase Surgical Trials
    • Robotics in Surgical Oncology

Peter McCulloch, Chair of IDEAL will be a speaker at this event. Peter McCulloch

» Flyer for NCRI Future of Surgery workshop, 20 Sept 2016 (PDF)

email Victoria Murphy with any queries.

Capacity issues in UK surgical research

Surgery is an important part of many cancer patients’ journeys, and the NCRI 2008–2013 strategic plan raised concerns about the UK’s capacity to undertake surgical research. Analysis of the NCRI’s Cancer Research Database showed a low overall volume of surgical research in cancer. This parallels the situation in surgery beyond cancer, as the Royal College of Surgeons of England (RCS England) found in their 2011 publication.

» RCS England report on overcoming barriers to innovation in surgery, 2011 (PDF)

NCRI has launched the Future of Surgery workshop series, bringing together experts to deal with challenging, cross-specialty topics and influence the future of surgery research in cancer. Five workshops will run over the next two years, each producing a report or publication that brings together expert opinion and identifies potential next steps for surgery research in cancer. The first workshop, on outcome measures in surgery studies, appeared in an editorial in The Lancet.

» The Lancet editorial, “The struggle for better research in surgery” (external website)

The second workshop of the series will take place on 20 September 2016. For more information and to register, visit the link below.

» 2016 NCRI Future of Surgery Workshop “Technology Trials in Surgical Oncology”: What evidence is required prior to introduction of new technologies into surgical practice?

The workshops will not be one-way education from those speaking to those attending. Instead, they will take the form of expert working meetings, drawing on the expertise of all those attending to build consensus, identify gaps in knowledge, or propose ways that things could be done differently in surgery research in cancer. Workshop leads have now been appointed. The five workshop topics and dates are:

  1. 4 May 2016: Outcome measures in surgery studies, led by Mr Angus McNair.  Looking beyond overall survival, what are the key surgical endpoints? Powering studies to make endpoints statistically meaningful and change surgical practice.
    » “Trials are only as credible as their endpoints”: Defining the future outcomes surgical research
  2. 20 September 2016: Technology trials in surgery, led by Mr Stephen Price. Discussing the evidence required prior to introduction of new technologies into surgical practice. Ways to incentivise surgeons to evaluate new technologies.
    » “Technology Trials in Surgical Oncology”: What evidence is required prior to introduction of new technologies into surgical practice?
  3. 23 November 2016: Selecting patients for surgery, led by Ms Lynda Wyld. Research into biomarkers/ algorithms required to predict benefit and identify which patients will benefit from surgery. Frailty in cancer surgery – how can surgeons design and select tools for clinical application, and determining prevalence of frailty?
    » Selecting patients for surgery
  4. 17 January 2017: Extent of surgery, led by Mr Stuart McIntosh. Determining the need for primary surgery after neoadjuvant treatment. Current standard of care and changing practice. Tissue banking and making the most of biopsies.
  5. 21 March 2017: Surgery for metastatic disease, led by Mr Prasanna Sooriakumaran & Mr Hassan Malik. Resection of metastatic disease. Surgery vs systemic therapy.

The plans for each workshop is being overseen by the NCRI Future of Surgery steering committee, which includes senior research surgeons and an NCRI representative, to ensure it fits with the overall programme of events. NCRI Future of Surgery steering committee.


Joint Prize-Winners for Best Poster at IDEAL Collaboration Conference 2016

The IDEAL Conference (Final Brochure_AH06Apr16 and see #IDEAL2016 on Twitter) held at St Catherine’s College, Oxford on 7th April closed with a presentation for Best Poster.

The Judging Team comprising Riaz Agha (Balliol College, Oxford and SpR Plastic Surgery) & Natalie Blencowe (NIHR Clinical Lecturer in Surgery, University of Bristol) awarded the Prize jointly to authors of Poster (0006) and (0030)

(0006) A Systematic Review of the Methodological and Reporting Quality of Case Series in Surgery

20160407_175510_Riaz_0006Riaz Agha 1, Alexander Fowler 2, Seon-Young Lee 3, Buket Gundogan 4, Katharine Whitehurst 4, Harkiran Sagoo 5, Kyung Jin Lee Jeong 6, Douglas Altman 7, Dennis Orgill 8

1 Guy’s and St. Thomas’ NHS Foundation Trust and Balliol College, University of Oxford, London & Oxford, UK, 2 Guys and St Thomas’ NHS Foundation Trust, London, UK, 3 Southampton Medical School, Southampton, UK, 4 University College London, London, UK, 5 GKT School of Medical Education, King’s College London, London, UK, 6 Norfolk and Norwich University Hospital, Norfolk, UK, 7 Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK, 8 Division of Plastic Surgery, Brigham and Women’s Hospital, Boston, USA


(0030) Autologous plasma coating improves the biocompatibility of mesh implants. On the IDEAL way from bench to bedside.


 Holger Gerullis 1,2,  Dimitri Barski 1, Thorsten Ecke 3, Christoph Eimer 1, Mihaly Boros 3, Bernd Klosterhalfen 4, Albert Ramon 5, Thomas Otto 1

1 Lukas Hospital Neuss, Neuss, Germany, 2 University of Oldenburg, Oldenburg, Germany, 3 University of Szeged, Szeged, Hungary, 4 German Centre for Implant-Pathology, Düren, Germany, 5 ITERA International Tissue Engineering Research Association, Antwerpen, Belgium

Practical guide to the Idea, Development and Exploration stages of the IDEAL Framework and Recommendations

Now available – our guide to identifying and using the pre-RCT stages of IDEAL

Early View Article in British Journal of Surgery

Br J Surg. 2016 Feb 10. doi: 10.1002/bjs.10115. [Epub ahead of print]

Practical guide to the Idea, Development and Exploration stages of the IDEAL Framework and Recommendations.

Pennell CP1, Hirst AD2, Campbell WB3, Sood A4, Agha RA5, Barkun JS6, McCulloch P2
1Department of Surgery, Maimonides Medical Center, Brooklyn, New York. 2IDEAL Collaboration, Nuffield Department of Surgical Science, Oxford, UK. 3Interventional Procedures and Medical Technologies Advisory Committees, National Institute for Health and Care Excellence, London, UK. 4Vattikuti Urology Institute, Henry Ford Hospital, Detroit, Michigan, USA. 5Department of Plastic Surgery, Guy’s and St Thomas’ NHS Foundation Trust, London, UK. 6Department of Surgery, Division of Hepatobiliary and Transplant Surgery, McGill University Health Centre, Royal Victoria Hospital., Montreal, Quebec, Canada.



Evaluation of new surgical procedures is a complex process challenged by evolution of technique, operator learning curves, the possibility of variable procedural quality, and strong treatment preferences among patients and clinicians. Preliminary studies that address these issues are needed to prepare for a successful randomized trial. The IDEAL (Idea, Development, Exploration, Assessment and Long-term follow-up) Framework and Recommendations provide an integrated step-by-step evaluation pathway that can help investigators achieve this.


A practical guide was developed for investigators evaluating new surgical interventions in the earlier phases before a randomized trial (corresponding to stages 1, 2a and 2b of the IDEAL Framework). The examples and practical tips included were chosen and agreed upon by consensus among authors with experience either in designing and conducting IDEAL format studies, or in helping others to design such studies. They address the most common challenges encountered by authors attempting to follow the IDEAL Recommendations.


A decision aid has been created to help identify the IDEAL stage of an innovation from literature reports, with advice on how to design and report the IDEAL study formats discussed, along with the ethical and scientific rationale for specific recommendations.


The guide helps readers and researchers to understand and implement the IDEAL Framework and Recommendations to improve the quality of evidence supporting surgical innovation.

BOSTiC 2016: Training in research methods for surgical trainees

Wednesday, 27 April 2016 to Friday, 29 April 2016, 9am – 4pm
University of Bristol
Hosted by Oxford and Bristol Surgical Trials Centres

The Oxford and Bristol Surgical Trials Units will be running a three-day surgical research training event (BOSTiC) for surgical trainees in Spring 2016 (Wednesday 27 April to Friday 29 April inclusive) at the University of Bristol. Research in surgery is challenging and requires multi-disciplinary collaboration with methodologists and clinicians to design and conduct high quality studies. The overall aim of all our training programmes is to inspire surgical trainees to learn and practice high quality research that will be relevant to surgical care of patients in the NHS. This course is designed for surgical trainees from all specialties wanting to learn about trials methodology, and to be involved in high quality research projects.

The programme consists of three back-to-back research training days and provides the opportunity for trainees to learn how to evaluate surgical interventions in well designed and conducted studies.

For more information on the event and how to apply please click here.


EFGCP Annual Conference 2016: 1 & 2 March 2016, Brussels, Belgium

Faster and Safer Paths to New Treatments in Medicines and Medical Devices
1 & 2 March 2016, Crowne Plaza Le Palace, Brussels, Belgium

Building faster but safe paths to new treatments: How do we safely develop new medicines and devices?

For 22 years the European Forum for Good Clinical Practice (EFGCP) has acted as a forum to bring patients, researchers, sponsors, competent authorities and ethics committees together and debate current topics. It is one of the most important platforms available for multi stakeholder debate.

The IDEAL Collaboration is happy to support this conference which aims to help the two worlds of pharma and devices develop research, medicines and devices for patient benefit. The EU Medical Device regulations are almost in effect and that will have significant consequences, similar to the Clinical Trial Regulations. The EFGCP Annual Conference 2016 will outline what impact the regulations will have on the devices industry and learn from the experience of pharma. The tangible outcomes will be an EFGCP report and recommendations that will be provided to those involved in the legislative process.

Peter McCulloch, Maroeska Rovers and Allison Hirst will be running an IDEAL workshop as part of the conference on “Lessons from Surgeons” to contribute to the report and recommendations. For the full programme please click here.

EFGCP Annual Conference 2016
Registration infomration.

Interventions in randomised controlled trials in surgery: issues to consider during trial design

Natalie B 

Natalie Blencowe, NIHR Clinical Lecturer in Surgery @NatalieBlencowe 


                          Jane B

and Jane Blazeby, Professor of Surgery, @BlazebyJ 

from the Bristol Surgical Trials Centre, UK discuss what you should consider when designing the intervention in your surgical trial.

“There are many reasons why trials involving surgery can be more difficult to design and conduct than drug trials. One is that “…unlike 20 milligram tablets, no two surgical interventions are the same.” An operation for oesophageal cancer, for example, can take all day and may involve opening two or three body cavities, or using keyhole techniques. It is common for different surgeons to perform this operation in different ways and/or for different hospitals to have a preference for one approach over another. There are also lots of other interventions that are delivered before, during or after the operation. For example invasive monitoring devices are inserted beforehand, and patients usually spend several days on a high care unit. The number of people involved with a patients care is also high, with patients receiving input from intensive care doctors, nurses, dieticians and physiotherapists. It’s therefore not hard to see why surgical interventions are considered to be complex.

This complexity has implications for trial design. To help with this, guidance for developing and evaluating complex interventions (such as surgery) suggests that the delivery of interventions are standardised. This sounds intuitive, helping to ensure that the interventions in each trial group are actually different from each other, and that there is enough information for effective interventions to be replicated in practice. The reality of achieving this ‘standardisation’, however, is complex in itself. Should every detail of the operation – including the anaesthetic and aftercare – be standardised? This may require surgeons to follow hundreds (if not thousands) of specific instructions, some of which they may not agree with. Such a prescriptive approach may dilute the generalisability of the trial results to the ‘real world’. The solution to this problem might be to strike a balance between ‘enough’ standardisation for trial results to be believed, and the practicality of achieving it.

So, how much standardisation is enough? It may be helpful for trialists to consider some key questions at the trial design stage (see below), such as the ‘who, what, where, how’. To answer these questions, the bigger picture – the trial’s main purpose – needs to be considered. For example, a multicentre pragmatic study – intended to establish whether an intervention is effective under usual conditions – may require very little standardisation at all. Conversely, a trial involving novel procedures may involve detailed descriptions and rigorous monitoring of whether they were delivered as intended. Either way, taking time to consider these aspects of surgical interventions during trial design, and documenting them within the protocol, may go some way towards helping successful interventions to be better adopted into clinical practice.”

 Questions to ask during the design of surgical interventions in RCTs

1.   Does the RCT involve a surgical intervention?

2.   What is/are the surgical intervention(s) under evaluation?

3.   What is/are the co-interventions accompanying the surgical intervention?

4.   How will the intervention(s) be standardised in the RCT?

5.   How will delivery of the intervention(s) be monitored?

6.   Who will deliver the intervention(s)?

7.   Where will the intervention(s) be delivered?

A recent paper co-authored by Natalie and Jane in Trials 2015, 16:392 raises these important issues and provides some guidance about addressing these challenges.

For further information contact Natalie by email

Bristol Unit





The PRECIS-2 tool: designing trials that are fit for purpose


Dr Kirsty Loudon, Nursing Midwifery and Allied Health Professionals Research Unit, University of Stirling, UK explains the rationale behind PRECIS-2.


Designing clinical trials is challenging. PRECISPRagmatic Explanatory Continuum Indicator Summaries – is a clever acronym for a tool to help trialists designing clinical trials consider where they would like their trial to be on the pragmatic “usual care”/explanatory “ideal care” continuum.

Pragmatic trials were first proposed by Schwartz and Lellouch in 1967 as trials performed under normal conditions with the intention of providing results that are more applicable to clinical practice and decision making. The alternative, taking a more explanatory approach, leads to tightly controlled trials under ideal conditions that aim to provide understanding of how treatments work. Explanatory trials have an important role but healthcare interventions are seldom given under circumstances similar to those used in such trials.

A large number of stakeholders including patients, funders of healthcare and of research, health care professionals who deliver clinical care and trialists as well as guideline developers are becoming increasingly interested in trials with greater applicability. There is an urgent need to produce trials results to improve healthcare from interventions that have been designed to be used in the real world – matching design decisions to the real world of how trial results are intended to be used.

Over 80 international trialists, clinicians and policymakers have developed PRECIS-2 from the original PRECIS tool, keeping the wheel format. The trial design tool has 9 domains: eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis. There is a scoring system of 1 (very explanatory) to 5 (very pragmatic). Trial teams can use PRECIS-2 to guide discussion on how explanatory or pragmatic they wish their trials to be and to guide decision making on the design of trial domains. PRECIS-2 thus enables trialists to consider how applicable their trial results will be in the “real world”.

PRECIS-2 Wheel

The PRagmatic-Explanatory Continuum Index Summary 2 – PRECIS-2 wheel. BMJ 2015;350:h2147  (Reproduced with permission)



PRECIS-2 is part of the Trial Forge initiative and could be an important part of the trial design pathway – from the first step of determining how to design the research intervention by considering the end user of the trial results at the outset.

More information:  Dr Kirsty Loudon, email:, Twitter @KirstyLoudon

Watch the podcast on PRECIS-2:

1. Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., et al. (2009). A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. Journal of Clinical Epidemiology, 62(5), 464-475.