DECIDE-AI

Developing new reporting guidelines for the early-stage clinical evaluation of decision support systems driven by artificial intelligence.

Much like IDEAL 2a/2b studies, robust early formative evaluation, with an emphasis on the human users, is a necessary bridge from the development of clinical decision support systems driven by artificial intelligence (AI) to their implementation and paves the way toward large-scale summative evaluation. The objective of DECIDE-AI is to improve reporting along four main axes:

• the performance of the algorithms when first used with humans in small-scale, actual clinical settings,
• the safety profile of the algorithms prior to large-scale utilisation,
• the human factors (ergonomic) evaluation of the algorithms,
• the preparatory steps towards large-scale (randomised controlled) clinical trials.

A Delphi process with an international group of over 120 experts is currently being conducted and the guidelines are expected for the second half of 2021.

People:

• From the IDEAL Collaboration: Baptiste Vasey (University of Oxford), Prof Peter McCulloch (University of Oxford)
• External collaborators: Prof David Clifton (University of Oxford), Prof Gary Collins (University of Oxford), Prof Alastair Denniston (University of Birmingham), Dr Livia Faes (Moorfields Eye Hospital), Dr Bart Geerts (healthplus.ai), Dr Xiaoxuan Liu (University of Birmingham), Dr Piyush Mathur (Cleveland Clinic, Ohio), Dr Lauren Morgan (Morgan Human Systems Ltd), Dr Suchi Saria (Johns Hopkins University), Dr Peter Watkinson (University of Oxford)

Link to Correspondence in Nature Medicine: https://www.nature.com/articles/s41591-021-01229-5

Link to the systematic review performed as part of the preliminary investigations: https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2777403

Contact: Baptiste Vasey, baptiste.vasey@nds.ox.ac.uk