DECIDE-AI Checklist: The 17 AI Items Applied to a Real Early-Stage Clinical Evaluation

Published November 1, 2026

.tp{color:#333333 !important;line-height:1.7 !important;margin:0 0 16px 0 !important}
.th2{color:#0d2137 !important;margin:36px 0 14px 0 !important;line-height:1.3 !important}
.th3{color:#0d2137 !important;margin:24px 0 10px 0 !important;line-height:1.35 !important}
.tl{color:#003057 !important;font-weight:600;text-decoration:underline}
.tt{border-collapse:collapse;width:100%}
.tt th{background-color:#0d2137 !important;color:#ffffff !important;text-align:left;padding:10px 12px !important;font-size:.9rem;border:1px solid #0d2137 !important}
.tt td{color:#333333 !important;padding:9px 12px !important;font-size:.9rem;vertical-align:top;border:1px solid #d8dee6 !important;line-height:1.5}
.tt td.w{background-color:#ffffff !important}
.tt td.g{background-color:#f6f9fc !important}
.tt td.rg{background-color:#f1f8f3 !important}
.tt td.rp{background-color:#fff9ec !important}
.tt td.rn{background-color:#fef1f1 !important}
.tt td.rj{background-color:#f0f4fc !important}
.tt td.vg{background-color:#d9eee0 !important;color:#14532d !important;font-weight:700}
.tt td.vp{background-color:#ffecc2 !important;color:#6b4200 !important;font-weight:700}
.tt td.vn{background-color:#fbd5d5 !important;color:#7f1d1d !important;font-weight:700}
.tt td.vj{background-color:#d6e0f7 !important;color:#1e3a8a !important;font-weight:700}

The DECIDE-AI checklist is the reporting guideline for the first live, small-scale tests of an artificial intelligence decision support system on real patients. Published in May 2022 in both the BMJ and Nature Medicine by Vasey and colleagues, it lists 17 AI-specific items (28 rows once sub-items are counted) and 10 generic items, and it asks for details that trial and model guidelines never needed, such as how users were trained, which version of the software ran for which patient, and what went wrong.^1,2

Most guides stop at listing the items. This one goes further. After the checklist itself, we apply it row by row to a real open-access study of a surgical AI system, published in October 2026, that states it followed DECIDE-AI, with the section of the paper beside every verdict. The DECIDE-AI checklist is the earlier-stage companion to our CONSORT-AI checklist, and we do not repeat that material here.

What the DECIDE-AI Checklist Is and Where It Fits Among AI Reporting Guidelines

DECIDE-AI stands for Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence. The DECIDE-AI checklist is stage specific and not design specific. The guideline targets live evaluation, meaning the decisions made with the AI system have a real effect on patient care, as opposed to offline or shadow mode testing where they do not.¹ It does not prescribe a study design, so a prospective cohort, a non-randomised comparison or a case series can all use it, and it covers detection, diagnostic, prognostic and therapeutic systems.¹

The guideline came from a two-round modified Delphi process followed by a virtual consensus meeting. Of the experts who joined, 123 completed the first round, 138 completed the second and 16 attended the consensus meeting, and the authors report 151 experts from 18 countries across 20 stakeholder groups in total.¹ The checklist is described as a set of minimum reporting standards, and it is not a substitute for regulatory requirements.¹

The table below shows where the DECIDE-AI checklist sits among the AI guidelines. It follows the structure of the stage map in the original paper, in our own words.¹

Table 1. Where the DECIDE-AI checklist sits among AI reporting guidelines. Structure adapted from Vasey et al., BMJ 2022.

Three points matter for your submission. First, the DECIDE-AI checklist sits between preclinical work and the large comparative trial, so a study that has already moved to randomisation should use the CONSORT-AI checklist and, for a protocol, the SPIRIT protocol guide. Our CONSORT 2025 guide explains why CONSORT 2025, with its 30 items, replaced CONSORT 2010. Second, the DECIDE-AI checklist is a reporting guideline and not an appraisal tool. It checks whether you described what you did, not whether what you did was sound, and for appraisal of prediction models reviewers use PROBAST. Third, if your question is purely about model accuracy, read our comparison of STARD and TRIPOD first, and use our decision tree for choosing a reporting guideline for the wider set of options.

The DECIDE-AI Checklist at a Glance, All 17 AI Items and 10 Generic Items

The table condenses the official checklist into 38 rows. Arabic numbers mark the 17 AI-specific items and Roman numerals mark the 10 generic items. We paraphrased the wording to keep it short, so use the official checklist from the BMJ or Nature Medicine paper for the exact text you submit.^1,2

Table 2. The 38 rows of the DECIDE-AI checklist, condensed. Source structure from Vasey et al., BMJ 2022.

Notice how many rows sit under safety, errors and human factors. Rows 3c, 6a, 6b, 7, 11, 12, 13a, 13b, 14a, 14b and 16 together account for 11 of the 28 AI rows, which reflects the guideline’s focus on safety and on the human side of the system.

What Makes the DECIDE-AI Checklist Different From CONSORT-AI and TRIPOD+AI

The original paper lists methodological challenges that appear as soon as an AI system meets a clinic, and each one maps to rows you will be asked to complete.¹ Four of them change how a manuscript is written.

Human factors are part of the intervention (rows 3c, 7 and 14)

The DECIDE-AI checklist treats the AI system and its users as two collaborating forms of intelligence, so how people used the tool is reported as an outcome and not left to the discussion.¹ Row 7 asks which human factors methods were used, row 14a asks for a usability evaluation against recognised standards, and row 14b asks for learning curves. A study that combines questionnaires, observation and interviews is a mixed methods research design, and the same integration rules apply when you write it up. A small first-in-human series is also close in spirit to the pilot studies we describe elsewhere, with feasibility and refinement as the aim.

The system is allowed to change (rows 4a and 11)

In a drug trial the product is fixed. In early AI work the software, the hardware and the user interface are often revised between patients. Row 4a asks for the version, and row 11 asks for each change, when it happened, why, and what happened to the outcomes afterwards. Without that log, a reader cannot tell which system a given result belongs to.

Errors are data (rows 6, 13 and 16)

The guideline asks authors to define significant errors in advance, list every one that occurred with its rate and apparent cause, say whether it could be corrected, and discuss what it means for patient care.¹ The framing mirrors phase 1 and 2 drug studies, where safety is the first question. It also guards against dataset shift, where small differences between training data and clinical data change performance.¹

Human and AI agreement is reported (row 12)

Row 12 of the DECIDE-AI checklist asks how often users agreed with the system, why they departed from it, and whether the AI changed anyone’s mind. CONSORT-AI and TRIPOD+AI ask about user interaction and the expertise users need, but we found no row in either that asks for agreement rates and reasons for departing from the AI output. For a model-focused paper, our TRIPOD+AI checklist guide covers the reporting that comes earlier in the pathway.

Worked Example Applying the DECIDE-AI Checklist to a Real Open-Access Study

To show the checklist in use, we scored one published paper. Khan and colleagues reported a first-in-human evaluation of real-time AI anatomical segmentation during endoscopic pituitary surgery, in Pituitary in October 2026. It is published under a CC BY 4.0 licence, which is why we can quote it and reproduce two of its figures. We confirmed the licence in three places, the Crossref version-of-record licence field, the Europe PMC licence field and the JATS permissions of the full text.³

We chose it for a practical reason. It is a real paper that applies the DECIDE-AI checklist, with an IDEAL Stage 1 design, a registered study, eight patients and a surgical team of four users. It also includes its own completed DECIDE-AI checklist as Supplementary Material 2. We scored the article text first and compared with the authors’ checklist afterwards, so the verdicts below are an independent reading of the full text. The IDEAL framework, for readers who do not know it, is a staged pathway for evaluating surgical innovation.⁴

In the study, a computer vision model outlined the sella and the carotid arteries on a secondary monitor beside the unaltered surgical feed, and the surgical team switched the overlay on with a foot pedal. The registration is listed on ClinicalTrials.gov, and the full text is on PubMed Central.

Table 3. DECIDE-AI checklist rows scored against Khan et al., Pituitary 2026 (CC BY 4.0). Editverse reading of the full text.

Across the 38 rows, we scored 18 as Reported, 11 as Partly reported, 6 as Judgement call and 3 as Not found. The three Not found rows (8, VI and 14b) are the ones the authors themselves flag as not reported in their own checklist, with a reason, and row VIII is the fourth they flag. The paper is strong on the parts DECIDE-AI was written to elicit, which are errors, modifications and usability, and its gaps sit in governance detail, participant numbers, data and code, and the roles of companies.

What the Annotated Example Teaches About the DECIDE-AI Checklist

Give planned and actual numbers, and say why they differ

Row 3a asks how the number of patients was decided. The Methods plan six patients, the Results recruit eight, and the Discussion calls six of the eight deployments the planned cases. Each statement may be true, but a reader cannot reconcile them without a sentence on why the target moved. In an early-stage series, where the sample is small by design, the numbers are the first thing a reviewer checks.

Write the negative answers, as the authors did

The authors’ supplementary checklist records rows 8, VI, VIII and 14b as not reported, each with a short reason such as the IDEAL Stage 1 design. That is good practice, and it is what the guideline wants, because a stated reason is better than a blank. The reasons sit in the supplement, though, so a reader of the article alone does not see them. One sentence in the limitations paragraph would carry the same information into the paper.

Log every version, as the paper’s Figure 3 does

Row 11 asks for the timing and rationale of each change and the outcomes after it. Figure 3 of the paper is a compact model of the first two, with a column per operating session and a row each for AI model, user interface, hardware and implementation. The Methods add that the model was retrained "using data from all 110 patients and each subsequent patient included in the trial". When training data grow with every case, row 4a is best met by giving the training size for each labelled version.

Define a significant error before the first case

Row 6a asks how significant errors were defined. The example runs a pre-case functionality check and a root cause analysis for failures, which is sound, but it does not say in advance what counts as a significant error. That matters here because the first model’s sella outline spilled over the carotid (a “go zone” over a “no-go zone” in the figure above), the pattern a surgeon would consider most dangerous, and the carotid class was itself weak in preclinical testing (Dice 21.8%, recall 13.6%). A one-line definition, such as any overlay that places a safe outline over a no-go structure, would let the reader count events the same way the team did.

Count agreement, do not only describe it

Row 12 is where early-stage AI papers most often stay vague. The example records alignment per case in words, for example sella accurate and carotid only partly correct. A simple tally per structure and per case, with a reason for each departure from the overlay, turns the same observations into data. The authors name this gap themselves, noting that cases where the model helped a decision received less attention.

Name what companies and funders did

Row X asks for the role of funders and of any company. The example declares shareholdings and employment clearly, and it thanks three companies for collaboration, but it does not say what each contributed. A single sentence on who supplied the hardware, the recording platform and the annotation tools, and whether any of them saw the manuscript, completes the row. For row 17, a code and model statement belongs beside the data statement, even when the answer is that neither can be shared.

Run a consistency pass between abstract and tables

The abstract reports that the system demonstrated "acceptable real-time pixel-level sella segmentation accuracy". In the Results, accuracy in live use appears as qualitative alignment notes in Table 1, with no quantitative live accuracy. The FIM column in Table 1, as we read it, is empty for two of six deployed cases while the text gives a median. None of this changes the findings, and all of it is cheap to fix before submission.

How to Complete the DECIDE-AI Checklist Before You Submit

The routine below works for a finished manuscript and a DECIDE-AI checklist submission, and the first two steps work better if you take them while the study is still being planned.

• Download the official checklist from the BMJ or Nature Medicine paper and keep the version date in your file name.
• Plan the study around the rows so that familiarisation (3c), error definitions (6a), the change log (11) and usability instruments (14a) are in the protocol before the first patient.
• Decide what applies by marking rows that do not apply, such as subgroups or learning curves, and write the reason in the manuscript as well as in the checklist.
• Fill every row of the DECIDE-AI checklist with a location using page and line numbers from the submission file, and write a short negative answer for every row you did not do.
• Reconcile all numbers across the abstract, Methods, Results, tables and figures, starting with planned and recruited patients and users.
• Add the version log as a table or figure, with the date, the reason and the effect of each change to the model, hardware or interface.
• Count the errors by type with a rate, a cause and whether each could be corrected, and discuss what each means for patient care.
• State data, code and company roles in the back matter, including who supplied hardware, software and recording tools.

Mixed designs also need good survey practice, and our guide to questionnaire design in epidemiological studies helps with the usability and acceptability instruments. If the system is a prediction tool built on routine data, our walkthrough of AI in healthcare implementation gives the wider context, and a student mapping a thesis chapter to the right guideline can use the medical dissertation reporting guidelines guide. For the observational parts of a clinical series, the STROBE guide remains the standard companion.

The next stage after the DECIDE-AI checklist is a comparative trial. When you reach it, move to the CONSORT-AI checklist and the SPIRIT protocol guide. The guideline papers are open access, and the registry for the example study is on ClinicalTrials.gov. The IDEAL framework that DECIDE-AI aligns with is described by the IDEAL Collaboration and in a 2019 update in Annals of Surgery.⁴

Professional Support for DECIDE-AI Reporting

An early clinical evaluation is easier to report when the design questions are settled before the first patient and the manuscript is edited against the checklist afterwards. Two Editverse services map onto that. The first is a set of methodology consultation sessions, useful when you are deciding outcomes, sampling of users and patients, data collection and validity for a mixed-methods feasibility study. The second is substantive editing of the manuscript itself. Editverse editing and writing support aligns with COPE, ICMJE and GPP 2022, and with the ethics requirements of your institution and country.

Conclusion

The DECIDE-AI checklist asks for 38 rows that together describe an AI tool’s first contact with real patients, and the rows that matter most are the ones trial guidelines never asked about, namely errors, versions, user agreement and usability. Our worked example shows the pattern. A paper can log every model change and name its failures openly, and still leave unexplained a change in the number of patients, a definition of a significant error, a code statement and the role of its commercial partners. Start from the DECIDE-AI checklist when you plan the study, write a location against every row when you draft, and run a consistency pass before you submit. A reviewer who can find every row in minutes spends the review on your science.

Frequently Asked Questions

How many items does the DECIDE-AI checklist have?

The DECIDE-AI checklist has 17 AI-specific items, which expand to 28 rows once sub-items are counted, plus 10 generic items numbered with Roman numerals. That gives 38 rows in total.

Is the DECIDE-AI checklist only for randomised trials?

No. The DECIDE-AI checklist is stage specific and does not prescribe a study design. It is meant for early, small-scale, live clinical evaluation, and a prospective cohort, a non-randomised comparison or a case series can all use it. Randomised trials of AI use CONSORT-AI.

What counts as early-stage clinical evaluation for DECIDE-AI?

The AI system is used on real patients and its output affects care, at small scale. Offline or shadow-mode testing, where the output does not affect decisions, falls outside the guideline.

How is the DECIDE-AI checklist different from TRIPOD+AI and CONSORT-AI?

TRIPOD+AI covers prediction model development and validation, and CONSORT-AI covers randomised trials. The DECIDE-AI checklist sits between them and adds human factors, user agreement, system modifications and detailed error reporting.

Does the DECIDE-AI checklist replace the other AI guidelines?

No. The guidelines cover different stages, so a project may use the DECIDE-AI checklist for its first clinical series and CONSORT-AI later for a randomised trial. Note that CONSORT 2025 replaced CONSORT 2010, and that CONSORT-AI was built as an extension of CONSORT 2010.

Should authors submit a completed DECIDE-AI checklist with the manuscript?

Many journals ask for a completed checklist with a page or line number for each row. Check the author instructions, and write a short negative answer for any row you did not address.

Where is the official DECIDE-AI checklist published?

The checklist is published in the BMJ and in Nature Medicine, both from May 2022. The BMJ version is open access.

References

1. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ. 2022;377:e070904. doi:10.1136/bmj-2022-070904. PMID: 35584845. PMCID: PMC9116198. PubMed, PMC.
2. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-933. doi:10.1038/s41591-022-01772-9. PMID: 35585198. Nature Medicine.
3. Khan DZ, Mao Z, Wijekoon A, Das A, Williams SC, Blandford A, et al. First-in-human evaluation of real-time pixel-level AI-assisted anatomical segmentation in neurosurgery: pituitary surgery as an exemplar. Pituitary. 2026;29(5):163. doi:10.1007/s11102-026-01762-x. PMID: 42829367. PMCID: PMC13634013. CC BY 4.0. PubMed, PMC.
4. Hirst A, Philippou Y, Blazeby J, Campbell B, Campbell M, Feinberg J, et al. No surgical innovation without evaluation: evolution and further development of the IDEAL framework and recommendations. Ann Surg. 2019;269(2):211-220. doi:10.1097/SLA.0000000000002794. PMID: 29697448. PubMed.