Full description
We selected seven clinical checklists used for neurotrauma management: International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) Scoring Outlines and American Spinal Injury Association (ASIA) Impairment Scale (AIS) (IAIS), Pre-Arrival Trauma Checklist (PATC), Adult Trauma Triaging Tool (ATTT), Canadian CT Head Rule (CCHR), Canadian C Spine Rule (CCR), Nexus Criteria for C-Spine Imaging (NCCI), and Nexus Head CT Instrument (NHCI). Five models (GPT-3.5-Turbo, GPT-4, GPT-4o, GPT-o1-mini and Claude 3.5 Sonnet) were evaluated using four prompting techniques (role-play, instruction following, chain-of-thought and few-shot prompting) across the seven checklists, giving 140 model-prompt-checklist combinations with ten repetitions each (1,400 model runs). For each model run, we compared the LLM-extracted checklist with the corresponding manually extracted reference checklist using string similarity percentage and TF-IDF cosine similarity. Primary analyses included only runs with both scores available. We report medians (Q1, Q3) and compared models within each checklist and prompt using exact unpaired rank-permutation tests, with Bonferroni correction across comparisons.
User Contributed Tags
Login to tag this record with meaningful keywords to make it easier to discover
- DOI : 10.26187/DEAKIN.34926213
