Data

Trauma Checklist Extraction from Clinical Documents: A Study of Large Language Models and Prompting Techniques

Deakin University
Kon Mouzakis (Aggregated by) Rajesh Vasa (Aggregated by) Rena Logothetis (Aggregated by)
Viewed: [[ro.stat.viewed]] Cited: [[ro.stat.cited]] Accessed: [[ro.stat.accessed]]
ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rfr_id=info%3Asid%2FANDS&rft_id=info:doi10.26187/deakin.34926213&rft.title=Trauma Checklist Extraction from Clinical Documents: A Study of Large Language Models and Prompting Techniques&rft.identifier=10.26187/deakin.34926213&rft.publisher=Deakin University&rft.description=We selected seven clinical checklists used for neurotrauma management: International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) Scoring Outlines and American Spinal Injury Association (ASIA) Impairment Scale (AIS) (IAIS), Pre-Arrival Trauma Checklist (PATC), Adult Trauma Triaging Tool (ATTT), Canadian CT Head Rule (CCHR), Canadian C Spine Rule (CCR), Nexus Criteria for C-Spine Imaging (NCCI), and Nexus Head CT Instrument (NHCI). Five models (GPT-3.5-Turbo, GPT-4, GPT-4o, GPT-o1-mini and Claude 3.5 Sonnet) were evaluated using four prompting techniques (role-play, instruction following, chain-of-thought and few-shot prompting) across the seven checklists, giving 140 model-prompt-checklist combinations with ten repetitions each (1,400 model runs). For each model run, we compared the LLM-extracted checklist with the corresponding manually extracted reference checklist using string similarity percentage and TF-IDF cosine similarity. Primary analyses included only runs with both scores available. We report medians (Q1, Q3) and compared models within each checklist and prompt using exact unpaired rank-permutation tests, with Bonferroni correction across comparisons.&rft.creator=Kon Mouzakis&rft.creator=Rajesh Vasa&rft.creator=Rena Logothetis&rft.date=2026&rft_rights= https://creativecommons.org/licenses/by/4.0/&rft_subject=Information and computing sciences&rft_subject=neurotrauma&rft_subject=checklists&rft_subject=large language models&rft_subject=prompting techniques&rft_subject=clinical decision support systems&rft.type=dataset&rft.language=English Access the data

Full description

We selected seven clinical checklists used for neurotrauma management: International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) Scoring Outlines and American Spinal Injury Association (ASIA) Impairment Scale (AIS) (IAIS), Pre-Arrival Trauma Checklist (PATC), Adult Trauma Triaging Tool (ATTT), Canadian CT Head Rule (CCHR), Canadian C Spine Rule (CCR), Nexus Criteria for C-Spine Imaging (NCCI), and Nexus Head CT Instrument (NHCI). Five models (GPT-3.5-Turbo, GPT-4, GPT-4o, GPT-o1-mini and Claude 3.5 Sonnet) were evaluated using four prompting techniques (role-play, instruction following, chain-of-thought and few-shot prompting) across the seven checklists, giving 140 model-prompt-checklist combinations with ten repetitions each (1,400 model runs). For each model run, we compared the LLM-extracted checklist with the corresponding manually extracted reference checklist using string similarity percentage and TF-IDF cosine similarity. Primary analyses included only runs with both scores available. We report medians (Q1, Q3) and compared models within each checklist and prompt using exact unpaired rank-permutation tests, with Bonferroni correction across comparisons.

This dataset is part of a larger collection

Click to explore relationships graph
Subjects

User Contributed Tags    

Login to tag this record with meaningful keywords to make it easier to discover

Identifiers
ACN 633 798 857