No Cover Image

Journal article 395 views 70 downloads

External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study

Banafsheh Arshi Orcid Logo, Laura Cowley, Eline Rijnhart Orcid Logo, Kelly Reeve, Luc J. Smits, Laure Wynants

Journal of Clinical Epidemiology, Volume: 186, Start page: 111902

Swansea University Author: Laura Cowley

  • 70067.VoR.pdf

    PDF | Version of Record

    Copyright: 2025 The Authors. This is an open access article under the CC BY license.

    Download (353.04KB)

Check full text

DOI (Published version): 10.1016/j.jclinepi.2025.111902

Abstract

We aimed to assess paths taken by clinical prediction models (CPM) after development by quantifying external validation, impact assessment, and utilization in clinical practice. We followed a random sample of 109 regression-based CPM development articles published between 1995-2020 by performing a f...

Full description

Published in: Journal of Clinical Epidemiology
ISSN: 0895-4356 1878-5921
Published: Elsevier BV 2025
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa70067
first_indexed 2025-07-31T10:03:55Z
last_indexed 2025-08-14T05:39:45Z
id cronfa70067
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2025-08-13T15:40:22.0284485</datestamp><bib-version>v2</bib-version><id>70067</id><entry>2025-07-31</entry><title>External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study</title><swanseaauthors><author><sid>a80501f280e89fee276510b25fc68e77</sid><firstname>Laura</firstname><surname>Cowley</surname><name>Laura Cowley</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2025-07-31</date><deptcode>MEDS</deptcode><abstract>We aimed to assess paths taken by clinical prediction models (CPM) after development by quantifying external validation, impact assessment, and utilization in clinical practice. We followed a random sample of 109 regression-based CPM development articles published between 1995-2020 by performing a forward citation search. We estimated five- and ten-year probabilities of validation and impact assessment after development of CPMs using Kaplan-Meier analysis. In addition, we conducted a survey among the authors of the development articles to determine whether the CPMs had been used in clinical settings. Eighteen (17%) CPM development articles reported a CPM that was externally validated after development. Five- and ten-year probabilities of validation were 0.13 (0.06 to 0.19) and 0.16 (0.08 to 0.23), respectively. Only one article had a CPM with impact assessment during follow-up (ten-year probability: 0.01 [0 to 0.04]). Among the 34 (31%) articles with a survey response, 17 (50%) had CPMs that had been used in clinical practice, in a median of 5 sites (IQR: 1-347). Of these models, only 4 (24%) were externally validated, and none had undergone impact assessment. Despite evidence of utilization in clinical settings, few models are externally validated after development, and published impact assessment is scarce. To prevent compromising patient safety, it is crucial to intensify efforts to promote external validation and impact assessment of prediction models.</abstract><type>Journal Article</type><journal>Journal of Clinical Epidemiology</journal><volume>186</volume><journalNumber/><paginationStart>111902</paginationStart><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>0895-4356</issnPrint><issnElectronic>1878-5921</issnElectronic><keywords>Prediction research; clinical prediction models; external validation; impact assessment; clinical utilization</keywords><publishedDay>1</publishedDay><publishedMonth>10</publishedMonth><publishedYear>2025</publishedYear><publishedDate>2025-10-01</publishedDate><doi>10.1016/j.jclinepi.2025.111902</doi><url/><notes/><college>COLLEGE NANME</college><department>Medical School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MEDS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2025-08-13T15:40:22.0284485</lastEdited><Created>2025-07-31T10:57:41.1042981</Created><path><level id="1">Faculty of Medicine, Health and Life Sciences</level><level id="2">Swansea University Medical School - Health Data Science</level></path><authors><author><firstname>Banafsheh</firstname><surname>Arshi</surname><orcid>0000-0003-0223-7872</orcid><order>1</order></author><author><firstname>Laura</firstname><surname>Cowley</surname><order>2</order></author><author><firstname>Eline</firstname><surname>Rijnhart</surname><orcid>0000-0001-5300-998x</orcid><order>3</order></author><author><firstname>Kelly</firstname><surname>Reeve</surname><order>4</order></author><author><firstname>Luc J.</firstname><surname>Smits</surname><order>5</order></author><author><firstname>Laure</firstname><surname>Wynants</surname><order>6</order></author></authors><documents><document><filename>70067__34949__b70adf61411a4e978d2cb03a8f5df7fc.pdf</filename><originalFilename>70067.VoR.pdf</originalFilename><uploaded>2025-08-13T15:27:36.6881069</uploaded><type>Output</type><contentLength>361516</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>Copyright: 2025 The Authors. This is an open access article under the CC BY license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>http://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2025-08-13T15:40:22.0284485 v2 70067 2025-07-31 External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study a80501f280e89fee276510b25fc68e77 Laura Cowley Laura Cowley true false 2025-07-31 MEDS We aimed to assess paths taken by clinical prediction models (CPM) after development by quantifying external validation, impact assessment, and utilization in clinical practice. We followed a random sample of 109 regression-based CPM development articles published between 1995-2020 by performing a forward citation search. We estimated five- and ten-year probabilities of validation and impact assessment after development of CPMs using Kaplan-Meier analysis. In addition, we conducted a survey among the authors of the development articles to determine whether the CPMs had been used in clinical settings. Eighteen (17%) CPM development articles reported a CPM that was externally validated after development. Five- and ten-year probabilities of validation were 0.13 (0.06 to 0.19) and 0.16 (0.08 to 0.23), respectively. Only one article had a CPM with impact assessment during follow-up (ten-year probability: 0.01 [0 to 0.04]). Among the 34 (31%) articles with a survey response, 17 (50%) had CPMs that had been used in clinical practice, in a median of 5 sites (IQR: 1-347). Of these models, only 4 (24%) were externally validated, and none had undergone impact assessment. Despite evidence of utilization in clinical settings, few models are externally validated after development, and published impact assessment is scarce. To prevent compromising patient safety, it is crucial to intensify efforts to promote external validation and impact assessment of prediction models. Journal Article Journal of Clinical Epidemiology 186 111902 Elsevier BV 0895-4356 1878-5921 Prediction research; clinical prediction models; external validation; impact assessment; clinical utilization 1 10 2025 2025-10-01 10.1016/j.jclinepi.2025.111902 COLLEGE NANME Medical School COLLEGE CODE MEDS Swansea University Another institution paid the OA fee 2025-08-13T15:40:22.0284485 2025-07-31T10:57:41.1042981 Faculty of Medicine, Health and Life Sciences Swansea University Medical School - Health Data Science Banafsheh Arshi 0000-0003-0223-7872 1 Laura Cowley 2 Eline Rijnhart 0000-0001-5300-998x 3 Kelly Reeve 4 Luc J. Smits 5 Laure Wynants 6 70067__34949__b70adf61411a4e978d2cb03a8f5df7fc.pdf 70067.VoR.pdf 2025-08-13T15:27:36.6881069 Output 361516 application/pdf Version of Record true Copyright: 2025 The Authors. This is an open access article under the CC BY license. true eng http://creativecommons.org/licenses/by/4.0/
title External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
spellingShingle External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
Laura Cowley
title_short External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
title_full External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
title_fullStr External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
title_full_unstemmed External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
title_sort External validation, impact assessment and clinical utilization of clinical prediction models: a prospective cohort study
author_id_str_mv a80501f280e89fee276510b25fc68e77
author_id_fullname_str_mv a80501f280e89fee276510b25fc68e77_***_Laura Cowley
author Laura Cowley
author2 Banafsheh Arshi
Laura Cowley
Eline Rijnhart
Kelly Reeve
Luc J. Smits
Laure Wynants
format Journal article
container_title Journal of Clinical Epidemiology
container_volume 186
container_start_page 111902
publishDate 2025
institution Swansea University
issn 0895-4356
1878-5921
doi_str_mv 10.1016/j.jclinepi.2025.111902
publisher Elsevier BV
college_str Faculty of Medicine, Health and Life Sciences
hierarchytype
hierarchy_top_id facultyofmedicinehealthandlifesciences
hierarchy_top_title Faculty of Medicine, Health and Life Sciences
hierarchy_parent_id facultyofmedicinehealthandlifesciences
hierarchy_parent_title Faculty of Medicine, Health and Life Sciences
department_str Swansea University Medical School - Health Data Science{{{_:::_}}}Faculty of Medicine, Health and Life Sciences{{{_:::_}}}Swansea University Medical School - Health Data Science
document_store_str 1
active_str 0
description We aimed to assess paths taken by clinical prediction models (CPM) after development by quantifying external validation, impact assessment, and utilization in clinical practice. We followed a random sample of 109 regression-based CPM development articles published between 1995-2020 by performing a forward citation search. We estimated five- and ten-year probabilities of validation and impact assessment after development of CPMs using Kaplan-Meier analysis. In addition, we conducted a survey among the authors of the development articles to determine whether the CPMs had been used in clinical settings. Eighteen (17%) CPM development articles reported a CPM that was externally validated after development. Five- and ten-year probabilities of validation were 0.13 (0.06 to 0.19) and 0.16 (0.08 to 0.23), respectively. Only one article had a CPM with impact assessment during follow-up (ten-year probability: 0.01 [0 to 0.04]). Among the 34 (31%) articles with a survey response, 17 (50%) had CPMs that had been used in clinical practice, in a median of 5 sites (IQR: 1-347). Of these models, only 4 (24%) were externally validated, and none had undergone impact assessment. Despite evidence of utilization in clinical settings, few models are externally validated after development, and published impact assessment is scarce. To prevent compromising patient safety, it is crucial to intensify efforts to promote external validation and impact assessment of prediction models.
published_date 2025-10-01T05:30:23Z
_version_ 1856805612748275712
score 11.09611

Similar Items