No Cover Image

Journal article 59 views 6 downloads

Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors

Saeed Ur Rehman Orcid Logo, Anwar Ali Orcid Logo, Adil Mehmood Khan, Cynthia Okpala

Algorithms, Volume: 17, Issue: 12, Start page: 556

Swansea University Author: Anwar Ali Orcid Logo

  • 68579.VOR.pdf

    PDF | Version of Record

    © 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.

    Download (2MB)

Check full text

DOI (Published version): 10.3390/a17120556

Abstract

With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inap...

Full description

Published in: Algorithms
ISSN: 1999-4893
Published: MDPI AG 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa68579
first_indexed 2025-01-09T20:33:52Z
last_indexed 2025-01-30T20:25:20Z
id cronfa68579
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2025-01-30T13:09:04.7332457</datestamp><bib-version>v2</bib-version><id>68579</id><entry>2024-12-17</entry><title>Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors</title><swanseaauthors><author><sid>f206105e1de57bebba0fd04fe9870779</sid><ORCID>0000-0001-7366-9002</ORCID><firstname>Anwar</firstname><surname>Ali</surname><name>Anwar Ali</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-12-17</date><deptcode>ACEM</deptcode><abstract>With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.</abstract><type>Journal Article</type><journal>Algorithms</journal><volume>17</volume><journalNumber>12</journalNumber><paginationStart>556</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>1999-4893</issnElectronic><keywords>Machine learning; LOSO; human activity recognition</keywords><publishedDay>5</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-12-05</publishedDate><doi>10.3390/a17120556</doi><url/><notes/><college>COLLEGE NANME</college><department>Aerospace, Civil, Electrical, and Mechanical Engineering</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>ACEM</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This research received no external funding.</funders><projectreference/><lastEdited>2025-01-30T13:09:04.7332457</lastEdited><Created>2024-12-17T13:21:11.5144374</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering</level></path><authors><author><firstname>Saeed Ur</firstname><surname>Rehman</surname><orcid>0009-0009-4566-7144</orcid><order>1</order></author><author><firstname>Anwar</firstname><surname>Ali</surname><orcid>0000-0001-7366-9002</orcid><order>2</order></author><author><firstname>Adil Mehmood</firstname><surname>Khan</surname><order>3</order></author><author><firstname>Cynthia</firstname><surname>Okpala</surname><order>4</order></author></authors><documents><document><filename>68579__33161__0cf79af76f4a4cb1a67f3231075d59ea.pdf</filename><originalFilename>68579.VOR.pdf</originalFilename><uploaded>2024-12-17T13:26:40.3479993</uploaded><type>Output</type><contentLength>2101400</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2025-01-30T13:09:04.7332457 v2 68579 2024-12-17 Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors f206105e1de57bebba0fd04fe9870779 0000-0001-7366-9002 Anwar Ali Anwar Ali true false 2024-12-17 ACEM With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems. Journal Article Algorithms 17 12 556 MDPI AG 1999-4893 Machine learning; LOSO; human activity recognition 5 12 2024 2024-12-05 10.3390/a17120556 COLLEGE NANME Aerospace, Civil, Electrical, and Mechanical Engineering COLLEGE CODE ACEM Swansea University Another institution paid the OA fee This research received no external funding. 2025-01-30T13:09:04.7332457 2024-12-17T13:21:11.5144374 Faculty of Science and Engineering School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering Saeed Ur Rehman 0009-0009-4566-7144 1 Anwar Ali 0000-0001-7366-9002 2 Adil Mehmood Khan 3 Cynthia Okpala 4 68579__33161__0cf79af76f4a4cb1a67f3231075d59ea.pdf 68579.VOR.pdf 2024-12-17T13:26:40.3479993 Output 2101400 application/pdf Version of Record true © 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. true eng https://creativecommons.org/licenses/by/4.0/
title Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
spellingShingle Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
Anwar Ali
title_short Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_full Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_fullStr Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_full_unstemmed Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_sort Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
author_id_str_mv f206105e1de57bebba0fd04fe9870779
author_id_fullname_str_mv f206105e1de57bebba0fd04fe9870779_***_Anwar Ali
author Anwar Ali
author2 Saeed Ur Rehman
Anwar Ali
Adil Mehmood Khan
Cynthia Okpala
format Journal article
container_title Algorithms
container_volume 17
container_issue 12
container_start_page 556
publishDate 2024
institution Swansea University
issn 1999-4893
doi_str_mv 10.3390/a17120556
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering
document_store_str 1
active_str 0
description With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.
published_date 2024-12-05T08:23:52Z
_version_ 1824111022889238528
score 11.050721

Similar Items