No Cover Image

Journal article 90 views 11 downloads

Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet

Mandeep Singh Devgan, Gurvinder Singh, Purushottam Sharma Orcid Logo, Tajinder Kumar, Cheng Cheng Orcid Logo, Deepak Ahlawat

IET Intelligent Transport Systems, Volume: 20, Issue: 1, Start page: e70167

Swansea University Author: Cheng Cheng Orcid Logo

  • 71396.VOR.pdf

    PDF | Version of Record

    © 2026 The Author(s). IET Intelligent Transport Systems published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology. This is an open access article under the terms of the Creative Commons Attribution License.

    Download (6.2MB)

Check full text

DOI (Published version): 10.1049/itr2.70167

Abstract

Autonomous driving is also impossible without traffic sign recognition (TSR; also known as traffic sign-on-road), which limits its reliability to domain changes, unfavourable weather, obstruction and hardware capacity. This paper proposes HACTNet, a low-complexity CNN-Transformer hybrid model that p...

Full description

Published in: IET Intelligent Transport Systems
ISSN: 1751-956X 1751-9578
Published: Wiley 2026
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa71396
first_indexed 2026-02-09T11:03:15Z
last_indexed 2026-03-17T05:37:07Z
id cronfa71396
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2026-03-16T14:27:36.7620932</datestamp><bib-version>v2</bib-version><id>71396</id><entry>2026-02-09</entry><title>Attention&#x2010;Guided Lightweight CNN&#x2010;Transformer Fusion for Real&#x2010;Time Traffic Sign Recognition in Adverse Environments: HACTNet</title><swanseaauthors><author><sid>11ddf61c123b99e59b00fa1479367582</sid><ORCID>0000-0003-0371-9646</ORCID><firstname>Cheng</firstname><surname>Cheng</surname><name>Cheng Cheng</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2026-02-09</date><deptcode>MACS</deptcode><abstract>Autonomous driving is also impossible without traffic sign recognition (TSR; also known as traffic sign-on-road), which limits its reliability to domain changes, unfavourable weather, obstruction and hardware capacity. This paper proposes HACTNet, a low-complexity CNN-Transformer hybrid model that pushes the state-of-art in TSR by making a noteworthy set of contributions including (i) efficient convaps to model parts of the image, (ii) transformer encoder to capture the global context and (iii) an attention-based fusion block to dynamically combine the two complementary sets of features. This synergy facilitates strong recognition in presence of blur and occlusion and in varying illumination. In addition to accuracy, HACTNet achieves high robustness (52.8%) against strong PGD adversarial attacks (8/255), but is still efficient (7.9 M parameters and 22.1 FPS) on the NVIDIA Jetson Nano. Moreover, the comparative analysis between the hybrid models (EATFormer, local-ViT) and HACTNet proves that HACTNet has a better accuracy-efficiency ratio. The extraordinary capability to counteract adverse weather conditions, fog, night, rain, snow etc., which is proven by the extensive testing of the real-world ACDC adverse conditions data set, supports the viability of the proposed solutions in the real world. It is plug and play modularity with on-going learning via elastic weight consolidation (3.3% less forgetting) and unsupervised domain adaptation via MMD loss (5.3% better on TT100K with no labels). Moreover, INT8 quantization with quantization-aware training (QAT) incurs little accuracy loss (less than 0.5 percent) and much lower energy (0.27 J/sample) usage, which forms an edge deployment preparedness. Additionally, when adjusting to new traffic signs over time, the model shows compatibility with continuous learning, achieving a low forgetting rate (3.3%), highlighting its practical viability for long-term autonomous deployment. Overall, HACTNet produces a versatile and expandable solution for next-generation intelligent transportation systems by striking a balance between accuracy, robustness and efficiency.</abstract><type>Journal Article</type><journal>IET Intelligent Transport Systems</journal><volume>20</volume><journalNumber>1</journalNumber><paginationStart>e70167</paginationStart><paginationEnd/><publisher>Wiley</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>1751-956X</issnPrint><issnElectronic>1751-9578</issnElectronic><keywords>traffic, traffic information systems, traffic management and control, traffic management, mobility as a service</keywords><publishedDay>31</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2026</publishedYear><publishedDate>2026-12-31</publishedDate><doi>10.1049/itr2.70167</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>SU Library paid the OA fee (TA Institutional Deal)</apcterm><funders>Authors have been supported by UKRI EPSRC Grant funded Doctoral Training Centre at Swansea University, through project RS718. Authors also have been supported by UKRI EPSRC Grant EP/W020408/1.</funders><projectreference/><lastEdited>2026-03-16T14:27:36.7620932</lastEdited><Created>2026-02-09T11:01:00.6437544</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Mandeep Singh</firstname><surname>Devgan</surname><order>1</order></author><author><firstname>Gurvinder</firstname><surname>Singh</surname><order>2</order></author><author><firstname>Purushottam</firstname><surname>Sharma</surname><orcid>0000-0002-8037-7152</orcid><order>3</order></author><author><firstname>Tajinder</firstname><surname>Kumar</surname><order>4</order></author><author><firstname>Cheng</firstname><surname>Cheng</surname><orcid>0000-0003-0371-9646</orcid><order>5</order></author><author><firstname>Deepak</firstname><surname>Ahlawat</surname><order>6</order></author></authors><documents><document><filename>71396__36421__46018cefdb194869aa26ec61df4269d4.pdf</filename><originalFilename>71396.VOR.pdf</originalFilename><uploaded>2026-03-16T14:24:17.7197719</uploaded><type>Output</type><contentLength>6498720</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; 2026 The Author(s). IET Intelligent Transport Systems published by John Wiley &amp; Sons Ltd on behalf of The Institution of Engineering and Technology. This is an open access article under the terms of the Creative Commons Attribution License.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>http://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2026-03-16T14:27:36.7620932 v2 71396 2026-02-09 Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet 11ddf61c123b99e59b00fa1479367582 0000-0003-0371-9646 Cheng Cheng Cheng Cheng true false 2026-02-09 MACS Autonomous driving is also impossible without traffic sign recognition (TSR; also known as traffic sign-on-road), which limits its reliability to domain changes, unfavourable weather, obstruction and hardware capacity. This paper proposes HACTNet, a low-complexity CNN-Transformer hybrid model that pushes the state-of-art in TSR by making a noteworthy set of contributions including (i) efficient convaps to model parts of the image, (ii) transformer encoder to capture the global context and (iii) an attention-based fusion block to dynamically combine the two complementary sets of features. This synergy facilitates strong recognition in presence of blur and occlusion and in varying illumination. In addition to accuracy, HACTNet achieves high robustness (52.8%) against strong PGD adversarial attacks (8/255), but is still efficient (7.9 M parameters and 22.1 FPS) on the NVIDIA Jetson Nano. Moreover, the comparative analysis between the hybrid models (EATFormer, local-ViT) and HACTNet proves that HACTNet has a better accuracy-efficiency ratio. The extraordinary capability to counteract adverse weather conditions, fog, night, rain, snow etc., which is proven by the extensive testing of the real-world ACDC adverse conditions data set, supports the viability of the proposed solutions in the real world. It is plug and play modularity with on-going learning via elastic weight consolidation (3.3% less forgetting) and unsupervised domain adaptation via MMD loss (5.3% better on TT100K with no labels). Moreover, INT8 quantization with quantization-aware training (QAT) incurs little accuracy loss (less than 0.5 percent) and much lower energy (0.27 J/sample) usage, which forms an edge deployment preparedness. Additionally, when adjusting to new traffic signs over time, the model shows compatibility with continuous learning, achieving a low forgetting rate (3.3%), highlighting its practical viability for long-term autonomous deployment. Overall, HACTNet produces a versatile and expandable solution for next-generation intelligent transportation systems by striking a balance between accuracy, robustness and efficiency. Journal Article IET Intelligent Transport Systems 20 1 e70167 Wiley 1751-956X 1751-9578 traffic, traffic information systems, traffic management and control, traffic management, mobility as a service 31 12 2026 2026-12-31 10.1049/itr2.70167 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University SU Library paid the OA fee (TA Institutional Deal) Authors have been supported by UKRI EPSRC Grant funded Doctoral Training Centre at Swansea University, through project RS718. Authors also have been supported by UKRI EPSRC Grant EP/W020408/1. 2026-03-16T14:27:36.7620932 2026-02-09T11:01:00.6437544 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Mandeep Singh Devgan 1 Gurvinder Singh 2 Purushottam Sharma 0000-0002-8037-7152 3 Tajinder Kumar 4 Cheng Cheng 0000-0003-0371-9646 5 Deepak Ahlawat 6 71396__36421__46018cefdb194869aa26ec61df4269d4.pdf 71396.VOR.pdf 2026-03-16T14:24:17.7197719 Output 6498720 application/pdf Version of Record true © 2026 The Author(s). IET Intelligent Transport Systems published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology. This is an open access article under the terms of the Creative Commons Attribution License. true eng http://creativecommons.org/licenses/by/4.0/
title Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
spellingShingle Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
Cheng Cheng
title_short Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
title_full Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
title_fullStr Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
title_full_unstemmed Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
title_sort Attention‐Guided Lightweight CNN‐Transformer Fusion for Real‐Time Traffic Sign Recognition in Adverse Environments: HACTNet
author_id_str_mv 11ddf61c123b99e59b00fa1479367582
author_id_fullname_str_mv 11ddf61c123b99e59b00fa1479367582_***_Cheng Cheng
author Cheng Cheng
author2 Mandeep Singh Devgan
Gurvinder Singh
Purushottam Sharma
Tajinder Kumar
Cheng Cheng
Deepak Ahlawat
format Journal article
container_title IET Intelligent Transport Systems
container_volume 20
container_issue 1
container_start_page e70167
publishDate 2026
institution Swansea University
issn 1751-956X
1751-9578
doi_str_mv 10.1049/itr2.70167
publisher Wiley
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Autonomous driving is also impossible without traffic sign recognition (TSR; also known as traffic sign-on-road), which limits its reliability to domain changes, unfavourable weather, obstruction and hardware capacity. This paper proposes HACTNet, a low-complexity CNN-Transformer hybrid model that pushes the state-of-art in TSR by making a noteworthy set of contributions including (i) efficient convaps to model parts of the image, (ii) transformer encoder to capture the global context and (iii) an attention-based fusion block to dynamically combine the two complementary sets of features. This synergy facilitates strong recognition in presence of blur and occlusion and in varying illumination. In addition to accuracy, HACTNet achieves high robustness (52.8%) against strong PGD adversarial attacks (8/255), but is still efficient (7.9 M parameters and 22.1 FPS) on the NVIDIA Jetson Nano. Moreover, the comparative analysis between the hybrid models (EATFormer, local-ViT) and HACTNet proves that HACTNet has a better accuracy-efficiency ratio. The extraordinary capability to counteract adverse weather conditions, fog, night, rain, snow etc., which is proven by the extensive testing of the real-world ACDC adverse conditions data set, supports the viability of the proposed solutions in the real world. It is plug and play modularity with on-going learning via elastic weight consolidation (3.3% less forgetting) and unsupervised domain adaptation via MMD loss (5.3% better on TT100K with no labels). Moreover, INT8 quantization with quantization-aware training (QAT) incurs little accuracy loss (less than 0.5 percent) and much lower energy (0.27 J/sample) usage, which forms an edge deployment preparedness. Additionally, when adjusting to new traffic signs over time, the model shows compatibility with continuous learning, achieving a low forgetting rate (3.3%), highlighting its practical viability for long-term autonomous deployment. Overall, HACTNet produces a versatile and expandable solution for next-generation intelligent transportation systems by striking a balance between accuracy, robustness and efficiency.
published_date 2026-12-31T05:38:33Z
_version_ 1860430005524234240
score 11.099917

Similar Items