No Cover Image

Journal article 832 views 570 downloads

Pruning CNN filters via quantifying the importance of deep visual representations

Ali Alqahtani, Xianghua Xie Orcid Logo, Mark Jones Orcid Logo, Ehab Mohamed Mahmoud Essa Orcid Logo

Computer Vision and Image Understanding, Volume: 208-209, Start page: 103220

Swansea University Authors: Ali Alqahtani, Xianghua Xie Orcid Logo, Mark Jones Orcid Logo, Ehab Mohamed Mahmoud Essa Orcid Logo

  • cviu_swansea.pdf

    PDF | Accepted Manuscript

    ©2021 All rights reserved. All article content, except where otherwise noted, is licensed under a Creative Commons Attribution Non-Commercial No Derivatives License (CC-BY-NC-ND)

    Download (8.25MB)

Check full text

DOI (Published version): 10.1016/j.cviu.2021.103220

Abstract

The achievement of convolutional neural networks (CNNs) in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we pro- pose a novel framework to measure the importance of individual hidden units by computing a measure of rele...

Full description

Published in: Computer Vision and Image Understanding
ISSN: 1077-3142
Published: Elsevier BV 2021
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa56831
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2021-05-09T10:59:42Z
last_indexed 2022-02-09T04:25:58Z
id cronfa56831
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2022-02-08T12:54:32.2666876</datestamp><bib-version>v2</bib-version><id>56831</id><entry>2021-05-09</entry><title>Pruning CNN filters via quantifying the importance of deep visual representations</title><swanseaauthors><author><sid>c0c682a8d9d12520f9639b89f9500946</sid><firstname>Ali</firstname><surname>Alqahtani</surname><name>Ali Alqahtani</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>2e1030b6e14fc9debd5d5ae7cc335562</sid><ORCID>0000-0001-8991-1190</ORCID><firstname>Mark</firstname><surname>Jones</surname><name>Mark Jones</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>1e2e134b148109090dadd3c27585e0d5</sid><ORCID>NULL</ORCID><firstname>Ehab Mohamed Mahmoud</firstname><surname>Essa</surname><name>Ehab Mohamed Mahmoud Essa</name><active>true</active><ethesisStudent>true</ethesisStudent></author></swanseaauthors><date>2021-05-09</date><deptcode>SCS</deptcode><abstract>The achievement of convolutional neural networks (CNNs) in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we pro- pose a novel framework to measure the importance of individual hidden units by computing a measure of relevance to identify the most critical filters and prune them to compress and accelerate CNNs. Unlike exist- ing methods, we introduce the use of the activation of feature maps to detect valuable information and the essential semantic parts to evaluate the importance of feature maps, inspired by novel neural network inter- pretability. A majority voting technique based on the degree of alignment between a semantic concept and individual hidden unit representations is proposed to quantitatively evaluate the importance of feature maps. We also propose a simple yet effective method to estimate new convolution kernels based on the remaining, crucial channels to accomplish effective CNN compression.</abstract><type>Journal Article</type><journal>Computer Vision and Image Understanding</journal><volume>208-209</volume><journalNumber/><paginationStart>103220</paginationStart><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>1077-3142</issnPrint><issnElectronic/><keywords>Deep learning; Convolutional neural networks; Filter pruning; Model compression</keywords><publishedDay>1</publishedDay><publishedMonth>7</publishedMonth><publishedYear>2021</publishedYear><publishedDate>2021-07-01</publishedDate><doi>10.1016/j.cviu.2021.103220</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>This work is supported by EPSRC, UK EP/N028139/1</funders><lastEdited>2022-02-08T12:54:32.2666876</lastEdited><Created>2021-05-09T11:54:08.1604963</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Ali</firstname><surname>Alqahtani</surname><order>1</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>2</order></author><author><firstname>Mark</firstname><surname>Jones</surname><orcid>0000-0001-8991-1190</orcid><order>3</order></author><author><firstname>Ehab Mohamed Mahmoud</firstname><surname>Essa</surname><orcid>NULL</orcid><order>4</order></author></authors><documents><document><filename>56831__19827__c64cae8a23ae419694fb3ec085e64f8a.pdf</filename><originalFilename>cviu_swansea.pdf</originalFilename><uploaded>2021-05-09T11:56:41.1659385</uploaded><type>Output</type><contentLength>8655835</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><embargoDate>2022-05-18T00:00:00.0000000</embargoDate><documentNotes>&#xA9;2021 All rights reserved. All article content, except where otherwise noted, is licensed under a Creative Commons Attribution Non-Commercial No Derivatives License (CC-BY-NC-ND)</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by-nc-nd/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2022-02-08T12:54:32.2666876 v2 56831 2021-05-09 Pruning CNN filters via quantifying the importance of deep visual representations c0c682a8d9d12520f9639b89f9500946 Ali Alqahtani Ali Alqahtani true false b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2e1030b6e14fc9debd5d5ae7cc335562 0000-0001-8991-1190 Mark Jones Mark Jones true false 1e2e134b148109090dadd3c27585e0d5 NULL Ehab Mohamed Mahmoud Essa Ehab Mohamed Mahmoud Essa true true 2021-05-09 SCS The achievement of convolutional neural networks (CNNs) in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we pro- pose a novel framework to measure the importance of individual hidden units by computing a measure of relevance to identify the most critical filters and prune them to compress and accelerate CNNs. Unlike exist- ing methods, we introduce the use of the activation of feature maps to detect valuable information and the essential semantic parts to evaluate the importance of feature maps, inspired by novel neural network inter- pretability. A majority voting technique based on the degree of alignment between a semantic concept and individual hidden unit representations is proposed to quantitatively evaluate the importance of feature maps. We also propose a simple yet effective method to estimate new convolution kernels based on the remaining, crucial channels to accomplish effective CNN compression. Journal Article Computer Vision and Image Understanding 208-209 103220 Elsevier BV 1077-3142 Deep learning; Convolutional neural networks; Filter pruning; Model compression 1 7 2021 2021-07-01 10.1016/j.cviu.2021.103220 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University This work is supported by EPSRC, UK EP/N028139/1 2022-02-08T12:54:32.2666876 2021-05-09T11:54:08.1604963 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Ali Alqahtani 1 Xianghua Xie 0000-0002-2701-8660 2 Mark Jones 0000-0001-8991-1190 3 Ehab Mohamed Mahmoud Essa NULL 4 56831__19827__c64cae8a23ae419694fb3ec085e64f8a.pdf cviu_swansea.pdf 2021-05-09T11:56:41.1659385 Output 8655835 application/pdf Accepted Manuscript true 2022-05-18T00:00:00.0000000 ©2021 All rights reserved. All article content, except where otherwise noted, is licensed under a Creative Commons Attribution Non-Commercial No Derivatives License (CC-BY-NC-ND) true eng https://creativecommons.org/licenses/by-nc-nd/4.0/
title Pruning CNN filters via quantifying the importance of deep visual representations
spellingShingle Pruning CNN filters via quantifying the importance of deep visual representations
Ali Alqahtani
Xianghua Xie
Mark Jones
Ehab Mohamed Mahmoud Essa
title_short Pruning CNN filters via quantifying the importance of deep visual representations
title_full Pruning CNN filters via quantifying the importance of deep visual representations
title_fullStr Pruning CNN filters via quantifying the importance of deep visual representations
title_full_unstemmed Pruning CNN filters via quantifying the importance of deep visual representations
title_sort Pruning CNN filters via quantifying the importance of deep visual representations
author_id_str_mv c0c682a8d9d12520f9639b89f9500946
b334d40963c7a2f435f06d2c26c74e11
2e1030b6e14fc9debd5d5ae7cc335562
1e2e134b148109090dadd3c27585e0d5
author_id_fullname_str_mv c0c682a8d9d12520f9639b89f9500946_***_Ali Alqahtani
b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
2e1030b6e14fc9debd5d5ae7cc335562_***_Mark Jones
1e2e134b148109090dadd3c27585e0d5_***_Ehab Mohamed Mahmoud Essa
author Ali Alqahtani
Xianghua Xie
Mark Jones
Ehab Mohamed Mahmoud Essa
author2 Ali Alqahtani
Xianghua Xie
Mark Jones
Ehab Mohamed Mahmoud Essa
format Journal article
container_title Computer Vision and Image Understanding
container_volume 208-209
container_start_page 103220
publishDate 2021
institution Swansea University
issn 1077-3142
doi_str_mv 10.1016/j.cviu.2021.103220
publisher Elsevier BV
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description The achievement of convolutional neural networks (CNNs) in a variety of applications is accompanied by a dramatic increase in computational costs and memory requirements. In this paper, we pro- pose a novel framework to measure the importance of individual hidden units by computing a measure of relevance to identify the most critical filters and prune them to compress and accelerate CNNs. Unlike exist- ing methods, we introduce the use of the activation of feature maps to detect valuable information and the essential semantic parts to evaluate the importance of feature maps, inspired by novel neural network inter- pretability. A majority voting technique based on the degree of alignment between a semantic concept and individual hidden unit representations is proposed to quantitatively evaluate the importance of feature maps. We also propose a simple yet effective method to estimate new convolution kernels based on the remaining, crucial channels to accomplish effective CNN compression.
published_date 2021-07-01T04:12:05Z
_version_ 1763753827013492736
score 11.036706

Similar Items