No Cover Image

Journal article 685 views 101 downloads

Literature Review of Deep Network Compression

Ali Alqahtani, Xianghua Xie Orcid Logo, Mark Jones Orcid Logo

Informatics, Volume: 8, Issue: 4, Start page: 77

Swansea University Authors: Xianghua Xie Orcid Logo, Mark Jones Orcid Logo

  • 58687.VOR.pdf

    PDF | Version of Record

    Copyright: © 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) Licence.

    Download (232.3KB)

Check full text

DOI (Published version): 10.3390/informatics8040077

Abstract

Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on reducing the complexity of models while maint...

Full description

Published in: Informatics
ISSN: 2227-9709
Published: MDPI AG 2021
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa58687
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2021-11-17T08:30:38Z
last_indexed 2021-12-08T04:18:54Z
id cronfa58687
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2021-12-07T15:00:58.5976893</datestamp><bib-version>v2</bib-version><id>58687</id><entry>2021-11-17</entry><title>Literature Review of Deep Network Compression</title><swanseaauthors><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>2e1030b6e14fc9debd5d5ae7cc335562</sid><ORCID>0000-0001-8991-1190</ORCID><firstname>Mark</firstname><surname>Jones</surname><name>Mark Jones</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2021-11-17</date><deptcode>SCS</deptcode><abstract>Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on reducing the complexity of models while maintaining their powerful performance. In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks. We consider not only pruning methods but also quantization methods, and low-rank factorization methods. This review also intends to clarify these major concepts, and highlights their characteristics, advantages, and shortcomings.</abstract><type>Journal Article</type><journal>Informatics</journal><volume>8</volume><journalNumber>4</journalNumber><paginationStart>77</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2227-9709</issnElectronic><keywords>deep learning; neural networks pruning; model compression</keywords><publishedDay>17</publishedDay><publishedMonth>11</publishedMonth><publishedYear>2021</publishedYear><publishedDate>2021-11-17</publishedDate><doi>10.3390/informatics8040077</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This work was supported by the Deanship of Scientific Research, King Khalid University of Kingdom of Saudi Arabia under research grant number (RGP1/207/42).</funders><lastEdited>2021-12-07T15:00:58.5976893</lastEdited><Created>2021-11-17T08:26:18.4110099</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Ali</firstname><surname>Alqahtani</surname><order>1</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>2</order></author><author><firstname>Mark</firstname><surname>Jones</surname><orcid>0000-0001-8991-1190</orcid><order>3</order></author></authors><documents><document><filename>58687__21577__b2f1ed63dee54d52899d3e28bf731e58.pdf</filename><originalFilename>58687.VOR.pdf</originalFilename><uploaded>2021-11-18T13:00:40.8878145</uploaded><type>Output</type><contentLength>237872</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>Copyright: &#xA9; 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) Licence.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2021-12-07T15:00:58.5976893 v2 58687 2021-11-17 Literature Review of Deep Network Compression b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2e1030b6e14fc9debd5d5ae7cc335562 0000-0001-8991-1190 Mark Jones Mark Jones true false 2021-11-17 SCS Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on reducing the complexity of models while maintaining their powerful performance. In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks. We consider not only pruning methods but also quantization methods, and low-rank factorization methods. This review also intends to clarify these major concepts, and highlights their characteristics, advantages, and shortcomings. Journal Article Informatics 8 4 77 MDPI AG 2227-9709 deep learning; neural networks pruning; model compression 17 11 2021 2021-11-17 10.3390/informatics8040077 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Another institution paid the OA fee This work was supported by the Deanship of Scientific Research, King Khalid University of Kingdom of Saudi Arabia under research grant number (RGP1/207/42). 2021-12-07T15:00:58.5976893 2021-11-17T08:26:18.4110099 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Ali Alqahtani 1 Xianghua Xie 0000-0002-2701-8660 2 Mark Jones 0000-0001-8991-1190 3 58687__21577__b2f1ed63dee54d52899d3e28bf731e58.pdf 58687.VOR.pdf 2021-11-18T13:00:40.8878145 Output 237872 application/pdf Version of Record true Copyright: © 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) Licence. true eng https://creativecommons.org/licenses/by/4.0/
title Literature Review of Deep Network Compression
spellingShingle Literature Review of Deep Network Compression
Xianghua Xie
Mark Jones
title_short Literature Review of Deep Network Compression
title_full Literature Review of Deep Network Compression
title_fullStr Literature Review of Deep Network Compression
title_full_unstemmed Literature Review of Deep Network Compression
title_sort Literature Review of Deep Network Compression
author_id_str_mv b334d40963c7a2f435f06d2c26c74e11
2e1030b6e14fc9debd5d5ae7cc335562
author_id_fullname_str_mv b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
2e1030b6e14fc9debd5d5ae7cc335562_***_Mark Jones
author Xianghua Xie
Mark Jones
author2 Ali Alqahtani
Xianghua Xie
Mark Jones
format Journal article
container_title Informatics
container_volume 8
container_issue 4
container_start_page 77
publishDate 2021
institution Swansea University
issn 2227-9709
doi_str_mv 10.3390/informatics8040077
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on reducing the complexity of models while maintaining their powerful performance. In this paper, we present an overview of popular methods and review recent works on compressing and accelerating deep neural networks. We consider not only pruning methods but also quantization methods, and low-rank factorization methods. This review also intends to clarify these major concepts, and highlights their characteristics, advantages, and shortcomings.
published_date 2021-11-17T04:15:24Z
_version_ 1763754036127858688
score 11.016235

Similar Items