A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning

Lin, Hexiu; Liu, Yukun; Shi, Daming; Cheng, Cheng

doi:10.3390/math11153394

Journal article 46 views 15 downloads

A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning

Hexiu Lin

, Yukun Liu

, Daming Shi, Cheng Cheng

Mathematics, Volume: 11, Issue: 15, Start page: 3394

Swansea University Author: Cheng Cheng

PDF | Version of Record

© 2023 by the authors. This is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Download (1.44MB)

Check full text

DOI (Published version): 10.3390/math11153394

Abstract

Learning novel classes with a few samples per class is a very challenging task in deep learning. To mitigate this issue, previous studies have utilized an additional dataset with extensively labeled samples to realize transfer learning. Alternatively, many studies have used unlabeled samples that or...

Full description

Published in:	Mathematics
ISSN:	2227-7390
Published:	MDPI AG 2023
Online Access:	Check full text
URI:	https://cronfa.swan.ac.uk/Record/cronfa65944
Tags:	Add Tag No Tags, Be the first to tag this record!

first_indexed	2024-05-29T15:10:28Z
last_indexed	2024-05-29T15:10:28Z
id	cronfa65944
recordtype	SURis
fullrecord	<?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>65944</id><entry>2024-04-03</entry><title>A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning</title><swanseaauthors><author><sid>11ddf61c123b99e59b00fa1479367582</sid><ORCID>0000-0003-0371-9646</ORCID><firstname>Cheng</firstname><surname>Cheng</surname><name>Cheng Cheng</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-04-03</date><deptcode>MACS</deptcode><abstract>Learning novel classes with a few samples per class is a very challenging task in deep learning. To mitigate this issue, previous studies have utilized an additional dataset with extensively labeled samples to realize transfer learning. Alternatively, many studies have used unlabeled samples that originated from the novel dataset to achieve few-shot learning, i.e., semi-supervised few-shot learning. In this paper, an easy but efficient semi-supervised few-shot learning model is proposed to address the embeddings mismatch problem that results from inconsistent data distributions between the novel and base datasets, where samples with the same label approach each other while samples with different labels separate from each other in the feature space. This model emphasizes pseudo-labeling guided contrastive learning. We also develop a novel local factor clustering module to improve the ability to obtain pseudo-labels from unlabeled samples, and this module fuses the local feature information of labeled and unlabeled samples. We report our experimental results on the mini-ImageNet and tiered-ImageNet datasets for both five-way one-shot and five-way five-shot settings and achieve better performance than previous models. In particular, the classification accuracy of our model is improved by approximately 11.53% and 14.87% compared to the most advanced semi-supervised few-shot learning model we know in the five-way one-shot scenario. Moreover, ablation experiments in this paper show that our proposed clustering strategy demonstrates accuracy improvements of about 4.00% in the five-way one-shot and five-way five-shot scenarios compared to two popular clustering methods.</abstract><type>Journal Article</type><journal>Mathematics</journal><volume>11</volume><journalNumber>15</journalNumber><paginationStart>3394</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2227-7390</issnElectronic><keywords>few-shot learning; clustering; semi-supervised learning; local features; contrastive learning</keywords><publishedDay>3</publishedDay><publishedMonth>8</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-08-03</publishedDate><doi>10.3390/math11153394</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This work is supported by Ministry of Science and Technology China (MOST) Major Program on New Generation of Artificial Intelligence 2030 No. 2018AAA0102200. It is also supported by Natural Science Foundation China (NSFC) Major Projects No. U22A2097 and No. 61827814, as well as Shenzhen Science and Technology Innovation Commission (SZSTI) project No. JCYJ20190808153619413.</funders><projectreference/><lastEdited>2024-05-29T16:12:24.5539835</lastEdited><Created>2024-04-03T17:21:58.8909510</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Hexiu</firstname><surname>Lin</surname><orcid>0000-0003-2356-5154</orcid><order>1</order></author><author><firstname>Yukun</firstname><surname>Liu</surname><orcid>0000-0003-2994-5393</orcid><order>2</order></author><author><firstname>Daming</firstname><surname>Shi</surname><order>3</order></author><author><firstname>Cheng</firstname><surname>Cheng</surname><orcid>0000-0003-0371-9646</orcid><order>4</order></author></authors><documents><document><filename>65944__30485__a70e21d6722b48c09b50dd1ac5376c61.pdf</filename><originalFilename>65944.VoR.pdf</originalFilename><uploaded>2024-05-29T16:10:58.4288220</uploaded><type>Output</type><contentLength>1510121</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>© 2023 by the authors. This is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling	v2 65944 2024-04-03 A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning 11ddf61c123b99e59b00fa1479367582 0000-0003-0371-9646 Cheng Cheng Cheng Cheng true false 2024-04-03 MACS Learning novel classes with a few samples per class is a very challenging task in deep learning. To mitigate this issue, previous studies have utilized an additional dataset with extensively labeled samples to realize transfer learning. Alternatively, many studies have used unlabeled samples that originated from the novel dataset to achieve few-shot learning, i.e., semi-supervised few-shot learning. In this paper, an easy but efficient semi-supervised few-shot learning model is proposed to address the embeddings mismatch problem that results from inconsistent data distributions between the novel and base datasets, where samples with the same label approach each other while samples with different labels separate from each other in the feature space. This model emphasizes pseudo-labeling guided contrastive learning. We also develop a novel local factor clustering module to improve the ability to obtain pseudo-labels from unlabeled samples, and this module fuses the local feature information of labeled and unlabeled samples. We report our experimental results on the mini-ImageNet and tiered-ImageNet datasets for both five-way one-shot and five-way five-shot settings and achieve better performance than previous models. In particular, the classification accuracy of our model is improved by approximately 11.53% and 14.87% compared to the most advanced semi-supervised few-shot learning model we know in the five-way one-shot scenario. Moreover, ablation experiments in this paper show that our proposed clustering strategy demonstrates accuracy improvements of about 4.00% in the five-way one-shot and five-way five-shot scenarios compared to two popular clustering methods. Journal Article Mathematics 11 15 3394 MDPI AG 2227-7390 few-shot learning; clustering; semi-supervised learning; local features; contrastive learning 3 8 2023 2023-08-03 10.3390/math11153394 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Another institution paid the OA fee This work is supported by Ministry of Science and Technology China (MOST) Major Program on New Generation of Artificial Intelligence 2030 No. 2018AAA0102200. It is also supported by Natural Science Foundation China (NSFC) Major Projects No. U22A2097 and No. 61827814, as well as Shenzhen Science and Technology Innovation Commission (SZSTI) project No. JCYJ20190808153619413. 2024-05-29T16:12:24.5539835 2024-04-03T17:21:58.8909510 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Hexiu Lin 0000-0003-2356-5154 1 Yukun Liu 0000-0003-2994-5393 2 Daming Shi 3 Cheng Cheng 0000-0003-0371-9646 4 65944__30485__a70e21d6722b48c09b50dd1ac5376c61.pdf 65944.VoR.pdf 2024-05-29T16:10:58.4288220 Output 1510121 application/pdf Version of Record true © 2023 by the authors. This is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. true eng https://creativecommons.org/licenses/by/4.0/
title	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
spellingShingle	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning Cheng Cheng
title_short	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
title_full	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
title_fullStr	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
title_full_unstemmed	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
title_sort	A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning
author_id_str_mv	11ddf61c123b99e59b00fa1479367582
author_id_fullname_str_mv	11ddf61c123b99e59b00fa1479367582_***_Cheng Cheng
author	Cheng Cheng
author2	Hexiu Lin Yukun Liu Daming Shi Cheng Cheng
format	Journal article
container_title	Mathematics
container_volume	11
container_issue	15
container_start_page	3394
publishDate	2023
institution	Swansea University
issn	2227-7390
doi_str_mv	10.3390/math11153394
publisher	MDPI AG
college_str	Faculty of Science and Engineering
hierarchytype
hierarchy_top_id	facultyofscienceandengineering
hierarchy_top_title	Faculty of Science and Engineering
hierarchy_parent_id	facultyofscienceandengineering
hierarchy_parent_title	Faculty of Science and Engineering
department_str	School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str	1
active_str	0
description	Learning novel classes with a few samples per class is a very challenging task in deep learning. To mitigate this issue, previous studies have utilized an additional dataset with extensively labeled samples to realize transfer learning. Alternatively, many studies have used unlabeled samples that originated from the novel dataset to achieve few-shot learning, i.e., semi-supervised few-shot learning. In this paper, an easy but efficient semi-supervised few-shot learning model is proposed to address the embeddings mismatch problem that results from inconsistent data distributions between the novel and base datasets, where samples with the same label approach each other while samples with different labels separate from each other in the feature space. This model emphasizes pseudo-labeling guided contrastive learning. We also develop a novel local factor clustering module to improve the ability to obtain pseudo-labels from unlabeled samples, and this module fuses the local feature information of labeled and unlabeled samples. We report our experimental results on the mini-ImageNet and tiered-ImageNet datasets for both five-way one-shot and five-way five-shot settings and achieve better performance than previous models. In particular, the classification accuracy of our model is improved by approximately 11.53% and 14.87% compared to the most advanced semi-supervised few-shot learning model we know in the five-way one-shot scenario. Moreover, ablation experiments in this paper show that our proposed clustering strategy demonstrates accuracy improvements of about 4.00% in the five-way one-shot and five-way five-shot scenarios compared to two popular clustering methods.
published_date	2023-08-03T16:12:22Z
_version_	1800400318515642368
score	11.012678

A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning

Similar Items