E-Thesis 398 views 200 downloads
Face Reenactment with Generative Landmark Guidance / CHEN HU
Swansea University Author: CHEN HU
Abstract
Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving pro...
Published: |
Swansea
2023
|
---|---|
Institution: | Swansea University |
Degree level: | Master of Research |
Degree name: | MSc by Research |
Supervisor: | Xie, Xianghua |
URI: | https://cronfa.swan.ac.uk/Record/cronfa62597 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
first_indexed |
2023-02-07T11:03:49Z |
---|---|
last_indexed |
2023-02-08T04:17:09Z |
id |
cronfa62597 |
recordtype |
RisThesis |
fullrecord |
<?xml version="1.0"?><rfc1807><datestamp>2023-02-07T11:12:10.5284680</datestamp><bib-version>v2</bib-version><id>62597</id><entry>2023-02-07</entry><title>Face Reenactment with Generative Landmark Guidance</title><swanseaauthors><author><sid>f166c1a306bc35bfaef8c8a4189752d0</sid><firstname>CHEN</firstname><surname>HU</surname><name>CHEN HU</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-02-07</date><abstract>Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning.</abstract><type>E-Thesis</type><journal/><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher/><placeOfPublication>Swansea</placeOfPublication><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic/><keywords>face reeanctment, optical flow, style transfer, federated learning, generative adversarial network</keywords><publishedDay>1</publishedDay><publishedMonth>2</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-02-01</publishedDate><doi/><url/><notes/><college>COLLEGE NANME</college><CollegeCode>COLLEGE CODE</CollegeCode><institution>Swansea University</institution><supervisor>Xie, Xianghua</supervisor><degreelevel>Master of Research</degreelevel><degreename>MSc by Research</degreename><apcterm/><funders/><projectreference/><lastEdited>2023-02-07T11:12:10.5284680</lastEdited><Created>2023-02-07T11:01:16.8969064</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>CHEN</firstname><surname>HU</surname><order>1</order></author></authors><documents><document><filename>62597__26501__b4c07b6fbca941178c7c94df3c8f1d0a.pdf</filename><originalFilename>Hu_Chen_MSc_Research_Thesis_Final_Redacted_Signature.pdf</originalFilename><uploaded>2023-02-07T11:07:08.6781923</uploaded><type>Output</type><contentLength>6740201</contentLength><contentType>application/pdf</contentType><version>E-Thesis – open access</version><cronfaStatus>true</cronfaStatus><documentNotes>Copyright: The author, Chen Hu, 2023.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language></document></documents><OutputDurs/></rfc1807> |
spelling |
2023-02-07T11:12:10.5284680 v2 62597 2023-02-07 Face Reenactment with Generative Landmark Guidance f166c1a306bc35bfaef8c8a4189752d0 CHEN HU CHEN HU true false 2023-02-07 Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning. E-Thesis Swansea face reeanctment, optical flow, style transfer, federated learning, generative adversarial network 1 2 2023 2023-02-01 COLLEGE NANME COLLEGE CODE Swansea University Xie, Xianghua Master of Research MSc by Research 2023-02-07T11:12:10.5284680 2023-02-07T11:01:16.8969064 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science CHEN HU 1 62597__26501__b4c07b6fbca941178c7c94df3c8f1d0a.pdf Hu_Chen_MSc_Research_Thesis_Final_Redacted_Signature.pdf 2023-02-07T11:07:08.6781923 Output 6740201 application/pdf E-Thesis – open access true Copyright: The author, Chen Hu, 2023. true eng |
title |
Face Reenactment with Generative Landmark Guidance |
spellingShingle |
Face Reenactment with Generative Landmark Guidance CHEN HU |
title_short |
Face Reenactment with Generative Landmark Guidance |
title_full |
Face Reenactment with Generative Landmark Guidance |
title_fullStr |
Face Reenactment with Generative Landmark Guidance |
title_full_unstemmed |
Face Reenactment with Generative Landmark Guidance |
title_sort |
Face Reenactment with Generative Landmark Guidance |
author_id_str_mv |
f166c1a306bc35bfaef8c8a4189752d0 |
author_id_fullname_str_mv |
f166c1a306bc35bfaef8c8a4189752d0_***_CHEN HU |
author |
CHEN HU |
author2 |
CHEN HU |
format |
E-Thesis |
publishDate |
2023 |
institution |
Swansea University |
college_str |
Faculty of Science and Engineering |
hierarchytype |
|
hierarchy_top_id |
facultyofscienceandengineering |
hierarchy_top_title |
Faculty of Science and Engineering |
hierarchy_parent_id |
facultyofscienceandengineering |
hierarchy_parent_title |
Faculty of Science and Engineering |
department_str |
School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science |
document_store_str |
1 |
active_str |
0 |
description |
Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning. |
published_date |
2023-02-01T04:22:20Z |
_version_ |
1763754471939112960 |
score |
11.01628 |