No Cover Image

Conference Paper/Proceeding/Abstract 324 views 27 downloads

Effective Video Mirror Detection with Inconsistent Motion Cues

Alex Warren, Ke Xu, Jiaying Lin, Gary Tam Orcid Logo, Rynson W.H. Lau, Rynson Lau

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Pages: 17244 - 17252

Swansea University Authors: Alex Warren, Gary Tam Orcid Logo, Rynson Lau

  • cvpr2024_supp.pdf

    PDF | Accepted Manuscript

    Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).

    Download (5.79MB)

Check full text

DOI (Published version): 10.1109/cvpr52733.2024.01632

Abstract

Image-based mirror detection has recently undergone rapid research due to its significance in applications such as robotic navigation, semantic segmentation and scene re-construction. Recently, VMD-Net was proposed as the first video mirror detection technique, by modeling dual correspondences betwe...

Full description

Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
ISBN: 979-8-3503-5301-3 979-8-3503-5300-6
ISSN: 1063-6919 2575-7075
Published: IEEE 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa65886
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2024-04-15T13:43:24Z
last_indexed 2024-04-15T13:43:24Z
id cronfa65886
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>65886</id><entry>2024-03-23</entry><title>Effective Video Mirror Detection with Inconsistent Motion Cues</title><swanseaauthors><author><sid>38cd1eebf16295dbe5e1ff6769d6af69</sid><firstname>Alex</firstname><surname>Warren</surname><name>Alex Warren</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>e75a68e11a20e5f1da94ee6e28ff5e76</sid><ORCID>0000-0001-7387-5180</ORCID><firstname>Gary</firstname><surname>Tam</surname><name>Gary Tam</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>8d230434b6eadb1be5928241b0beecd0</sid><firstname>Rynson</firstname><surname>Lau</surname><name>Rynson Lau</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-03-23</date><deptcode>MACS</deptcode><abstract>Image-based mirror detection has recently undergone rapid research due to its significance in applications such as robotic navigation, semantic segmentation and scene re-construction. Recently, VMD-Net was proposed as the first video mirror detection technique, by modeling dual correspondences between the inside and outside of the mirror both spatially and temporally. However, this approach is not reliable, as correspondences can occur completely inside or outside of the mirrors. In addition, the proposed dataset VMD-D contains many small mirrors, limiting its applicability to real-world scenarios. To address these problems, we developed a more challenging dataset that includes mirrors of various shapes and sizes at different locations of the frames, providing a better reflection of real-world scenarios. Next, we observed that the motions between the inside and outside of the mirror are often in-consistent. For instance, when moving in front of a mirror, the motion inside the mirror is often much smaller than the motion outside due to increased depth perception. With these observations, we propose modeling inconsistent motion cues to detect mirrors, and a new network with two novel modules. The Motion Attention Module (MAM) ex-plicitly models inconsistent motions around mirrors via optical flow, and the Motion-Guided Edge Detection Module (MEDM) uses motions to guide mirror edge feature learning. Experimental results on our proposed dataset show that our method outperforms state-of-the-arts. The code and dataset are available at ht tps: // gi th ub. com/ AlexAnthonyWarren/MG-VMD.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)</journal><volume>0</volume><journalNumber/><paginationStart>17244</paginationStart><paginationEnd>17252</paginationEnd><publisher>IEEE</publisher><placeOfPublication/><isbnPrint>979-8-3503-5301-3</isbnPrint><isbnElectronic>979-8-3503-5300-6</isbnElectronic><issnPrint>1063-6919</issnPrint><issnElectronic>2575-7075</issnElectronic><keywords/><publishedDay>16</publishedDay><publishedMonth>9</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-09-16</publishedDate><doi>10.1109/cvpr52733.2024.01632</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Not Required</apcterm><funders>Alex is supported by a Swansea GTA Research Scholarship. This project is in part supported by a GRF grant from the Research Grants Council of Hong Kong (Ref.: 11211223). We gratefully acknowledge the support of the HEFCW HERC fund (W21/21HE) for the provision of GPU equipment used in this research.</funders><projectreference>Alex is supported by a Swansea GTA Research Scholarship. This project is in part supported by a GRF grant from the Research Grants Council of Hong Kong (Ref.: 11211223). We gratefully acknowledge the support of the HEFCW HERC fund (W21/21HE) for the provision of GPU equipment used in this research.</projectreference><lastEdited>2024-10-02T11:55:47.5879020</lastEdited><Created>2024-03-23T18:38:09.9376188</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Alex</firstname><surname>Warren</surname><order>1</order></author><author><firstname>Ke</firstname><surname>Xu</surname><order>2</order></author><author><firstname>Jiaying</firstname><surname>Lin</surname><order>3</order></author><author><firstname>Gary</firstname><surname>Tam</surname><orcid>0000-0001-7387-5180</orcid><order>4</order></author><author><firstname>Rynson W.H.</firstname><surname>Lau</surname><order>5</order></author><author><firstname>Rynson</firstname><surname>Lau</surname><order>6</order></author></authors><documents><document><filename>65886__29817__f739ba90ec0a4f189b62a73a302d042e.pdf</filename><originalFilename>cvpr2024_supp.pdf</originalFilename><uploaded>2024-03-25T10:03:41.6058758</uploaded><type>Output</type><contentLength>6074495</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><documentNotes>Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/deed.en</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 65886 2024-03-23 Effective Video Mirror Detection with Inconsistent Motion Cues 38cd1eebf16295dbe5e1ff6769d6af69 Alex Warren Alex Warren true false e75a68e11a20e5f1da94ee6e28ff5e76 0000-0001-7387-5180 Gary Tam Gary Tam true false 8d230434b6eadb1be5928241b0beecd0 Rynson Lau Rynson Lau true false 2024-03-23 MACS Image-based mirror detection has recently undergone rapid research due to its significance in applications such as robotic navigation, semantic segmentation and scene re-construction. Recently, VMD-Net was proposed as the first video mirror detection technique, by modeling dual correspondences between the inside and outside of the mirror both spatially and temporally. However, this approach is not reliable, as correspondences can occur completely inside or outside of the mirrors. In addition, the proposed dataset VMD-D contains many small mirrors, limiting its applicability to real-world scenarios. To address these problems, we developed a more challenging dataset that includes mirrors of various shapes and sizes at different locations of the frames, providing a better reflection of real-world scenarios. Next, we observed that the motions between the inside and outside of the mirror are often in-consistent. For instance, when moving in front of a mirror, the motion inside the mirror is often much smaller than the motion outside due to increased depth perception. With these observations, we propose modeling inconsistent motion cues to detect mirrors, and a new network with two novel modules. The Motion Attention Module (MAM) ex-plicitly models inconsistent motions around mirrors via optical flow, and the Motion-Guided Edge Detection Module (MEDM) uses motions to guide mirror edge feature learning. Experimental results on our proposed dataset show that our method outperforms state-of-the-arts. The code and dataset are available at ht tps: // gi th ub. com/ AlexAnthonyWarren/MG-VMD. Conference Paper/Proceeding/Abstract 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 0 17244 17252 IEEE 979-8-3503-5301-3 979-8-3503-5300-6 1063-6919 2575-7075 16 9 2024 2024-09-16 10.1109/cvpr52733.2024.01632 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Not Required Alex is supported by a Swansea GTA Research Scholarship. This project is in part supported by a GRF grant from the Research Grants Council of Hong Kong (Ref.: 11211223). We gratefully acknowledge the support of the HEFCW HERC fund (W21/21HE) for the provision of GPU equipment used in this research. Alex is supported by a Swansea GTA Research Scholarship. This project is in part supported by a GRF grant from the Research Grants Council of Hong Kong (Ref.: 11211223). We gratefully acknowledge the support of the HEFCW HERC fund (W21/21HE) for the provision of GPU equipment used in this research. 2024-10-02T11:55:47.5879020 2024-03-23T18:38:09.9376188 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Alex Warren 1 Ke Xu 2 Jiaying Lin 3 Gary Tam 0000-0001-7387-5180 4 Rynson W.H. Lau 5 Rynson Lau 6 65886__29817__f739ba90ec0a4f189b62a73a302d042e.pdf cvpr2024_supp.pdf 2024-03-25T10:03:41.6058758 Output 6074495 application/pdf Accepted Manuscript true Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention). true eng https://creativecommons.org/licenses/by/4.0/deed.en
title Effective Video Mirror Detection with Inconsistent Motion Cues
spellingShingle Effective Video Mirror Detection with Inconsistent Motion Cues
Alex Warren
Gary Tam
Rynson Lau
title_short Effective Video Mirror Detection with Inconsistent Motion Cues
title_full Effective Video Mirror Detection with Inconsistent Motion Cues
title_fullStr Effective Video Mirror Detection with Inconsistent Motion Cues
title_full_unstemmed Effective Video Mirror Detection with Inconsistent Motion Cues
title_sort Effective Video Mirror Detection with Inconsistent Motion Cues
author_id_str_mv 38cd1eebf16295dbe5e1ff6769d6af69
e75a68e11a20e5f1da94ee6e28ff5e76
8d230434b6eadb1be5928241b0beecd0
author_id_fullname_str_mv 38cd1eebf16295dbe5e1ff6769d6af69_***_Alex Warren
e75a68e11a20e5f1da94ee6e28ff5e76_***_Gary Tam
8d230434b6eadb1be5928241b0beecd0_***_Rynson Lau
author Alex Warren
Gary Tam
Rynson Lau
author2 Alex Warren
Ke Xu
Jiaying Lin
Gary Tam
Rynson W.H. Lau
Rynson Lau
format Conference Paper/Proceeding/Abstract
container_title 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
container_volume 0
container_start_page 17244
publishDate 2024
institution Swansea University
isbn 979-8-3503-5301-3
979-8-3503-5300-6
issn 1063-6919
2575-7075
doi_str_mv 10.1109/cvpr52733.2024.01632
publisher IEEE
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Image-based mirror detection has recently undergone rapid research due to its significance in applications such as robotic navigation, semantic segmentation and scene re-construction. Recently, VMD-Net was proposed as the first video mirror detection technique, by modeling dual correspondences between the inside and outside of the mirror both spatially and temporally. However, this approach is not reliable, as correspondences can occur completely inside or outside of the mirrors. In addition, the proposed dataset VMD-D contains many small mirrors, limiting its applicability to real-world scenarios. To address these problems, we developed a more challenging dataset that includes mirrors of various shapes and sizes at different locations of the frames, providing a better reflection of real-world scenarios. Next, we observed that the motions between the inside and outside of the mirror are often in-consistent. For instance, when moving in front of a mirror, the motion inside the mirror is often much smaller than the motion outside due to increased depth perception. With these observations, we propose modeling inconsistent motion cues to detect mirrors, and a new network with two novel modules. The Motion Attention Module (MAM) ex-plicitly models inconsistent motions around mirrors via optical flow, and the Motion-Guided Edge Detection Module (MEDM) uses motions to guide mirror edge feature learning. Experimental results on our proposed dataset show that our method outperforms state-of-the-arts. The code and dataset are available at ht tps: // gi th ub. com/ AlexAnthonyWarren/MG-VMD.
published_date 2024-09-16T11:55:47Z
_version_ 1811799392500318208
score 11.035634

Similar Items