Journal article 267 views 8 downloads
Inferring Attention Shifts for Salient Instance Ranking
,
Xianghua Xie ,
Rynson Lau
International Journal of Computer Vision, Volume: 132, Issue: 3, Pages: 964 - 986
Swansea University Authors:
Avishek Siris, Gary Tam , Xianghua Xie , Rynson Lau
-
PDF | Version of Record
© The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License.
Download (4.23MB)
DOI (Published version): 10.1007/s11263-023-01906-7
Abstract
The human visual system has limited capacity in simultaneously processing multiple visual inputs. Consequently, humans rely on shifting their attention from one location to another. When viewing an image of complex scenes, psychology studies and behavioural observations show that humans prioritise a...
| Published in: | International Journal of Computer Vision |
|---|---|
| ISSN: | 0920-5691 1573-1405 |
| Published: |
Springer Science and Business Media LLC
2024
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa64287 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract: |
The human visual system has limited capacity in simultaneously processing multiple visual inputs. Consequently, humans rely on shifting their attention from one location to another. When viewing an image of complex scenes, psychology studies and behavioural observations show that humans prioritise and sequentially shift attention among multiple visual stimuli. In this paper, we propose to predict the saliency rank of multiple objects by inferring human attention shift. We first construct a new large-scale salient object ranking dataset, with the saliency rank of objects defined by the order that an observer attends to these objects via attention shift. We then propose a new deep learning based model to leverage both bottom-up and top-down attention mechanisms for saliency rank prediction. Our model includes three novel modules: Spatial Mask Module (SMM), Selective Attention Module (SAM) and Salient Instance Edge Module (SIEM). SMM integrates bottom-up and semantic object properties to enhance contextual object features, from which SAM learns the dependencies between object features and image features for saliency reasoning. SIEM is designed to improve segmentation of salient objects, which helps further improve their rank predictions. Experimental results show that our proposed network achieves state-of-the-art performances on the salient object ranking task across multiple datasets. |
|---|---|
| Keywords: |
Attention Shift, Saliency, Saliency Ranking, Salient Object Detection. |
| College: |
Faculty of Science and Engineering |
| Funders: |
This work was funded by a Swansea University Doctoral Training Postgraduate Research Scholarship 0301[164]. For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission. Jianbo Jiao is supported by the EPSRC Programme Grant Visual AI EP/T028572/1. Gary Tam is supported by the Royal Society grant IEC/NSFC/211159. This work was supported by the Research Grants Council of Hong Kong (Grant No.: 11205620), and a Strategic Research Grant from City University of Hong Kong (Ref.: 7005674). |
| Issue: |
3 |
| Start Page: |
964 |
| End Page: |
986 |