AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
Experiments demonstrated that our Intra-saliency Correlation Network achieves better performance than state-of-the-art Co-Saliency Object Detection methods on three benchmarks

ICNet: Intra-saliency Correlation Network for Co-Saliency Detection

NIPS 2020, (2020)

Cited by: 3|Views101
EI
Full Text
Bibtex
Weibo

Abstract

Intra-saliency and inter-saliency cues3 have been extensively studied for co-saliency detection (Co-SOD). Model-based methods produce coarse Co-SOD results due to hand-crafted intra- and inter-saliency features. Current data-driven models exploit inter-saliency cues, but undervalue the potential power of intra-saliency cues. In this paper...More

Code:

Data:

0
Introduction
  • Co-Saliency Object Detection (Co-SOD) aims to discover the commonly salient objects in a group of relevant images [36].
  • It serves as a preliminary step for various computer vision tasks, e.g., co-segmentation [9], co-localization [27], and image retrieval [21], etc.
  • With subjective priors [4, 8], e.g., low-rank constraint, central bias rule, co-saliency distribution consistency and histogram-based contrast, these Co-SOD methods [4, 8] are usually unstable in capturing robust inter cues in complex real-world scenarios [36, 43]
Highlights
  • Co-Saliency Object Detection (Co-SOD) aims to discover the commonly salient objects in a group of relevant images [36]
  • Extensive experiments on three Co-SOD benchmarks demonstrate that our Intra-saliency Correlation Network (ICNet) outperforms stateof-the-art Co-SOD methods on standard objective metrics and subjective visual quality
  • In order to maintain the consistency of features F and co-salient attention (CSA) maps A in terms of category independence, we propose to compute the self-correlation within features F and an additional rearranging operation, obtaining the rearranged self-correlation features (RSCFs) F r = {Fir}ni=1 (§3.4)
  • We proposed an Intra-saliency Correlation Network (ICNet) for co-saliency detection (Co-SOD)
  • By leveraging correlations within the image features, we devised a Rearranged SelfCorrelation Feature strategy combined with the inter cues, to further boost our ICNet on Co-SOD
  • Experiments demonstrated that our ICNet achieves better performance than state-of-the-art Co-SOD methods on three benchmarks
Methods
  • The authors compare the ICNet with seven state-of-the-art Co-SOD methods and two well known Single-SOD ones.
  • The evaluations on Cosal2015 could reflect better the capability of the comparison methods on Co-SOD than those on MSRC and iCoseg.
  • The authors' ICNet obtains large improvements on the challenging Cosal2015, but small gains on MSRC and iCoseg.
  • This demonstrates that the ICNet is very capable of tackling the Co-SOD problem.
  • It can be seen that, on MSRC and iCoseg, the gains on Fβ and Sα are
Results
  • Evaluation metrics

    To quantitatively evaluate the performance of the ICNet, the authors adopt three widelyused metrics, including max F-measure score [1], S-measure [6] and mean absolute error (MAE) [3].

    Datasets.
  • The authors compare the ICNet with state-of-the-art competitors on three popular benchmarks: MSRC [33], iCoseg [2] and Cosal2015 [37].
  • Cosal2015 [37] includes 50 groups of 2015 images, and each group has 25 ∼ 52 images
  • It is a more challenging benchmark due to the diverse variance in the appearance of co-salient objects with complex backgrounds
Conclusion
  • The authors proposed an Intra-saliency Correlation Network (ICNet) for co-saliency detection (Co-SOD).
  • By directly integrating single image saliency maps produced by any off-the-shelf SOD method into the deep neural network for discriminative intra cues extraction, the authors further exploited correlations between intra cues and single-image features to capture accurate inter cues for CoSOD.
  • Experiments demonstrated that the ICNet achieves better performance than state-of-the-art Co-SOD methods on three benchmarks.
Summary
  • Introduction:

    Co-Saliency Object Detection (Co-SOD) aims to discover the commonly salient objects in a group of relevant images [36].
  • It serves as a preliminary step for various computer vision tasks, e.g., co-segmentation [9], co-localization [27], and image retrieval [21], etc.
  • With subjective priors [4, 8], e.g., low-rank constraint, central bias rule, co-saliency distribution consistency and histogram-based contrast, these Co-SOD methods [4, 8] are usually unstable in capturing robust inter cues in complex real-world scenarios [36, 43]
  • Methods:

    The authors compare the ICNet with seven state-of-the-art Co-SOD methods and two well known Single-SOD ones.
  • The evaluations on Cosal2015 could reflect better the capability of the comparison methods on Co-SOD than those on MSRC and iCoseg.
  • The authors' ICNet obtains large improvements on the challenging Cosal2015, but small gains on MSRC and iCoseg.
  • This demonstrates that the ICNet is very capable of tackling the Co-SOD problem.
  • It can be seen that, on MSRC and iCoseg, the gains on Fβ and Sα are
  • Results:

    Evaluation metrics

    To quantitatively evaluate the performance of the ICNet, the authors adopt three widelyused metrics, including max F-measure score [1], S-measure [6] and mean absolute error (MAE) [3].

    Datasets.
  • The authors compare the ICNet with state-of-the-art competitors on three popular benchmarks: MSRC [33], iCoseg [2] and Cosal2015 [37].
  • Cosal2015 [37] includes 50 groups of 2015 images, and each group has 25 ∼ 52 images
  • It is a more challenging benchmark due to the diverse variance in the appearance of co-salient objects with complex backgrounds
  • Conclusion:

    The authors proposed an Intra-saliency Correlation Network (ICNet) for co-saliency detection (Co-SOD).
  • By directly integrating single image saliency maps produced by any off-the-shelf SOD method into the deep neural network for discriminative intra cues extraction, the authors further exploited correlations between intra cues and single-image features to capture accurate inter cues for CoSOD.
  • Experiments demonstrated that the ICNet achieves better performance than state-of-the-art Co-SOD methods on three benchmarks.
Tables
  • Table1: Quantitative comparisons on max F-measure (Fβ), S-measure (Sα) and MAE over three benchmark datasets. “Co” and “Sin” in the “Type” column represent the corresponding methods are Co-SOD models and Single-SOD ones, respectively. “↑” (“↓”) means that larger (smaller) is better. The best, second best and third best results are highlighted in red, blue and bold, respectively
  • Table2: Results of different variants to our ICNet. NFs: 2-normalized features. CFM: correlation fusion module (§3.3). SISMs: extracting SIVs via SISMs (§3.2). SCFs: self-correlation features. R: rearrange (the channel order of SCFs, §3.4)
  • Table3: Results of our ICNet with SISMs by various SOD methods. “Baseline”: the basic Single-SOD methods. “ICNet”: our ICNet with SISMs produced by corresponding SOD methods
  • Table4: Results of our ICNet with different batch size settings. “ntrain” and “ntest” denote training and test batch size, respectively
Download tables as Excel
Related work
  • Previous model-based Co-SOD methods [4, 14, 15] mainly utilized single image saliency maps (SISMs) produced by off-the-shelf SOD methods as intra-saliency cues, and explored various intersaliency cues for Co-SOD. The work of [14] measured the similarities between different regions as inter cues, and linearly integrated them with intra cues to derive the co-saliency maps. The method of [15] employed manifold ranking to explore inter cues based on intra cues. Specifically, each image in a group along with its intra cue was utilized to compute correlations with all images in that group. Based on the correlations produced by each pair of images, the inter consistency is extracted to generate final Co-SOD results. Under a low-rank constraint, the method of [4] fused SISMs yielded by multiple SOD models with adaptive weights for Co-SOD predictions. The weights indicate the importance of each SOD model, acting as inter cues to guide the fusion process. However, modelbased Co-SOD methods [4, 14, 15] are limited by hand-crafted features and manually-designed inter cues, which are not robust to complex real-world scenarios.
Funding
  • This work was supported in part by the Major Project for New Generation of AI under Grant No 2018AAA0100400, National Natural Science Foundation of China (61702359, 61922046), and Tianjin Natural Science Foundation (18ZXZNGX00110)
Study subjects and analysis
benchmark datasets: 3
. Quantitative comparisons on max F-measure (Fβ), S-measure (Sα) and MAE over three benchmark datasets. “Co” and “Sin” in the “Type” column represent the corresponding methods are Co-SOD models and Single-SOD ones, respectively. “↑” (“↓”) means that larger (smaller) is better. The best, second best and third best results are highlighted in red, blue and bold, respectively. Results of different variants to our ICNet. NFs: 2-normalized features. CFM: correlation fusion module (§3.3). SISMs: extracting SIVs via SISMs (§3.2). SCFs: self-correlation features. R: rearrange (the channel order of SCFs, §3.4)

benchmark datasets: 3
Visualization of generated co-salient attention (CSA) maps. The 1-st and 2-nd rows are the input image groups and corresponding SISMs produced by [42], respectively. The 3-rd row shows the CSA maps yielded by our correlation fusion module (CFM). With the CSA maps, our ICNet obtains predictions (4-th row) that are more accurate than the used SISMs. Qualitative comparisons of different methods on three benchmark datasets.

Reference
  • Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk. Frequencytuned salient region detection. In CVPR, pages 1597–1604, 2009.
    Google ScholarLocate open access versionFindings
  • Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen. icoseg: Interactive co-segmentation with intelligent scribble guidance. In CVPR, pages 3169–3176, 2010.
    Google ScholarLocate open access versionFindings
  • Ali Borji, Ming-Ming Cheng, Huaizu Jiang, and Jia Li. Salient object detection: A benchmark. IEEE Transactions on Image Processing, 24(12):5706–5722, 2015.
    Google ScholarLocate open access versionFindings
  • Xiaochun Cao, Zhiqiang Tao, Bao Zhang, Huazhu Fu, and Wei Feng. Self-adaptively weighted co-saliency detection via rank constraint. IEEE Transactions on Image Processing, 23(9):4175– 4186, 2014.
    Google ScholarLocate open access versionFindings
  • Ming-Ming Cheng, Niloy J Mitra, Xiaolei Huang, Philip HS Torr, and Shi-Min Hu. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3):569–582, 2014.
    Google ScholarLocate open access versionFindings
  • Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji. Structure-measure: A new way to evaluate foreground maps. In ICCV, pages 4548–4557, 2017.
    Google ScholarLocate open access versionFindings
  • Deng-Ping Fan, Zheng Lin, Ge-Peng Ji, Dingwen Zhang, Huazhu Fu, and Ming-Ming Cheng. Taking a deeper look at co-salient object detection. In CVPR, pages 2919–2929, 2020.
    Google ScholarLocate open access versionFindings
  • Huazhu Fu, Xiaochun Cao, and Zhuowen Tu. Cluster-based co-saliency detection. IEEE Transactions on Image Processing, 22(10):3766–3778, 2013.
    Google ScholarLocate open access versionFindings
  • Kuang-Jui Hsu, Yen-Yu Lin, and Yung-Yu Chuang. Deepco3: Deep instance co-segmentation by co-peak search and co-saliency detection. In CVPR, pages 8846–8855, 2019.
    Google ScholarLocate open access versionFindings
  • Kuang-Jui Hsu, Chung-Chi Tsai, Yen-Yu Lin, Xiaoning Qian, and Yung-Yu Chuang. Unsupervised cnn-based co-saliency detection with graphical optimization. In ECCV, pages 485–501, 2018.
    Google ScholarLocate open access versionFindings
  • Bo Jiang, Xingyue Jiang, Ajian Zhou, Jin Tang, and Bin Luo. A unified multiple graph learning and convolutional network model for co-saliency estimation. In ACM MM, 2019.
    Google ScholarLocate open access versionFindings
  • Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
    Google ScholarLocate open access versionFindings
  • Bo Li, Zhengxing Sun, Lv Tang, Yunhan Sun, and Jinlong Shi. Detecting robust co-saliency with recurrent co-attention neural network. In IJCAI, pages 818–825, 2019.
    Google ScholarLocate open access versionFindings
  • Hongliang Li and King Ngi Ngan. A co-saliency model of image pairs. IEEE Transactions on Image Processing, 20(12):3365–3375, 2011.
    Google ScholarLocate open access versionFindings
  • Yijun Li, Keren Fu, Zhi Liu, and Jie Yang. Efficient saliency-model-guided visual co-saliency detection. IEEE Signal Processing Letters, 22(5):588–592, 2014.
    Google ScholarLocate open access versionFindings
  • Huaijia Lin, Xiaojuan Qi, and Jiaya Jia. Agss-vos: Attention guided single-shot video object segmentation. In ICCV, pages 3949–3957, 2019.
    Google ScholarLocate open access versionFindings
  • Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755, 2014.
    Google ScholarLocate open access versionFindings
  • Jiang-Jiang Liu, Qibin Hou, Ming-Ming Cheng, Jiashi Feng, and Jianmin Jiang. A simple pooling-based design for real-time salient object detection. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Zhi Liu, Wenbin Zou, Lina Li, Liquan Shen, and Olivier Le Meur. Co-saliency detection based on hierarchical segmentation. IEEE Signal Processing Letters, 21(1):88–92, 2013.
    Google ScholarLocate open access versionFindings
  • Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431–3440, 2015.
    Google ScholarLocate open access versionFindings
  • Alex Papushoy and Adrian G Bors. Image retrieval based on query by saliency content. Digital Signal Processing, 36:156–173, 2015.
    Google ScholarLocate open access versionFindings
  • Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8026–8037. 2019.
    Google ScholarLocate open access versionFindings
  • Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand. Basnet: Boundary-aware salient object detection. In CVPR, pages 7479–7489, 2019.
    Google ScholarLocate open access versionFindings
  • Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234–241.
    Google ScholarLocate open access versionFindings
  • Mennatullah Siam, Boris N Oreshkin, and Martin Jagersand. Amp: Adaptive masked proxies for few-shot segmentation. In ICCV, pages 5249–5258, 2019.
    Google ScholarLocate open access versionFindings
  • Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
    Google ScholarLocate open access versionFindings
  • Kevin Tang, Armand Joulin, Li-Jia Li, and Li Fei-Fei. Co-localization in real-world images. In CVPR, pages 1464–1471, 2014.
    Google ScholarLocate open access versionFindings
  • Chung-Chi Tsai, Kuang-Jui Hsu, Yen-Yu Lin, Xiaoning Qian, and Yung-Yu Chuang. Deep co-saliency detection via stacked autoencoder-enabled fusion and self-trained cnns. IEEE Transactions on Multimedia, 22(4):1016–1031, 2019.
    Google ScholarLocate open access versionFindings
  • Chong Wang, Zheng-Jun Zha, Dong Liu, and Hongtao Xie. Robust deep co-saliency detection with group semantic. In AAAI, volume 33, pages 8917–8924, 2019.
    Google ScholarLocate open access versionFindings
  • Ziqin Wang, Jun Xu, Li Liu, Fan Zhu, and Ling Shao. Ranet: Ranking attention network for fast video object segmentation. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Jun Wei, Shuhui Wang, and Qingming Huang. F3net: Fusion, feedback and focus for salient object detection. In AAAI, 2020.
    Google ScholarLocate open access versionFindings
  • Lina Wei, Shanshan Zhao, Omar El Farouk Bourahla, Xi Li, and Fei Wu. Group-wise deep co-saliency detection. In IJCAI, pages 3041–3047, 2017.
    Google ScholarLocate open access versionFindings
  • John Winn, Antonio Criminisi, and Thomas Minka. Object categorization by learned universal visual dictionary. In ICCV, volume 2, pages 1800–1807, 2005.
    Google ScholarLocate open access versionFindings
  • Zhe Wu, Li Su, and Qingming Huang. Cascaded partial decoder for fast and accurate salient object detection. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Zhe Wu, Li Su, and Qingming Huang. Stacked cross refinement network for edge-aware salient object detection. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Dingwen Zhang, Huazhu Fu, Junwei Han, Ali Borji, and Xuelong Li. A review of co-saliency detection algorithms: Fundamentals, applications, and challenges. ACM Transactions on Intelligent Systems and Technology, 9(4):1–31, 2018.
    Google ScholarLocate open access versionFindings
  • Dingwen Zhang, Junwei Han, Chao Li, Jingdong Wang, and Xuelong Li. Detection of co-salient objects by looking deep and wide. International Journal of Computer Vision, 120(2):215–232, 2016.
    Google ScholarLocate open access versionFindings
  • Dingwen Zhang, Deyu Meng, Chao Li, Lu Jiang, Qian Zhao, and Junwei Han. A self-paced multiple-instance learning framework for co-saliency detection. In ICCV, pages 594–602, 2015.
    Google ScholarLocate open access versionFindings
  • Kaihua Zhang, Tengpeng Li, Bo Liu, and Qingshan Liu. Co-saliency detection via mask-guided fully convolutional networks with multi-scale label smoothing. In CVPR, 2019.
    Google ScholarLocate open access versionFindings
  • Kaihua Zhang, Tengpeng Li, Shiwen Shen, Bo Liu, Jin Chen, and Qingshan Liu. Adaptive graph convolutional network with attention graph clustering for co-saliency detection. In CVPR, pages 9050–9059, 2020.
    Google ScholarLocate open access versionFindings
  • Zhao Zhang, Wenda Jin, Jun Xu, and Ming-Ming Cheng. Gradient-induced co-saliency detection. In ECCV, 2020.
    Google ScholarLocate open access versionFindings
  • Jia-Xing Zhao, Jiang-Jiang Liu, Deng-Ping Fan, Yang Cao, Jufeng Yang, and Ming-Ming Cheng. Egnet: Edge guidance network for salient object detection. In ICCV, 2019.
    Google ScholarLocate open access versionFindings
  • Xiaoju Zheng, Zheng-Jun Zha, and Liansheng Zhuang. A feature-adaptive semi-supervised framework for co-saliency detection. In ACM MM, pages 959–966, 2018.
    Google ScholarLocate open access versionFindings
  • Wenbin Zou, Kidiyo Kpalma, Zhi Liu, and Joseph Ronsin. Segmentation driven low-rank matrix recovery for saliency detection. In BMVC, pages 1–13, 2013.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科