Few-Shot Open-Set Recognition Using Meta-Learning

CVPR, pp. 8795-8804, 2020.

Cited by: 0|Bibtex|Views139|DOI:https://doi.org/10.1109/CVPR42600.2020.00882
EI
Other Links: arxiv.org|dblp.uni-trier.de|academic.microsoft.com
Weibo:
We proposed an extension of meta-learning that includes an open-set loss, and a better metric learning design

Abstract:

The problem of open-set recognition is considered. While previous approaches only consider this problem in the context of large-scale classifier training, we seek a unified solution for this and the low-shot classification setting. It is argued that the classic softmax classifier is a poor solution for open-set recognition, since it ten...More

Code:

Data:

0
Introduction
  • The introduction of deep convolutional neural networks (CNNs) has catalyzed large advances in computer vision.
  • Most of these advances can be traced back to advances on object recognition, due to the introduction of large scale datasets, such as ImageNet [2], containing many classes and many examples per class.
Highlights
  • The introduction of deep convolutional neural networks (CNNs) has catalyzed large advances in computer vision
  • In the large scale setting, convolutional neural networks-based classifiers trained by cross-entropy loss and mini-batch SGD have excellent recognition performance, achieving state of the art results on most recognition benchmarks
  • We investigate the role of the metric used in the feature space on open-set recognition performance, showing that a particular form of the Mahalanobis distance enables significant gains over the commonly used Euclidean distance
  • We proposed an extension of meta-learning that includes an open-set loss, and a better metric learning design
Methods
  • The proposed method was compared to state-of-the-art (SOTA) approached to open-set recognition.
  • Following Neal et al [21], the authors evaluate both classification accuracy and open-set detection performance.
  • The authors use the testing samples from both the training and open-set classes.
  • Classification accuracy is used to measure how well the model classifies closed-set samples, i.e., test samples from closedset classes.
  • The AUROC (Area Under ROC Curve) metric is used to measure how well the model detects open-set samples, i.e., test samples from open-set classes, within all test samples.
  • The authors define the following acronyms: the basic represents prototypical networks with euclidean distance for open-set detection; GaussianE represents the Gaussian Embedding introduced in Sec. 4.2; OpLoss represents the proposed open-set loss
Results
  • The authors compare the proposed method to several openset recognition SOTA methods, including OpenMax [1], G-OpenMax [5], and Counterfactual [21], and an out-ofdistribution SOTA method, Confidence [14].
  • All models use the same CNN backbone for fair comparison.
  • The authors tested both a CNN, denoted as ConvNet, proposed by [21], and ResNet18 [8].
  • Since not all prior open-set methods support the few-shot setting, some modifications were required.
  • Generative methods [5, 21] do not support few-shot samples.
  • The authors train the model on the pre-training set and fine-tune it on the support set.
  • The authors apply OpenMax on the activations of the pre-softmax layer, i.e. the negative of the distance
Conclusion
  • The authors have revisited the open-set recognition problem in the context of few-shot learning.
  • The authors proposed an extension of meta-learning that includes an open-set loss, and a better metric learning design.
  • The resulting classifiers provide a new stat-of-the-art for few-shot open-set recognition, on mini-Imagenet.
  • With few modifications, the approach can be applied to large-scale recognition, where it outperforms state of the art methods for open-set recognition.
  • The XJTU-Stevens Dataset was used to demonstrante the effectiveness of the proposed model on a weakly supervised object discovery task
Summary
  • Introduction:

    The introduction of deep convolutional neural networks (CNNs) has catalyzed large advances in computer vision.
  • Most of these advances can be traced back to advances on object recognition, due to the introduction of large scale datasets, such as ImageNet [2], containing many classes and many examples per class.
  • Methods:

    The proposed method was compared to state-of-the-art (SOTA) approached to open-set recognition.
  • Following Neal et al [21], the authors evaluate both classification accuracy and open-set detection performance.
  • The authors use the testing samples from both the training and open-set classes.
  • Classification accuracy is used to measure how well the model classifies closed-set samples, i.e., test samples from closedset classes.
  • The AUROC (Area Under ROC Curve) metric is used to measure how well the model detects open-set samples, i.e., test samples from open-set classes, within all test samples.
  • The authors define the following acronyms: the basic represents prototypical networks with euclidean distance for open-set detection; GaussianE represents the Gaussian Embedding introduced in Sec. 4.2; OpLoss represents the proposed open-set loss
  • Results:

    The authors compare the proposed method to several openset recognition SOTA methods, including OpenMax [1], G-OpenMax [5], and Counterfactual [21], and an out-ofdistribution SOTA method, Confidence [14].
  • All models use the same CNN backbone for fair comparison.
  • The authors tested both a CNN, denoted as ConvNet, proposed by [21], and ResNet18 [8].
  • Since not all prior open-set methods support the few-shot setting, some modifications were required.
  • Generative methods [5, 21] do not support few-shot samples.
  • The authors train the model on the pre-training set and fine-tune it on the support set.
  • The authors apply OpenMax on the activations of the pre-softmax layer, i.e. the negative of the distance
  • Conclusion:

    The authors have revisited the open-set recognition problem in the context of few-shot learning.
  • The authors proposed an extension of meta-learning that includes an open-set loss, and a better metric learning design.
  • The resulting classifiers provide a new stat-of-the-art for few-shot open-set recognition, on mini-Imagenet.
  • With few modifications, the approach can be applied to large-scale recognition, where it outperforms state of the art methods for open-set recognition.
  • The XJTU-Stevens Dataset was used to demonstrante the effectiveness of the proposed model on a weakly supervised object discovery task
Tables
  • Table1: Comparison between different recognition tasks
  • Table2: Comparison to SOTAs on large-scale open-set recognition: PEELER outperforms all others in terms of open-set sample detection AUROC, for a comparable classification accuracy
  • Table3: Few-shot open-set recognition results
  • Table4: Table 4
  • Table5: The number of misclassified frames (lower is better) by varying the number of annotated frames (I, II, III in the second row)
Download tables as Excel
Related work
  • Open-Set Recognition: Open-set recognition addresses the classification setting where inference can face samples from classes unseen during training. The goal is to endow the open-set classifier with a mechanism to reject such samples. One of the first deep learning approaches was the work of Scheirer et al [1], which proposed an extreme value parameter redistribution method for the logits generated by the classifier. Later works considered the problem in either discriminative or generative models. Schlachter et al [28] proposed an intra-class splitting method, where a closedset classifier is used to split data into typical and atypical subsets, reformulating open-set recognition as a traditional classification problem. G-OpenMax [5] utilizes a generator trained to synthesize examples from an extra class that represents all unknown classes. Neal et al [21] introduced counterfactual image generation, which aims to generate samples that cannot be classified into any of the seen classes, producing an extra class for classifier training.
Funding
  • Gang Hua was supported partly by National Key R&D Program of China Grant 2018AAA0101400 and NSFC Grant 61629301
  • Bo Liu and Nuno Vasconcelos were partially supported by NSF awards IIS-1637941, IIS-1924937, and NVIDIA GPU donations
Reference
  • Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem:1563–1572, 2016. 1, 2, 6, 7
    Google ScholarLocate open access versionFindings
  • Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255.
    Google ScholarLocate open access versionFindings
  • Chelsea Finn, Pieter Abbeel, and Sergey Levine. ModelAgnostic Meta-Learning for Fast Adaptation of Deep Networks. 2017. 1, 3
    Google ScholarFindings
  • Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic Model-Agnostic Meta-Learning. (NeurIPS), 2018. 3
    Google ScholarFindings
  • Zongyuan Ge, Sergey Demyanov, and Rahil Garnavi. Generative OpenMax for Multi-Class Open Set Classification. 2019. 1, 2, 6, 7
    Google ScholarFindings
  • Spyros Gidaris and Nikos Komodakis. Dynamic Few-Shot Visual Learning Without Forgetting. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 4367–4375, 2018. 3, 6
    Google ScholarLocate open access versionFindings
  • Albert Gordo, Jon Almazan, Jerome Revaud, and Diane Larlus. Deep image retrieval: Learning global representations for image search. In European conference on computer vision, pages 241–25Springer, 2016. 2
    Google ScholarLocate open access versionFindings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6, 7
    Google ScholarLocate open access versionFindings
  • Dan Hendrycks and Kevin Gimpel. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. pages 1–12, 2016. 1, 3
    Google ScholarFindings
  • Chih-Hui Ho, Pedro Morgado, Amir Persekian, and Nuno Vasconcelos. Pies: Pose invariant embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12377–12386, 2019. 2
    Google ScholarLocate open access versionFindings
  • Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.02284, 2017. 1
    Findings
  • Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 6
    Findings
  • Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://www.cs.toronto.edu/kriz/cifar.html, 55, 2014.1, 6
    Locate open access versionFindings
  • Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting outof-distribution samples. arXiv preprint arXiv:1711.09325, 2017. 6
    Findings
  • Hongyang Li, David Eigen, Samuel Dodge, Matthew Zeiler, and Xiaogang Wang. Finding Task-Relevant Features for Few-Shot Learning by Category Traversal. 1, 2019. 3
    Google ScholarFindings
  • Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. (2017), 2017. 3
    Google ScholarFindings
  • Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212–220, 202
    Google ScholarLocate open access versionFindings
  • Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large-Scale Long-Tailed Recognition in an Open World. pages 2537–2546, 2019. 1
    Google ScholarFindings
  • Jiajun Lu, Theerasit Issaranon, and David Forsyth. Safetynet: Detecting and rejecting adversarial examples robustly. In Proceedings of the IEEE International Conference on Computer Vision, pages 446–454, 2017. 1
    Google ScholarLocate open access versionFindings
  • Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In Advances in neural information processing systems, pages 1196–1204, 2013. 1
    Google ScholarLocate open access versionFindings
  • Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng Keen Wong, and Fuxin Li. Open set learning with counterfactual images. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11210 LNCS(1):620–635, 2018. 2, 6, 7
    Google ScholarLocate open access versionFindings
  • Boris N Oreshkin, Pau Rodriguez, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved fewshot learning. In Advances in Neural Information Processing Systems, volume 2018-Decem, pages 721–731, 2018. 3
    Google ScholarLocate open access versionFindings
  • Hang Qi, Matthew Brown, and David G. Lowe. Low-Shot Learning with Imprinted Weights. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 5822–5830, 2018. 3
    Google ScholarLocate open access versionFindings
  • Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan Yuille. FewShot Image Recognition by Predicting Parameters from Activations. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 7229–7238, 2018. 3
    Google ScholarLocate open access versionFindings
  • Sachin Ravi and Hugo Larochelle. Optimization as a Model For Few-Shot Learning. pages 1–11, 2017. 1, 3, 7
    Google ScholarFindings
  • Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-Shot Learning with Embedded Class Models and ShotFree Meta Training. 2019. 3
    Google ScholarFindings
  • Mengye Ren, Renjie Liao, Ethan Fetaya, and Richard S. Zemel. Incremental Few-Shot Learning with Attention Attractor Networks. 2018. 3
    Google ScholarFindings
  • Patrick Schlachter, Yiwen Liao, and Bin Yang. Open-Set Recognition Using Intra-Class Splitting. 2019. 2
    Google ScholarFindings
  • Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical Networks for Few-shot Learning. 2017. 1, 3, 4, 5
    Google ScholarFindings
  • Sainbayar Sukhbaatar and Rob Fergus. Learning from noisy labels with deep neural networks. arXiv preprint arXiv:1406.2080, 2(3):4, 2014. 1
    Findings
  • Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales. Learning to Compare: Relation Network for Few-Shot Learning. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1199–1208, 2018. 3
    Google ScholarLocate open access versionFindings
  • Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7167–7176, 2017. 1
    Google ScholarLocate open access versionFindings
  • Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching Networks for One Shot Learning. 2016. 1, 2, 3, 7
    Google ScholarFindings
  • Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L. Willke. Out-ofdistribution detection using an ensemble of self supervised leave-out classifiers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11212 LNCS:560– 574, 2018. 3
    Google ScholarLocate open access versionFindings
  • Le Wang, Gang Hua, Rahul Sukthankar, Jianru Xue, Zhenxing Niu, and Nanning Zheng. Video object discovery and co-segmentation with extremely weak supervision. IEEE transactions on pattern analysis and machine intelligence, 39(10):2074–2088, 2016. 8
    Google ScholarLocate open access versionFindings
  • Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7278–7286, 2018. 2
    Google ScholarLocate open access versionFindings
  • Lin Dahua Zhirong Wu, Yuanjun Xiong, Stella X. Yu. Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination. (CVPR), 2018. 2
    Google ScholarFindings
  • Zhun Zhong, Liang Zheng, Donglin Cao, and Shaozi Li. Reranking person re-identification with k-reciprocal encoding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1318–1327, 2017. 2
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments