Generative-Discriminative Feature Representations for Open-Set Recognition

CVPR, pp. 11811-11820, 2020.

Cited by: 2|Bibtex|Views15|Links
EI
Keywords:
Generative Adversarial Networkset samplecorrect classfeature representationgenerative modelMore(11+)
Weibo:
We explore the detection of open-set samples more effectively by learning richer feature representations than are usually needed for closed-set classification

Abstract:

We address the problem of open-set recognition, where the goal is to determine if a given sample belongs to one of the classes used for training a model (known classes). The main challenge in open-set recognition is to disentangle open-set samples that produce high class activations from known-set samples. We propose two techniques to for...More

Code:

Data:

0
Introduction
  • Supervised classification systems are trained with the knowledge of a finite set of labeled training examples.
  • Openset samples can be either discarded to prevent wrong association or used to improve the classification system [2], [23].
  • CNNs are trained with the objective of maximizing the probability of the correct class over the training data.
  • If the training process generalizes well enough, query samples from known classes can be expected to produce high probabilities.
  • The open-set recognition literature [26] points out the possibility of novel object samples producing high probabilities
Highlights
  • Supervised classification systems are trained with the knowledge of a finite set of labeled training examples
  • We evaluate the performance of the proposed method on standard datasets used for open-set recognition and compare with state-of-the art methods
  • We evaluate the performance of the proposed method in Out-of-distributional detection (OOD) [12] on CIFAR10 dataset
  • We explore the detection of open-set samples more effectively by learning richer feature representations than are usually needed for closed-set classification
  • We evaluated the proposed method in open-set detection and out-of-distributional image detection experiments where we produced state-of-the-art results
  • We carried out a study investigating the importance of each component of the proposed method
Methods
  • The authors motivate the need for a richer feature representation for effective open-set recognition.
  • An illustration of why open-set recognition is challenging is shown in Figure 1.
  • When a classifier is trained, the positive half spaces of each class are identified.
  • When a sample appears deeper in the identified positive half space, it will generate a larger class activation.
  • SoftMax [12] 63.9 OpenMax [3] 66.0.
  • Classifier is learned on augmented image space with self-supervision (Figure 2(c))
Results
  • The authors evaluate the performance of the proposed method on standard datasets used for open-set recognition and compare with state-of-the art methods.
  • The authors consider a case study on the CIFAR10 dataset to analyze performance of the proposed method qualitatively.
  • Recent deep learning based open-set recognition methods followed the protocol in [16] and used the numbers reported in [16] as a baseline for comparison.
  • The remaining classes are considered to be openset classes
  • This protocol is used to simulate five trials of open-set recognition and performance is measured using the average area under the curve of ROC (AUC-ROC) curve
Conclusion
  • The authors explore the detection of open-set samples more effectively by learning richer feature representations than are usually needed for closed-set classification.
  • The authors used selfsupervision and augmented the input image with a representation obtained from a generative model to enhance network’s ability to reject open-set samples.
  • These improvements forced the classifier to look beyond what is required to perform closed-set classification when producing decision regions.
  • The authors evaluated the proposed method in open-set detection and out-of-distributional image detection experiments where the authors produced state-of-the-art results.
  • The authors hope to investigate how this algorithm can be extended to other computer vision tasks such as object detection and semantic segmentation
Summary
  • Introduction:

    Supervised classification systems are trained with the knowledge of a finite set of labeled training examples.
  • Openset samples can be either discarded to prevent wrong association or used to improve the classification system [2], [23].
  • CNNs are trained with the objective of maximizing the probability of the correct class over the training data.
  • If the training process generalizes well enough, query samples from known classes can be expected to produce high probabilities.
  • The open-set recognition literature [26] points out the possibility of novel object samples producing high probabilities
  • Methods:

    The authors motivate the need for a richer feature representation for effective open-set recognition.
  • An illustration of why open-set recognition is challenging is shown in Figure 1.
  • When a classifier is trained, the positive half spaces of each class are identified.
  • When a sample appears deeper in the identified positive half space, it will generate a larger class activation.
  • SoftMax [12] 63.9 OpenMax [3] 66.0.
  • Classifier is learned on augmented image space with self-supervision (Figure 2(c))
  • Results:

    The authors evaluate the performance of the proposed method on standard datasets used for open-set recognition and compare with state-of-the art methods.
  • The authors consider a case study on the CIFAR10 dataset to analyze performance of the proposed method qualitatively.
  • Recent deep learning based open-set recognition methods followed the protocol in [16] and used the numbers reported in [16] as a baseline for comparison.
  • The remaining classes are considered to be openset classes
  • This protocol is used to simulate five trials of open-set recognition and performance is measured using the average area under the curve of ROC (AUC-ROC) curve
  • Conclusion:

    The authors explore the detection of open-set samples more effectively by learning richer feature representations than are usually needed for closed-set classification.
  • The authors used selfsupervision and augmented the input image with a representation obtained from a generative model to enhance network’s ability to reject open-set samples.
  • These improvements forced the classifier to look beyond what is required to perform closed-set classification when producing decision regions.
  • The authors evaluated the proposed method in open-set detection and out-of-distributional image detection experiments where the authors produced state-of-the-art results.
  • The authors hope to investigate how this algorithm can be extended to other computer vision tasks such as object detection and semantic segmentation
Tables
  • Table1: Impact of using different architectures on open-set recognition on the CIFAR10 dataset. We observe that using more sophisticated generative models and classifiers both improve openset performance
  • Table2: Open-set detection performance in terms of AUC-ROC curve. N.R. is used when the original work did not report a particular figure
  • Table3: Closed-set accuracy for the proposed method
  • Table4: Performance of out-of-distributional object detection for CIFAR10 dataset with VGG13 network. Performance is measured using macro-F1 measure
  • Table5: Tabulation of classification performance (accuracy) and open-set rejection performance(AUC) for the ablation study
Download tables as Excel
Related work
  • Open-set Recognition. Open-set recognition has received considerable attention in the computer vision community in recent years. The problem of open-set recognition was first formulated in [26], where authors pointed out the possibility of an open-set sample generating a very high activation score for one of the known class categories. Since then, several other works have analyzed this challenge in the context of deep networks [22],[11]. In [3], a k + 1 classifier for a k class problem was used where the extra class was treated as the open-set class. A statistical method was used to apportion class probabilities to the open-set class. This alternative formulation, OpenMax, was proposed as an alternative to the SoftMax operator. In [7], a Generative Adversarial Network (GAN) based framework was used to estimate open-set class activations. A similar approach was taken in [16] where counterfactual images that lie between decision boundaries were used to simulate open-set class instances.
Reference
  • Martin Arjovsky, Soumith Chintala, and Leon Bottou. Wasserstein generative adversarial networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 214–223, International Convention Centre, Sydney, Australia, 06–11 Aug 2017. PMLR. 4
    Google ScholarLocate open access versionFindings
  • Abhijit Bendale and Terrance Boult. Towards open world recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 1
    Google ScholarLocate open access versionFindings
  • Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 2, 7, 8
    Google ScholarLocate open access versionFindings
  • J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. FeiFei. ImageNet: A Large-Scale Hierarchical Image Database. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 6
    Google ScholarLocate open access versionFindings
  • Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV ’15, pages 1422–1430, 2012
    Google ScholarLocate open access versionFindings
  • Carl Doersch and Andrew Zisserman. Multi-task selfsupervised visual learning. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017. 3
    Google ScholarLocate open access versionFindings
  • Zongyuan Ge, Sergey Demyanov, and Rahil Garnavi. Generative openmax for multi-class open set classification. In British Machine Vision Conference 2017, BMVC 2017, London, UK, September 4-7, 2017, 2012, 7
    Google ScholarLocate open access versionFindings
  • Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ArXiv, abs/1803.07728, 2013
    Findings
  • Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018. 3
    Google ScholarLocate open access versionFindings
  • Izhak Golan and Ran El-Yaniv. Deep anomaly detection using geometric transformations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 9758–9769. Curran Associates, Inc., 2018. 3
    Google ScholarLocate open access versionFindings
  • Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 2
    Google ScholarLocate open access versionFindings
  • Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017. 1, 6, 8
    Google ScholarLocate open access versionFindings
  • Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). 5
    Google ScholarFindings
  • Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar100 (canadian institute for advanced research). 6
    Google ScholarFindings
  • Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. International Conference on Learning Representations (ICLR), 2018. 6
    Google ScholarLocate open access versionFindings
  • Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. Open set learning with counterfactual images. In The European Conference on Computer Vision (ECCV), September 2018. 2, 4, 5, 6, 7, 8
    Google ScholarLocate open access versionFindings
  • Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011. 5
    Google ScholarLocate open access versionFindings
  • P. Oza and V. M. Patel. Active authentication using an autoencoder regularized cnn-based one-class classifier. In 2019 14th IEEE International Conference on Automatic Face Gesture Recognition (FG 2019), pages 1–8, 2019.
    Google ScholarLocate open access versionFindings
  • Poojan Oza and Vishal M. Patel. C2ae: Class conditioned auto-encoder for open-set recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 202, 5, 7
    Google ScholarLocate open access versionFindings
  • Poojan Oza and Vishal M Patel. One-class convolutional neural network. IEEE Signal Processing Letters, 26(2):277– 281, 2019. 2
    Google ScholarLocate open access versionFindings
  • Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. Ocgan: One-class novelty detection using gans with constrained latent representations. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 2
    Google ScholarLocate open access versionFindings
  • Pramuditha Perera and Vishal M. Patel. Deep transfer learning for multiple class novelty detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 2
    Google ScholarLocate open access versionFindings
  • Pramuditha Perera and Vishal M. Patel. Face-based multiple user active authentication on mobile devices. IEEE Transactions on Information Forensics and Security, 14(5):1240– 1250, 2019. 1
    Google ScholarLocate open access versionFindings
  • Pramuditha Perera and Vishal M. Patel. Learning deep features for one-class classification. IEEE Transactions on Image Processing, 28(11):5450–5463, 2019. 2
    Google ScholarLocate open access versionFindings
  • Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi-supervised learning with ladder networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 3546–3554. Curran Associates, Inc., 2015. 2
    Google ScholarLocate open access versionFindings
  • Walter J. Scheirer, Anderson Rocha, Archana Sapkota, and Terrance E. Boult. Towards open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 35, July 2013. 2, 6
    Google ScholarLocate open access versionFindings
  • Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579–2605, 2008. 6
    Google ScholarLocate open access versionFindings
  • Yan Xia, Xudong Cao, Fang Wen, Gang Hua, and Jian Sun. Learning discriminative reconstructions for unsupervised outlier removal. In The IEEE International Conference on Computer Vision (ICCV), December 2015. 4
    Google ScholarLocate open access versionFindings
  • Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto Iida, and Takeshi Naemura. Classificationreconstruction learning for open-set recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 2, 5, 6, 7, 8
    Google ScholarLocate open access versionFindings
  • Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 6
    Findings
  • Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016. 4, 6
    Google ScholarLocate open access versionFindings
  • H. Zhang and V. M. Patel. Sparse representation-based open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(8):1690–1696, 2017.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments