AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We have proposed a deep network for sketch recognition by transferring knowledge from a cousin network trained on natural images

Cousin Network Guided Sketch Recognition via Latent Attribute Warehouse

THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF AR..., pp.9203-9210, (2019)

Cited by: 4|Views49
EI
Full Text
Bibtex
Weibo

Abstract

We study the problem of sketch image recognition. This problem is plagued with two major challenges: 1) sketch images are often scarce in contrast to the abundance of natural images, rendering the training task difficult, and 2) the significant domain gap between sketch image and its natural image counterpart makes the task of bridging th...More

Code:

Data:

0
Introduction
  • Automatic hand-drawn sketch recognition is an important task in computer vision, and has found many real-world applications.
  • Sketch can be used as a convenient user query input for content based image retrieval.
  • This becomes a popular user interface thanks to the wide use of digital pen.
  • (2) The amount of available sketch data is very scarce, and can be expensive to obtain.
  • It is unable to meet the requirement of training data needed by most deep neural networks
Highlights
  • Automatic hand-drawn sketch recognition is an important task in computer vision, and has found many real-world applications
  • It is unable to meet the requirement of training data needed by most deep neural networks
  • We propose to exploit models trained solely on natural images (ImageNet) to guide the model aimed for sketch classification
  • In order to cope with the loss of discriminative power due to the absence of visual attributes in sketch images, we develop a new mechanism called the “attribute warehouse”
  • We have proposed a deep network for sketch recognition by transferring knowledge from a cousin network trained on natural images
Methods
  • TU-Berlin Dataset
  • This dataset is proposed in (Eitz, Hays, and Alexa 2012) for sketch recognition, which contains 250 classes of objects.
  • TU-Berlin-extend Dataset.
  • The TU-Berlin dataset is extended by adding real images (Zhang et al 2016).
  • Following (Eitz, Hays, and Alexa 2012), the authors set up 8 kinds of training-testing protocols.
  • A number of t sketch instances in each category is used for training and the rest sketches in the category are used for testing.
Conclusion
  • The authors have proposed a deep network for sketch recognition by transferring knowledge from a cousin network trained on natural images.
  • The cousin network guides the target network to extract powerful features for recognition in an adversarial manner.
  • A latent attribute warehouse is developed to improve the transfer ability from sketch to natural image domain, boosting significantly the performance.
  • Experiment results demonstrate the effectiveness of the proposed model
Summary
  • Introduction:

    Automatic hand-drawn sketch recognition is an important task in computer vision, and has found many real-world applications.
  • Sketch can be used as a convenient user query input for content based image retrieval.
  • This becomes a popular user interface thanks to the wide use of digital pen.
  • (2) The amount of available sketch data is very scarce, and can be expensive to obtain.
  • It is unable to meet the requirement of training data needed by most deep neural networks
  • Methods:

    TU-Berlin Dataset
  • This dataset is proposed in (Eitz, Hays, and Alexa 2012) for sketch recognition, which contains 250 classes of objects.
  • TU-Berlin-extend Dataset.
  • The TU-Berlin dataset is extended by adding real images (Zhang et al 2016).
  • Following (Eitz, Hays, and Alexa 2012), the authors set up 8 kinds of training-testing protocols.
  • A number of t sketch instances in each category is used for training and the rest sketches in the category are used for testing.
  • Conclusion:

    The authors have proposed a deep network for sketch recognition by transferring knowledge from a cousin network trained on natural images.
  • The cousin network guides the target network to extract powerful features for recognition in an adversarial manner.
  • A latent attribute warehouse is developed to improve the transfer ability from sketch to natural image domain, boosting significantly the performance.
  • Experiment results demonstrate the effectiveness of the proposed model
Tables
  • Table1: The rank-1 accuracies [%] of different models on the TU-Berlin sketch dataset. The best results are shown in bold, which also applies in the following tables
  • Table2: The rank-1, rank-2, rank-3, rank-4, and rank-5 accuracies [%] of different models on TU-Berlin
Download tables as Excel
Related work
  • Sketch recognition/retrieval (Eitz et al 2011) has been studied for years. Roughly, existing approaches can be classified by whether it employs deep learning or not. Classical (nondeep-learning) sketch recognition methods often use handcrafted features. For example, local features along with the bag-of-words model (Sivic and Zisserman 2003) are used to represent sketch images and both SVM and the nearestneighbor classification are applied (Eitz, Hays, and Alexa 2012). SVM is a popular choice of classifier (Li, Song, and Gong 2013). Schneider et al (Schneider and Tuytelaars 2014) use Fisher vector to encode sketch images for recognition. Li et al (Li et al 2015) use multi-kernel learning to learn useful feature representation from various local features. To alleviate annotation cost in sketch recognition, active learning is introduced in (Yanık and Sezgin 2015). Cao et al (Cao et al 2013) propose a novel descriptor named
Funding
  • Kaihao Zhang’s PhD scholarship is funded by Australian National University
  • Hongdong Li is CI (Chief Inivestigator) on Australia Centre of Excellence for Robotic Vision (CE14) funded by Australia Research Council
  • This work is also supported by 2017 Tencent Rhino-Bird Elite Graduate Program
Reference
  • Cao, X.; Zhang, H.; Liu, S.; Guo, X.; and Lin, L. 2013. Sym-fish: A symmetry-aware flip invariant sketch histogram shape descriptor. In ICCV.
    Google ScholarFindings
  • Dai, G.; Xie, J.; Zhu, F.; and Fang, Y. 2017. Deep correlated metric learning for sketch-based 3d shape retrieval. In AAAI.
    Google ScholarFindings
  • Eitz, M.; Hildebrand, K.; Boubekeur, T.; and Alexa, M. 2011. Sketch-based image retrieval: Benchmark and bagof-features descriptors. TVCG.
    Google ScholarLocate open access versionFindings
  • Eitz, M.; Hays, J.; and Alexa, M. 2012. How do humans sketch objects? TOG.
    Google ScholarLocate open access versionFindings
  • Girshick, R. 201Fast R-CNN. In ICCV.
    Google ScholarFindings
  • Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2014. Generative adversarial nets. In NIPS.
    Google ScholarFindings
  • Hariharan, B.; Arbelaez, P.; Girshick, R.; and Malik, J. 2015. Hypercolumns for object segmentation and fine-grained localization. TPAMI.
    Google ScholarLocate open access versionFindings
  • He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR.
    Google ScholarFindings
  • Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS.
    Google ScholarFindings
  • Ledig, C.; Theis, L.; Huszar, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A. P.; Tejani, A.; Totz, J.; Wang, Z.; et al. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR.
    Google ScholarFindings
  • Li, Y.; Hospedales, T. M.; Song, Y.-Z.; and Gong, S. 2015. Free-hand sketch recognition by multi-kernel feature learning. CVIU.
    Google ScholarLocate open access versionFindings
  • Li, Y.; Fang, C.; Yang, J.; Wang, Z.; Lu, X.; and Yang, M.H. 2017. Universal style transfer via feature transforms. In NIPS.
    Google ScholarFindings
  • Li, Y.; Song, Y.-Z.; and Gong, S. 20Sketch recognition by ensemble matching of structured features. In BMVC.
    Google ScholarFindings
  • Lin, M.; Chen, Q.; and Yan, S. 2013. Network in network. In ICLR.
    Google ScholarFindings
  • Liu, L.; Shen, F.; Shen, Y.; Liu, X.; and Shao, L. 2017. Deep sketch hashing: Fast free-hand sketch-based image retrieval. In CVPR.
    Google ScholarFindings
  • Luo, W.; Sun, P.; Zhong, F.; Liu, W.; Zhang, T.; and Wang, Y. 2018. End-to-end active object tracking via reinforcement learning. In ICML.
    Google ScholarFindings
  • Ouyang, S.; Hospedales, T. M.; Song, Y.-Z.; and Li, X. 2016. Forgetmenot: Memory-aware forensic facial sketch matching. In CVPR.
    Google ScholarFindings
  • Ren, S.; He, K.; Girshick, R.; and Sun, J. 2015. Faster rcnn: Towards real-time object detection with region proposal networks. In NIPS.
    Google ScholarFindings
  • Schneider, R. G., and Tuytelaars, T. 2014. Sketch classification and classification-driven analysis using fisher vectors. TOG.
    Google ScholarFindings
  • Simonyan, K., and Zisserman, A. 2014. Two-stream convolutional networks for action recognition in videos. In NIPS.
    Google ScholarFindings
  • Simonyan, K., and Zisserman, A. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR.
    Google ScholarFindings
  • Sivic, J., and Zisserman, A. 2003. Video google: A text retrieval approach to object matching in videos. In ICCV.
    Google ScholarFindings
  • Sun, Y.; Wang, X.; and Tang, X. 2013. Deep convolutional network cascade for facial point detection. In CVPR.
    Google ScholarFindings
  • Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A.; et al. 2015. Going deeper with convolutions. In CVPR.
    Google ScholarFindings
  • Wang, F.; Kang, L.; and Li, Y. 2015. Sketch-based 3d shape retrieval using convolutional neural networks. In CVPR.
    Google ScholarFindings
  • Xie, J.; Dai, G.; Zhu, F.; and Fang, Y. 2017. Learning barycentric representations of 3d shapes for sketch-based 3d shape retrieval. In CVPR.
    Google ScholarFindings
  • Xiong, W.; Luo, W.; Ma, L.; Liu, W.; and Luo, J. 2018. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks. In CVPR.
    Google ScholarFindings
  • Yanık, E., and Sezgin, T. M. 2015. Active learning for sketch recognition. Computers & Graphics.
    Google ScholarFindings
  • Yu, Q.; Yang, Y.; Song, Y.-Z.; Xiang, T.; and Hospedales, T. 2015. Sketch-a-net that beats humans. In BMVC.
    Google ScholarFindings
  • Yu, Q.; Liu, F.; Song, Y.-Z.; Xiang, T.; Hospedales, T. M.; and Loy, C.-C. 2016. Sketch me that shoe. In CVPR.
    Google ScholarLocate open access versionFindings
  • Yu, Q.; Yang, Y.; Liu, F.; Song, Y.-Z.; Xiang, T.; and Hospedales, T. M. 2017. Sketch-a-net: A deep neural network that beats humans. IJCV.
    Google ScholarLocate open access versionFindings
  • Zhang, K.; Huang, Y.; Song, C.; Wu, H.; and Wang, L. 2015. Kinship verification with deep convolutional neural networks. In BMVC.
    Google ScholarFindings
  • Zhang, H.; Liu, S.; Zhang, C.; Ren, W.; Wang, R.; and Cao, X. 2016. Sketchnet: Sketch classification with web images. In CVPR.
    Google ScholarFindings
  • Zhang, K.; Huang, Y.; Du, Y.; and Wang, L. 2017. Facial expression recognition based on deep evolutional spatialtemporal networks. TIP.
    Google ScholarLocate open access versionFindings
  • Zhang, K.; Luo, W.; Zhong, Y.; Ma, L.; Liu, W.; and Li, H. 2019. Adversarial spatio-temporal learning for video deblurring. TIP.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科