AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We propose a novel Deep Asymmetric Transfer Network to perform unbalanced domain adaptation

Deep Asymmetric Transfer Network for Unbalanced Domain Adaptation

THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTI..., pp.443-450, (2018)

被引用14|浏览62
EI
下载 PDF 全文
引用
微博一下

摘要

Recently, domain adaptation based on deep models has been a promising way to deal with the domains with scarce labeled data, which is a critical problem for deep learning models. Domain adaptation propagates the knowledge from a source domain with rich information to the target domain. In reality, the source and target domains are mostly ...更多

代码

数据

简介
重点内容
  • Nowadays, deep learning models have been successfully applied to many applications (Krizhevsky and Hinton 2011; Mikolov et al 2010; Wang, Cui, and Zhu 2016)
  • To address the above challenges, we propose a novel Deep Asymmetric Transfer Network (DATN), to perform unbalanced domain adaptation
  • We propose a novel Deep Asymmetric Transfer Network (DATN) to perform unbalanced domain adaptation, whose framework is shown in Figure 1
  • Comparing Figure 6(a) and Figure 6(b), we find that α has a greater influence on the performance than β, which implies that the proposed supervised asymmetric transfer model, especially the classifier adaptation method is more important than the unsupervised transfer, which is consistent with the conclusion we just got in the previous experiment
  • We find that the proposed supervised asymmetric transfer model, especially the classifier adaptation method, has larger effect than the unsupervised transfer on the classification accuracy
方法
  • Even if in this case, the result that DATN can further improve the performance demonstrates the superiority of the method
结果
  • The authors first report the overall image classification accuracy and the accuracy over each category on NUS-WIDE using SIFT and VGG16 features.
  • Discussions about Asymmetric Transfer One assumption behind the unbalanced domain adaptation is that source domain data have richer and more reliable knowledge than those of target domain.
  • One natural question is that how the quality of the source domain data affects the transfer performance on the target domain?.
  • The authors compare the performance of DATN sup with the performance of WSDTN, CDLS and DNN.
  • WSDTN and CDLS are symmetric-transfer based methods.
  • DNN is non-transfer method.
结论
  • The authors use deep autoencoder as the basic block to achieve the transfer. There are other kinds of deep models, such as the CNN (Krizhevsky, Sutskever, and Hinton 2012) and LSTM (Mikolov et al 2010).
  • Since the main focus is to introduce the asymmetric transfer model to do unbalanced domain adaptation, the authors will omit the discussion about different deep architectures.
  • The authors' transfer method is scalable for real applications.In this paper, the authors propose a novel Deep Asymmetric Transfer Network (DATN) to perform unbalanced domain adaptation.
  • The authors find that the proposed supervised asymmetric transfer model, especially the classifier adaptation method, has larger effect than the unsupervised transfer on the classification accuracy.
  • The future directions may focus on transferring the knowledge from more complex data, such as the natural languages, heterogeneous networks and so on
表格
  • Table1: Terms and Notations
  • Table2: The statistics of the datasets
  • Table3: Number of neurons of each layer of DATN
  • Table4: Classification accuracy on AMAZON REVIEWS
  • Table5: Transfer Performance for DATN, DATN sup and
Download tables as Excel
相关工作
  • Domain adaptation, also known as transfer learning (Pan and Yang 2010) aims at propagating the knowledge in the source domain to the target domain. Most of existing methods (Oquab et al 2014; Glorot, Bordes, and Bengio 2011; Long and Wang 2015; Yosinski et al 2014) focus on homogeneous domain adaptation, which assumes that data of the source and target domains lie in the same domain, such as the images in NUS-WIDE and ImageNet. For this branch, methods often use a share-parameter model for the two domains. Some other works work on heterogeneous domain adaptation, which assumes that the data of source and target domains lie in different domains and different feature spaces. For both homogeneous and heterogeneous domain adaptation, their key bottleneck is to alleviate the domain discrepancy to perform knowledge transfer. Most of existing methods (Zhu et al 2011; Qi, Aggarwal, and Huang 2011; Shi et al 2009; Dai et al 2009) adopt shallow models attempting to explicitly reduce the discrepancy. However, the transferability of shallow models will be greatly limited due to the task-specific variability (Long and Wang 2015), thereby these models cannot achieve satisfied performance.
基金
  • This work was supported by National Program on Key Basic Research Project, No 2015CB352300; National Natural Science Foundation of China Major Project No U1611461; National Natural Science Foundation of China, No 61772304, No 61521002, No 61531006
引用论文
  • Blei, D. M.; Ng, A. Y.; and Jordan, M. I. 2003. Latent dirichlet allocation. Journal of machine Learning research 3(Jan):993–1022.
    Google ScholarLocate open access versionFindings
  • Chang, C.-C., and Lin, C.-J. 2011. Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST) 2(3):27.
    Google ScholarLocate open access versionFindings
  • Chua, T.-S.; Tang, J.; Hong, R.; Li, H.; Luo, Z.; and Zheng, Y. 2009. Nus-wide: a real-world web image database from national university of singapore. In Proceedings of the ACM international conference on image and video retrieval.
    Google ScholarLocate open access versionFindings
  • Dai, W.; Chen, Y.; Xue, G.-R.; Yang, Q.; and Yu, Y. 2009. Translated learning: Transfer learning across different feature spaces. In Advances in neural information processing systems, 353–360.
    Google ScholarLocate open access versionFindings
  • Glorot, X.; Bordes, A.; and Bengio, Y. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 513–520.
    Google ScholarLocate open access versionFindings
  • Hearst, M. A.; Dumais, S. T.; Osman, E.; Platt, J.; and Scholkopf, B. 1998. Support vector machines. IEEE Intelligent Systems and their Applications 13(4):18–28.
    Google ScholarLocate open access versionFindings
  • Hubert Tsai, Y.-H.; Yeh, Y.-R.; and Frank Wang, Y.-C. 2016. Learning cross-domain landmarks for heterogeneous domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5081–5090.
    Google ScholarLocate open access versionFindings
  • Krizhevsky, A., and Hinton, G. E. 2011. Using very deep autoencoders for content-based image retrieval. In ESANN.
    Google ScholarFindings
  • Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems.
    Google ScholarFindings
  • Long, M., and Wang, J. 2015. Learning transferable features with deep adaptation networks. CoRR, abs/1502.02791 1:2.
    Findings
  • Long, M.; Wang, J.; Ding, G.; Pan, S. J.; and Philip, S. Y. 2014. Adaptation regularization: A general framework for transfer learning. IEEE Transactions on Knowledge and Data Engineering 26(5):1076–1089.
    Google ScholarLocate open access versionFindings
  • Lowe, D. G. Object recognition from local scale-invariant features. In Computer vision, 1999. The proceedings of the seventh IEEE international conference on.
    Google ScholarLocate open access versionFindings
  • Mikolov, T.; Karafiat, M.; Burget, L.; Cernocky, J.; and Khudanpur, S. 2010. Recurrent neural network based language model. In Interspeech, volume 2, 3.
    Google ScholarLocate open access versionFindings
  • Nair, V., and Hinton, G. E. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), 807–814.
    Google ScholarLocate open access versionFindings
  • Ngiam, J.; Khosla, A.; Kim, M.; Nam, J.; Lee, H.; and Ng, A. Y. 2011. Multimodal deep learning. In Proceedings of the 28th international conference on machine learning (ICML-11), 689–696.
    Google ScholarLocate open access versionFindings
  • Oquab, M.; Bottou, L.; Laptev, I.; and Sivic, J. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1717–1724.
    Google ScholarLocate open access versionFindings
  • Pan, S. J., and Yang, Q. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22(10):1345– 1359.
    Google ScholarLocate open access versionFindings
  • Prettenhofer, P., and Stein, B. 2010. Cross-language text classification using structural correspondence learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 1118–1127. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Qi, G.-J.; Aggarwal, C.; and Huang, T. 2011. Towards semantic knowledge propagation from text corpus to web images. In Proceedings of the 20th international conference on World wide web, 297–306. ACM.
    Google ScholarLocate open access versionFindings
  • Salakhutdinov, R., and Hinton, G. 2009. Semantic hashing. International Journal of Approximate Reasoning.
    Google ScholarLocate open access versionFindings
  • Sejdinovic, D.; Sriperumbudur, B.; Gretton, A.; Fukumizu, K.; et al. 2013. Equivalence of distance-based and rkhs-based statistics in hypothesis testing. The Annals of Statistics 41(5):2263–2291.
    Google ScholarLocate open access versionFindings
  • Shi, Y.; Lan, Z.; Liu, W.; and Bi, W. 2009. Extending semisupervised learning methods for inductive transfer learning. In 2009 Ninth IEEE International Conference on Data Mining, 483– 492. IEEE.
    Google ScholarLocate open access versionFindings
  • Shu, X.; Qi, G.-J.; Tang, J.; and Wang, J. 2015. Weakly-shared deep transfer networks for heterogeneous-domain knowledge propagation. In Proceedings of the 23rd ACM international conference on Multimedia, 35–44. ACM.
    Google ScholarLocate open access versionFindings
  • Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
    Findings
  • Sontag, D.; Globerson, A.; and Jaakkola, T. 2011. Introduction to dual decomposition for inference. Optimization for Machine Learning 1(219-254):1.
    Google ScholarLocate open access versionFindings
  • Srivastava, N., and Salakhutdinov, R. 2012. Learning representations for multimodal data with deep belief nets. In International conference on machine learning workshop.
    Google ScholarLocate open access versionFindings
  • Wang, D.; Cui, P.; Ou, M.; and Zhu, W. 2015a. Deep multimodal hashing with orthogonal regularization. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Wang, D.; Cui, P.; Ou, M.; and Zhu, W. 2015b. Learning compact hash codes for multimodal representations using orthogonal deep structure. IEEE Transactions on Multimedia 17(9):1404–1416.
    Google ScholarLocate open access versionFindings
  • Wang, D.; Cui, P.; and Zhu, W. 2016. Structural deep network embedding. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
    Google ScholarLocate open access versionFindings
  • Yosinski, J.; Clune, J.; Bengio, Y.; and Lipson, H. 2014. How transferable are features in deep neural networks? In Advances in neural information processing systems.
    Google ScholarLocate open access versionFindings
  • Zhang, H.; Zha, Z.-J.; Yang, Y.; Yan, S.; Gao, Y.; and Chua, T.-S. 2013. Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. In Proceedings of the 21st ACM international conference on Multimedia, 33–42. ACM.
    Google ScholarLocate open access versionFindings
  • Zhou, J. T.; Pan, S. J.; Tsang, I. W.; and Yan, Y. 2014. Hybrid heterogeneous transfer learning through deep learning. In AAAI, 2213–2220.
    Google ScholarFindings
  • Zhu, Y.; Chen, Y.; Lu, Z.; Pan, S. J.; Xue, G.-R.; Yu, Y.; and Yang, Q. 2011. Heterogeneous transfer learning for image classification. In AAAI.
    Google ScholarFindings
  • Zhu, X. 2005. Semi-supervised learning literature survey.
    Google ScholarFindings
  • Zhuang, F.; Cheng, X.; Luo, P.; Pan, S. J.; and He, Q. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Int. Joint Conf. Artif. Intell.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科