AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
Different from the traditional nonnegative matrix factorization models which only focus on the regular data points, our models emphasizes potential test adversaries that are beyond the pre-defined constraints

Adversarial Nonnegative Matrix Factorization

ICML, pp.6479-6488, (2020)

Cited by: 0|Views116
EI
Full Text
Bibtex
Weibo

Abstract

Nonnegative Matrix Factorization (NMF) has become an increasingly important research topic in machine learning. Despite all the practical success, most of existing NMF models are still vulnerable to adversarial attacks. To overcome this limitation, we propose a novel Adversarial NMF (ANMF) approach in which an adversary can exercise some ...More

Code:

Data:

0
Introduction
  • The nonnegative matrix factorization (NMF) has been a prevalent nonnegative dimensionality reduction method and successfully applied to many fields such as feature extraction (Zhi et al, 2011), video tracking (Bucak & Gunsel, 2007), image processing (Geng et al, 2012) and document clustering (Guan et al, 2012).
  • Given a data matrix Y ∈ m×N with non-negative entries, the goal of NMF is to factorize Y into the product AX of two nonnegative matrices, with n columns in A, where n is generally small.
  • Most of them independently deal with each element of X, regardless of the relationship between elements.
  • To address this issue, some scholars considered the structural information of data or variate in modeling.
  • Haeffele et al (Haeffele et al, 2014) explored a matrix factorization technique suitable for large datasets that captures additional structure in the factors by using a projective tensor norm
Highlights
  • The nonnegative matrix factorization (NMF) has been a prevalent nonnegative dimensionality reduction method and successfully applied to many fields such as feature extraction (Zhi et al, 2011), video tracking (Bucak & Gunsel, 2007), image processing (Geng et al, 2012) and document clustering (Guan et al, 2012)
  • Since nonnegative matrix factorization was popularized by Lee and Seung (Lee & Seung, 1999), various nonnegative matrix factorization methods have been proposed
  • How to increase the robustness of models against the general perturbations has become a very important task in nonnegative matrix factorization. To address this challenging problem, in this paper, we introduce a novel Adversarial Nonnegative Matrix Factorization (ANMF) model by emphasizing potential test adversaries that are beyond the pre-defined constraints
  • From learner and attacker perspectives, we propose a novel Adversarial Nonnegative Matrix Factorization (ANMF) model which can handle different types of noise or perturbations
  • To provide the robustness against real perturbations, we propose a new Adversarial Nonnegative Matrix Factorization model The adversarial perturbations of Y is used to learn the desire feature matrix A and weight matrix X
  • Different from the traditional nonnegative matrix factorization models which only focus on the regular data points, our models emphasizes potential test adversaries that are beyond the pre-defined constraints
Methods
  • Experiments and Discussions

    Experiments were carried out on multiple real-world data sets.
  • Throughout the experiments, the authors set ANMF parameters as α = 0.6, β = 10−5, γ = 10−3, λ = 10−3, and μ = 1.
  • Some representative methods, including Standard Nonnegative Matrix Factorization (SNMF), L2,1-norm based NMF model (Ding et al, 2006), Orthogonal Nonnegative Matrix Factorization (ONMF) (Kong et al, 2011), and Capped norm Nonnegative Matrix Factorization (CNMF) (Gao et al, 2015), are compared with the ANMF.
  • It is unfair to compare the method with some robust methods such as Correntropy induced NMF model and Truncated CauchyNMF model
Conclusion
  • This paper focuses on nonnegative matrix factorization problem.
  • To provide the robustness against real perturbations, the authors propose a new Adversarial Nonnegative Matrix Factorization model The adversarial perturbations of Y is used to learn the desire feature matrix A and weight matrix X.
  • The authors formulate the proposed model as a bilevel optimization problem and utilize ADMM to solve it with convergence guarantee.
  • Experimental results on real data sets validate the effectiveness and robustness of the proposed algorithm.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6):1227– 1233, 2012
Summary
  • Introduction:

    The nonnegative matrix factorization (NMF) has been a prevalent nonnegative dimensionality reduction method and successfully applied to many fields such as feature extraction (Zhi et al, 2011), video tracking (Bucak & Gunsel, 2007), image processing (Geng et al, 2012) and document clustering (Guan et al, 2012).
  • Given a data matrix Y ∈ m×N with non-negative entries, the goal of NMF is to factorize Y into the product AX of two nonnegative matrices, with n columns in A, where n is generally small.
  • Most of them independently deal with each element of X, regardless of the relationship between elements.
  • To address this issue, some scholars considered the structural information of data or variate in modeling.
  • Haeffele et al (Haeffele et al, 2014) explored a matrix factorization technique suitable for large datasets that captures additional structure in the factors by using a projective tensor norm
  • Methods:

    Experiments and Discussions

    Experiments were carried out on multiple real-world data sets.
  • Throughout the experiments, the authors set ANMF parameters as α = 0.6, β = 10−5, γ = 10−3, λ = 10−3, and μ = 1.
  • Some representative methods, including Standard Nonnegative Matrix Factorization (SNMF), L2,1-norm based NMF model (Ding et al, 2006), Orthogonal Nonnegative Matrix Factorization (ONMF) (Kong et al, 2011), and Capped norm Nonnegative Matrix Factorization (CNMF) (Gao et al, 2015), are compared with the ANMF.
  • It is unfair to compare the method with some robust methods such as Correntropy induced NMF model and Truncated CauchyNMF model
  • Conclusion:

    This paper focuses on nonnegative matrix factorization problem.
  • To provide the robustness against real perturbations, the authors propose a new Adversarial Nonnegative Matrix Factorization model The adversarial perturbations of Y is used to learn the desire feature matrix A and weight matrix X.
  • The authors formulate the proposed model as a bilevel optimization problem and utilize ADMM to solve it with convergence guarantee.
  • Experimental results on real data sets validate the effectiveness and robustness of the proposed algorithm.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6):1227– 1233, 2012
Tables
  • Table1: Description of Benchmark Datasets
  • Table2: ACC of noise-free Real Datasets. The best results are marked in bold
  • Table3: ACC of noisy Real Datasets. The best results are marked in bold
  • Table4: ACC of Real Datasets with Salt & Pepper Noise. The best results are marked in bold
  • Table5: NMI of Real Datasets with Salt & Pepper Noise. The best results are marked in bold
  • Table6: ACC of Real Datasets with Corrupt Pixel Noise. The best results are marked in bold
  • Table7: NMI of Real Datasets with Corrupt Pixel Noise. The best results are marked in bold
  • Table8: ACC of Real Datasets with regular patch noise. The best results are marked in bold
  • Table9: NMI of Real Datasets with regular patch Noise. The best results are marked in bold
Download tables as Excel
Funding
  • This work was partially supported by U.S NSF IIS 1836945, IIS 1836938, IIS 1845666, IIS 1852606, IIS 1838627, IIS 1837956
Reference
  • Bucak, S. S. and Gunsel, B. Video content representation by incremental non-negative matrix factorization. In Image Processing, 2007. ICIP 2007. IEEE International Conference on, volume 2, pp. II–113. IEEE, 2007.
    Google ScholarLocate open access versionFindings
  • Cai, D., He, X., Han, J., and Huang, T. S. Graph regularized nonnegative matrix factorization for data representation. IEEE transactions on pattern analysis and machine intelligence, 33(8):1548–1560, 2010.
    Google ScholarLocate open access versionFindings
  • Chartrand, R. and Wohlberg, B. A nonconvex admm algorithm for group sparsity with sparse groups. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 6009–601IEEE, 2013.
    Google ScholarLocate open access versionFindings
  • Ding, C., Li, T., Peng, W., and Park, H. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 126–135. ACM, 2006.
    Google ScholarLocate open access versionFindings
  • Du, L., Li, X., and Shen, Y.-D. Robust nonnegative matrix factorization via half-quadratic minimization. In Data Mining (ICDM), 2012 IEEE 12th International Conference on, pp. 201–210. IEEE, 2012.
    Google ScholarLocate open access versionFindings
  • Farnia, F., Zhang, J. M., and Tse, D. Generalizable adversarial training via spectral normalization. arXiv preprint arXiv:1811.07457, 2018.
    Findings
  • Gabay, D. and Mercier, B. A dual algorithm for the solution of non linear variational problems via finite element approximation. Institut de recherche d’informatique et d’automatique, 1975.
    Google ScholarFindings
  • Gao, H., Nie, F., Cai, W., and Huang, H. Robust capped norm nonnegative matrix factorization: Capped norm nmf. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 871–880. ACM, 2015.
    Google ScholarLocate open access versionFindings
  • Gonzalez, E. F. and Zhang, Y. Accelerating the lee-seung algorithm for nonnegative matrix factorization. Technical report, 2005.
    Google ScholarFindings
  • Großhans, M., Sawade, C., Bruckner, M., and Scheffer, T. Bayesian games for adversarial regression problems. In International Conference on Machine Learning, pp. 55–63, 2013.
    Google ScholarLocate open access versionFindings
  • Guan, N., Tao, D., Luo, Z., and Yuan, B. Nenmf: An optimal gradient method for nonnegative matrix factorization. IEEE Transactions on Signal Processing, 60(6): 2882–2898, 2012.
    Google ScholarLocate open access versionFindings
  • Guan, N., Liu, T., Zhang, Y., Tao, D., and Davis, L. S. Truncated cauchy non-negative matrix factorization for robust subspace learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
    Google ScholarLocate open access versionFindings
  • Haeffele, B., Young, E., and Vidal, R. Structured low-rank matrix factorization: Optimality, algorithm, and applications to image processing. In International conference on machine learning, pp. 2007–2015, 2014.
    Google ScholarLocate open access versionFindings
  • Hajinezhad, D., Chang, T.-H., Wang, X., Shi, Q., and Hong, M. Nonnegative matrix factorization using admm: Algorithm and convergence analysis. In Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on, pp. 4742–4746. IEEE, 2016.
    Google ScholarLocate open access versionFindings
  • Huang, J., Nie, F., Huang, H., and Ding, C. Robust manifold nonnegative matrix factorization. ACM Transactions on Knowledge Discovery from Data (TKDD), 8(3):11, 2014.
    Google ScholarLocate open access versionFindings
  • Kim, J., Monteiro, R. D., and Park, H. Group sparsity in nonnegative matrix factorization. In Proceedings of the 2012 SIAM International Conference on Data Mining, pp. 851–862. SIAM, 2012.
    Google ScholarLocate open access versionFindings
  • Kong, D., Ding, C., and Huang, H. Robust nonnegative matrix factorization using l21-norm. In Proceedings of the 20th ACM international conference on Information and knowledge management, pp. 673–682. ACM, 2011.
    Google ScholarLocate open access versionFindings
  • Lee, D. D. and Seung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755): 788, 1999.
    Google ScholarLocate open access versionFindings
  • Li, B. and Vorobeychik, Y. Feature cross-substitution in adversarial classification. In Advances in neural information processing systems, pp. 2087–2095, 2014.
    Google ScholarLocate open access versionFindings
  • Lin, C.-J. Projected gradient methods for nonnegative matrix factorization. Neural computation, 19(10):2756–2779, 2007.
    Google ScholarLocate open access versionFindings
  • Liu, T., Gong, M., and Tao, D. Large-cone nonnegative matrix factorization. IEEE transactions on neural networks and learning systems, 28(9):2129–2142, 2016.
    Google ScholarLocate open access versionFindings
  • Maaten, L. v. d. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(Nov): 2579–2605, 2008.
    Google ScholarLocate open access versionFindings
  • Mei, J.-J., Dong, Y., Huang, T.-Z., and Yin, W. Cauchy noise removal by nonconvex admm with convergence guarantees. Journal of Scientific Computing, 74(2):743– 766, 2018.
    Google ScholarLocate open access versionFindings
  • Tong, L., Yu, S., Alfeld, S., and Vorobeychik, Y. Adversarial regression with multiple learners. arXiv preprint arXiv:1806.02256, 2018.
    Findings
  • Wang, Y., Yin, W., and Zeng, J. Global convergence of admm in nonconvex nonsmooth optimization. Journal of Scientific Computing, pp. 1–35, 2015.
    Google ScholarLocate open access versionFindings
  • Zhi, R., Flierl, M., Ruan, Q., and Kleijn, W. B. Graphpreserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 41(1):38–52, 2011.
    Google ScholarLocate open access versionFindings
Author
lei luo
lei luo
yanfu Zhang
yanfu Zhang
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科