AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
A graduated assignment approach is proposed for two challenging real-world problems – multi-graph matching and multi-graph matching and clustering, together with an unsupervised learning scheme for both problems

Graduated Assignment for Joint Multi-Graph Matching and Clustering with Application to Unsupervised Graph Matching Network Learning

NIPS 2020, (2020)

Cited by: 0|Views78
EI
Full Text
Bibtex
Weibo

Abstract

This paper considers the setting of jointly matching and clustering multiple graphs belonging to different groups, which naturally rises in many realistic problems. Both graph matching and clustering are challenging (NP-hard) and a joint solution is appealing due to the natural connection of the two tasks. In this paper, we resort to a gr...More

Code:

Data:

0
Introduction
  • Graph matching (GM) computes the node-to-node matching relationship among multiple graphs, by utilizing the structural information in graphs, which can be applied to align various real-world graphs.
  • Traditional GM algorithms mainly solve two-graph matching problems via continuous relaxation [2, 3, 4, 5, 6] or tree search algorithms [7, 8] with fixed affinity metric, while there exists increasing interest on developing more robust graph matching by fusing multiple graph information, known as multi-graph matching (MGM) [9, 10, 11, 12, 13, 14] and adopting learnable affinity functions.
  • Motivated by the fact that MGM will usually result in better matching results compared to pairwise matching [10], it is appealing to devise an unsupervised deep multi-graph matching learning algorithm by utilizing multi-graph matching information as the pseudo label for pairwise matchings
Highlights
  • Introduction and Related Works

    Graph matching (GM) computes the node-to-node matching relationship among multiple graphs, by utilizing the structural information in graphs, which can be applied to align various real-world graphs
  • Traditional GM algorithms mainly solve two-graph matching problems via continuous relaxation [2, 3, 4, 5, 6] or tree search algorithms [7, 8] with fixed affinity metric, while there exists increasing interest on developing more robust graph matching by fusing multiple graph information, known as multi-graph matching (MGM) [9, 10, 11, 12, 13, 14] and adopting learnable affinity functions
  • We have presented a joint graph matching and clustering method, whose matching results have been shown can serve as pseudo ground truth to train a two-graph matching network
  • Broader Impact a) Who may benefit from this research
  • A graduated assignment approach is proposed for two challenging real-world problems – multi-graph matching (MGM) and multi-graph matching and clustering (MGMC), together with an unsupervised learning scheme for both problems
  • The wide range of applications in pattern recognition and data mining may benefit from this research, as unsupervised learning with comparative performance with supervised learning is usually welcomed for real-world applications
Methods
  • Evaluation Protocol is built on two real-world graph matching benchmarks: Willow Object Class [16] dataset and CUB2011 dataset [46].
  • The authors' evaluation protocol is built in line with deep graph matching peer methods [18, 37], and the authors use the matlab code released by [20] for MGMC peer methods upon the authors’ approval.
  • Given one predicted assignment matrix X and its ground truth Xgt, precision tr(X Xgt sum(X).
Conclusion
  • The authors have presented a joint graph matching and clustering method, whose matching results have been shown can serve as pseudo ground truth to train a two-graph matching network.
  • A graduated assignment approach is proposed for two challenging real-world problems – multi-graph matching (MGM) and multi-graph matching and clustering (MGMC), together with an unsupervised learning scheme for both problems.
  • The wide range of applications in pattern recognition and data mining may benefit from this research, as unsupervised learning with comparative performance with supervised learning is usually welcomed for real-world applications
Summary
  • Introduction:

    Graph matching (GM) computes the node-to-node matching relationship among multiple graphs, by utilizing the structural information in graphs, which can be applied to align various real-world graphs.
  • Traditional GM algorithms mainly solve two-graph matching problems via continuous relaxation [2, 3, 4, 5, 6] or tree search algorithms [7, 8] with fixed affinity metric, while there exists increasing interest on developing more robust graph matching by fusing multiple graph information, known as multi-graph matching (MGM) [9, 10, 11, 12, 13, 14] and adopting learnable affinity functions.
  • Motivated by the fact that MGM will usually result in better matching results compared to pairwise matching [10], it is appealing to devise an unsupervised deep multi-graph matching learning algorithm by utilizing multi-graph matching information as the pseudo label for pairwise matchings
  • Methods:

    Evaluation Protocol is built on two real-world graph matching benchmarks: Willow Object Class [16] dataset and CUB2011 dataset [46].
  • The authors' evaluation protocol is built in line with deep graph matching peer methods [18, 37], and the authors use the matlab code released by [20] for MGMC peer methods upon the authors’ approval.
  • Given one predicted assignment matrix X and its ground truth Xgt, precision tr(X Xgt sum(X).
  • Conclusion:

    The authors have presented a joint graph matching and clustering method, whose matching results have been shown can serve as pseudo ground truth to train a two-graph matching network.
  • A graduated assignment approach is proposed for two challenging real-world problems – multi-graph matching (MGM) and multi-graph matching and clustering (MGMC), together with an unsupervised learning scheme for both problems.
  • The wide range of applications in pattern recognition and data mining may benefit from this research, as unsupervised learning with comparative performance with supervised learning is usually welcomed for real-world applications
Tables
  • Table1: Parameter configurations to reproduce our reported results in this paper
  • Table2: Matching accuracy with both learning-free MGM methods and supervised learning peer methods on Willow dataset (50 tests). Compared results are quoted from the original papers
  • Table3: Multiple graph matching and clustering evaluation (with inference time) on Willow dataset
Download tables as Excel
Related work
  • Closely related works on graduated assignment, multi-graph matching, and learning graph matching are reviewed here. Readers are referred to [21] for a comprehensive review.

    1) Graduated Assignment. The classical graduated assignment (GA) is originally proposed to approximately solve higher-order assignment, with applications to challenging combinatorial problems, e.g. two graph matching [6], multi-graph matching [22, 14] and traveling salesman problem [23]. GA works with the partial derivative of the original objective function, which can be computed via Taylor expansion, and the resulting linear assignment problem is solved approximately with regularized Sinkhorn algorithm [24, 25]. In this paper, the recent advances in neural network motivate us to develop a graduated assignment neural network (GANN) for two challenging real-world combinatorial tasks – MGM and MGMC.

    2) Multi-Graph Matching and Clustering. MGM arises in scenarios where more than two graphs should be matched jointly, e.g. analyzing video sequences, some authors propose to optimize matching based on a joint matching objective function [26, 9, 27], and others aim to develop a postprocessing technique to recover global consistency based on existing pairwise matching results [11, 28, 29, 30]. More realistic settings are also considered in recent literature, e.g. online incremental matching [31, 27], joint matching and partition [32], joint link prediction and matching [33]. The MGMC problem proposed by [20] can be regarded as a realistic extension from MGM, where graphs may belong to different categories. Both MGM and MGMC are considered in this paper.
Funding
  • Acknowledgments and Disclosure of Funding This work was partially supported by National Key Research and Development Program of China 2020AAA0107600, and NSFC (61972250, U19B2035, U1609220), and CCF-Tencent Open Fund RAGR20200113 and Tencent AI Lab Rhino-Bird Visiting Scholars Program
  • The author Runzhong Wang is also sponsored by Wen-Tsun Wu Honorary Doctoral Scholarship, AI Institute, Shanghai Jiao Tong University
Reference
  • E. M. Loiola, N. M. de Abreu, P. O. Boaventura-Netto, P. Hahn, and T. Querido, “A survey for the quadratic assignment problem,” EJOR, pp. 657–90, 2007.
    Google ScholarLocate open access versionFindings
  • M. Cho, J. Lee, and K. M. Lee, “Reweighted random walks for graph matching,” in ECCV, 2010.
    Google ScholarFindings
  • M. Leordeanu and M. Hebert, “A spectral technique for correspondence problems using pairwise constraints,” in ICCV, 2005.
    Google ScholarFindings
  • T. Yu, J. Yan, Y. Wang, W. Liu, and b. Li, “Generalizing graph matching beyond quadratic assignment model,” in Advances in Neural Information Processing Systems 31, 2018, pp. 853–863.
    Google ScholarLocate open access versionFindings
  • N. Dym, H. Maron, and Y. Lipman, “Ds++ a flexible, scalable and provably tight relaxation for matching problems,” ACM Transactions on Graphics (TOG), vol. 36, no. 6, pp. 1–14, 2017.
    Google ScholarLocate open access versionFindings
  • S. Gold and A. Rangarajan, “A graduated assignment algorithm for graph matching,” TPAMI, 1996.
    Google ScholarLocate open access versionFindings
  • C. Edwards, “A branch and bound algorithm for the koopmans-beckmann quadratic assignment problem,” in Combinatorial optimization II. Springer, 1980, pp. 35–52.
    Google ScholarFindings
  • P. Hahn, T. Grant, and N. Hall, “A branch-and-bound algorithm for the quadratic assignment problem based on the hungarian method,” EJOR, vol. 108, no. 3, pp. 629 – 640, 1998.
    Google ScholarLocate open access versionFindings
  • J. Yan, J. Wang, H. Zha, X. Yang, and S. Chu, “Consistency-driven alternating optimization for multigraph matching: A unified approach,” TIP, vol. 24, no. 3, pp. 994–1009, 2015.
    Google ScholarLocate open access versionFindings
  • J. Yan, M. Cho, H. Zha, X. Yang, and S. Chu, “Multi-graph matching via affinity optimization with graduated consistency regularization,” TPAMI, 2016.
    Google ScholarLocate open access versionFindings
  • D. Pachauri, R. Kondor, and S. Vikas, “Solving the multi-way matching problem by permutation synchronization,” in NIPS, 2013.
    Google ScholarFindings
  • Q. Wang, X. Zhou, and K. Daniilidis, “Multi-image semantic matching by mining consistent features,” in CVPR, 2018.
    Google ScholarFindings
  • Y. Chen, L. Guibas, and Q. Huang, “Near-optimal joint object matching via convex relaxation,” in ICML, 2014.
    Google ScholarFindings
  • A. Solé-Ribalta and F. Serratosa, “Graduated assignment algorithm for multiple graph matching based on a common labeling,” IJPRAI, 2013.
    Google ScholarLocate open access versionFindings
  • M. Leordeanu, R. Sukthankar, and M. Hebert, “Unsupervised learning for graph matching,” IJCV, pp. 28–45, 2012.
    Google ScholarLocate open access versionFindings
  • M. Cho, K. Alahari, and J. Ponce, “Learning graphs to match,” in ICCV, 2013.
    Google ScholarFindings
  • A. Zanfir and C. Sminchisescu, “Deep learning of graph matching,” in CVPR, 2018.
    Google ScholarFindings
  • R. Wang, J. Yan, and X. Yang, “Learning combinatorial embedding networks for deep graph matching,” in ICCV, 2019.
    Google ScholarFindings
  • Z. Zhang and W. S. Lee, “Deep graphical feature learning for the feature matching problem,” in ICCV, 2019, pp. 5087–5096.
    Google ScholarFindings
  • T. Wang, Z. Jiang, and J. Yan, “Clustering-aware multiple graph matching via decayed pairwise matching composition,” AAAI, 2020.
    Google ScholarLocate open access versionFindings
  • J. Yan, X. Yin, W. Lin, C. Deng, H. Zha, and X. Yang, “A short survey of recent advances in graph matching,” in ICMR, 2016.
    Google ScholarFindings
  • A. Solé-Ribalta and F. Serratosa, “Graduated assignment algorithm for finding the common labelling of a set of graphs,” in Structural, Syntactic, and Statistical Pattern Recognition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 180–190.
    Google ScholarFindings
  • S. Gold and A. Rangarajan, “Softmax to softassign: neural network algorithms for combinatorial optimization,” J. Artif. Neural Netw, 1996.
    Google ScholarLocate open access versionFindings
  • R. Sinkhorn and A. Rangarajan, “A relationship between arbitrary positive matrices and doubly stochastic matrices,” Ann. Math. Statistics, 1964.
    Google ScholarLocate open access versionFindings
  • M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” NIPS, pp. 2292–2300, 2013.
    Google ScholarLocate open access versionFindings
  • J. Yan, Y. Tian, H. Zha, X. Yang, Y. Zhang, and S. Chu, “Joint optimization for consistent multiple graph matching,” in ICCV, 2013.
    Google ScholarFindings
  • Z. Jiang, T. Wang, and J. Yan, “Unifying offline and online multi-graph matching via finding shortest paths on supergraph,” TPAMI, pp. 1–1, 2020.
    Google ScholarLocate open access versionFindings
  • Q. Huang and L. Guibas, “Consistent shape maps via semidefinite programming,” in SGP, 2013.
    Google ScholarFindings
  • R. Tron, X. Zhou, C. Esteves, and K. Daniilidis, “Fast multi-image matching via density-based clustering,” in CVPR, 2017, pp. 4057–4066.
    Google ScholarFindings
  • X. Zhou, M. Zhu, and K. Daniilidis, “Multi-image matching via fast alternating minimization,” in ICCV, 2015.
    Google ScholarFindings
  • T. Yu, J. Yan, W. Liu, and B. Li, “Incremental multi-graph matching via diversity and randomness based graph clustering,” in ECCV, 2018.
    Google ScholarFindings
  • T. Yu, J. Yan, J. Zhao, and B. Li, “Joint cuts and matching of partitions in one graph,” in CVPR, 2018.
    Google ScholarFindings
  • X. Du, J. Yan, and H. Zha, “Joint link prediction and network alignment via cross-graph embedding,” IJCAI, pp. 2251–2257, 2019.
    Google ScholarLocate open access versionFindings
  • F. Bernard, J. Thunberg, P. Swoboda, and C. Theobalt, “Hippi: Higher-order projected power iterations for scalable multi-matching,” in ICCV, 2019, pp. 10 284–10 293.
    Google ScholarFindings
  • K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in ICLR, 2014.
    Google ScholarLocate open access versionFindings
  • T. Yu, R. Wang, J. Yan, and B. Li, “Learning deep graph matching with channel-independent embedding and hungarian attention,” in ICLR, 2020.
    Google ScholarFindings
  • M. Fey, J. E. Lenssen, C. Morris, J. Masci, and N. M. Kriege, “Deep graph matching consensus,” in ICLR, 2020.
    Google ScholarFindings
  • J. Huang, M. Patwary, and G. Diamos, “Coloring big graphs with alphagozero,” arXiv:1902.10162, 2019.
    Findings
  • W. Kool and M. Welling, “Attention solves your tsp,” arXiv:1803.08475, 2018.
    Findings
  • A. Nowak, S. Villar, A. Bandeira, and J. Bruna, “Revised note on learning quadratic assignment with graph neural networks,” in DSW, 2018.
    Google ScholarFindings
  • R. Wang, J. Yan, and X. Yang, “Neural graph matching network: Learning lawler’s quadratic assignment problem with extension to hypergraph and multiple-graph matching,” arXiv preprint arXiv:1911.11308, 2019.
    Findings
  • T. C. Koopmans and M. Beckmann, “Assignment problems and the location of economic activities,” Econometrica, pp. 53–76, 1957.
    Google ScholarLocate open access versionFindings
  • H. W. Kuhn, “The hungarian method for the assignment problem,” in Export. Naval Research Logistics Quarterly, 1955, pp. 83–97.
    Google ScholarLocate open access versionFindings
  • J. J. Kosowsky and A. L. Yuille, “The invisible hand algorithm: solving the assignment problem with statistical physics,” Neural Networks, vol. 7, no. 3, pp. 477–490, 1994.
    Google ScholarLocate open access versionFindings
  • D. Arthur and S. Vassilvitskii, “K-means++: The advantages of careful seeding,” in Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, ser. SODA ’07, USA, 2007, p. 1027–1035.
    Google ScholarLocate open access versionFindings
  • C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The Caltech-UCSD Birds-200-2011 Dataset,” California Institute of Technology, Tech. Rep. CNS-TR-2011-001, 2011.
    Google ScholarFindings
  • J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
    Google ScholarLocate open access versionFindings
  • C. D. Manning, P. Raghavan, and H. Schutze, Introduction to Information Retrieval. Cambridge University Press, 2005.
    Google ScholarFindings
  • W. M. Rand, “Objective criteria for the evaluation of clustering methods,” Journal of the American Statistical Association, vol. 66, no. 336, pp. 846–850, 1971.
    Google ScholarLocate open access versionFindings
  • G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” California Institute of Technology, Tech. Rep., 2007.
    Google ScholarLocate open access versionFindings
  • M. Everingham, L. Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” IJCV, vol. 88, no. 2, p. 303–338, Jun. 2010.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科