Skip Abstract Section
Abstract
Multi-view clustering has attracted significant attention and application. Nonnegative matrix factorization is one popular feature learning technology in pattern recognition. In recent years, many semi-supervised nonnegative matrix factorization algorithms are proposed by considering label information, which has achieved outstanding performance for multi-view clustering. However, most of these existing methods have either failed to consider discriminative information effectively or included too much hyper-parameters. Addressing these issues, a semi-supervised multi-view nonnegative matrix factorization with a novel fusion regularization (FRSMNMF) is developed in this paper. In this work, we uniformly constrain alignment of multiple views and discriminative information among clusters with designed fusion regularization. Meanwhile, to align the multiple views effectively, two kinds of compensating matrices are used to normalize the feature scales of different views. Additionally, we preserve the geometry structure information of labeled and unlabeled samples by introducing the graph regularization simultaneously. Due to the proposed methods, two effective optimization strategies based on multiplicative update rules are designed. Experiments implemented on six real-world datasets have demonstrated the effectiveness of our FRSMNMF comparing with several state-of-the-art unsupervised and semi-supervised approaches.
- H. F. Bassani and A. F. R. Araujo. 2015. Dimension selective self-organizing maps with time-varying structure for subspace and projected clustering. IEEE Transactions on Neural Networks and Learning Systems 26, 3(2015), 458–471.Google Scholar
Cross Ref
- Steffen Bickel and Tobias Scheffer. 2004. Multi-view clustering. In Proceedings of the 2004 SIAM International Conference on Data Mining. SIAM, 19–26.Google ScholarCross Ref
- Stephen Boyd and Lieven Vandenberghe. 2004. Convex optimization. Cambridge University Press.Google ScholarDigital Library
- Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. 2011. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 8(2011), 1548–1560.Google ScholarDigital Library
- Hao Cai, Bo Liu, Yanshan Xiao, and LuYue Lin. 2019. Semi-supervised multi-view clustering based on constrained nonnegative matrix factorization. Knowledge-Based Systems 182 (2019), 104798.Google ScholarDigital Library
- Hao Cai, Bo Liu, Yanshan Xiao, and LuYue Lin. 2020. Semi-supervised multi-view clustering based on orthonormality-constrained nonnegative matrix factorization. Information Sciences 536(2020), 171–184.Google ScholarCross Ref
- Xiao Cai, Feiping Nie, and Heng Huang. 2013. Multi-view k-means clustering on big data. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Citeseer.Google Scholar
- Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. 2015. Diversity-induced multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586–594.Google ScholarCross Ref
- Guoqing Chao, Shiliang Sun, and Jinbo Bi. 2021. A survey on multi-view clustering. IEEE Transactions on Artificial Intelligence 2, 2(2021), 146–168.Google ScholarCross Ref
- Feiqiong Chen, Guopeng Li, Shuaihui Wang, and Zhisong Pan. 2019. Multiview clustering via robust neighboring constraint nonnegative matrix factorization. Mathematical Problems in Engineering 2019 (2019).Google Scholar
- Rui Chen, Yongqiang Tang, Wensheng Zhang, and Wenlong Feng. 2022. Deep multi-view semi-supervised clustering with sample pairwise constraints. Neurocomputing 500(2022), 832–845.Google ScholarDigital Library
- Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. 2009. Introduction to algorithms. MIT press.Google Scholar
- Beilei Cui, Hong Yu, Tiantian Zhang, and Siwen Li. 2019. Self-weighted multi-view clustering with deep matrix factorization. In Proceedings of the Asian Conference on Machine Learning. 567–582.Google Scholar
- Guosheng Cui and Ye Li. 2022. Nonredundancy regularization based nonnegative matrix factorization with manifold learning for multiview data representation. Information Fusion 82(2022), 86–98.Google ScholarDigital Library
- Charles Elkan. 2003. Using the triangle inequality to accelerate k-means. In Proceedings of the 20th International Conference on Machine Learning, Vol. 3. 147–153.Google Scholar
- Derek Greene and Padraig Cunningham. 2006. Practical solutions to the problem of diagonal dominance in kernel document clustering. In Proceedings of the 23rd International Conference on Machine Learning, Vol. 148. ACM, 377–384.Google ScholarDigital Library
- Menglei Hu and Songcan Chen. 2018. Doubly aligned incomplete multi-view clustering. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI). 2262–2268.Google ScholarCross Ref
- Ling Huang, Hong-Yang Chao, and Chang-Dong Wang. 2019. Multi-view intact space clustering. Pattern Recognition 86(2019), 344 – 353.Google ScholarCross Ref
- Xiaoxiang Huang, Guosheng Cui, Dan Wu, and Ye Li. 2020. A semi-supervised approach for early identifying the abnormal carotid arteries using a modified variational autoencoder. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine. IEEE, 595–600.Google ScholarCross Ref
- Zhenyu Huang, Peng Hu, Joey Tianyi Zhou, Jiancheng Lv, and Xi Peng. 2020. Partially view-aligned clustering. Advances in Neural Information Processing Systems 33 (2020), 2892–2902.Google Scholar
- A. K. Jain, M. N. Murty, and P. J. Flynn. 1999. Data clustering: a review. 31, 3 (1999).Google Scholar
- Yu Jiang, Jing Liu, Zechao Li, and Hanqing Lu. 2014. Semi-supervised unified latent factor learning with multi-view data. Machine Vision and Applications 25 (2014), 1635–1645.Google ScholarDigital Library
- Martin Lades, Jan C Vorbruggen, Joachim Buhmann, Jörg Lange, Christoph Von Der Malsburg, Rolf P Wurtz, and Wolfgang Konen. 1993. Distortion invariant object recognition in the dynamic link architecture. IEEE Trans. Comput.3(1993), 300–311.Google ScholarDigital Library
- Daniel D Lee and H Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788–791.Google Scholar
- Daniel D Lee and H Sebastian Seung. 2000. Algorithms for non-negative matrix factorization. In Proceedings of the Advances in Neural Information Processing Systems. 556–562.Google Scholar
- Jianqiang Li, Guoxu Zhou, Yuning Qiu, Yanjiao Wang, Yu Zhang, and Shengli Xie. 2020. Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing 390(2020), 108–116.Google ScholarCross Ref
- Naiyao Liang, Zuyuan Yang, Zhenni Li, Weijun Sun, and Shengli Xie. 2020. Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints. Knowledge-Based Systems 194 (2020), 105582.Google ScholarCross Ref
- Naiyao Liang, Zuyuan Yang, Zhenni Li, Shengli Xie, and Chun-Yi Su. 2020. Semi-supervised multi-view clustering with graph-regularized partially shared non-negative matrix factorization. Knowledge-Based Systems 190 (2020), 105185.Google ScholarDigital Library
- Naiyao Liang, Zuyuan Yang, Zhenni Li, Shengli Xie, and Weijun Sun. 2021. Semi-supervised multi-view learning by using label propagation based non-negative matrix factorization. Knowledge-Based Systems 228 (2021), 107244.Google ScholarDigital Library
- Haifeng Liu, Zhaohui Wu, Xuelong Li, Deng Cai, and Thomas S Huang. 2012. Constrained nonnegative matrix factorization for image representation. Transactions on Pattern Analysis and Machine Intelligence 34, 7(2012), 1299–1311.Google ScholarDigital Library
- Jing Liu, Yu Jiang, Zechao Li, Zhi-Hua Zhou, and Hanqing Lu. 2014. Partially shared latent factor learning with multiview data. IEEE Transactions on Neural Networks and Learning Systems 26, 6(2014), 1233–1246.Google Scholar
- Jialu Liu, Chi Wang, Jing Gao, and Jiawei Han. 2013. Multi-view clustering via joint nonnegative matrix factorization. In Proceedings of the 2013 SIAM International Conference on Data Mining. SIAM, 252–260.Google ScholarCross Ref
- László Lovász and Michael D Plummer. 2009. Matching theory. Vol. 367. American Mathematical Soc.Google Scholar
- Chun-Ta Lu, Lifang He, Hao Ding, Bokai Cao, and Philip S Yu. 2018. Learning from multi-view multi-way data via structural factorization machines. In Proceedings of the 2018 World Wide Web Conference. 1593–1602.Google ScholarDigital Library
- Peng Luo, Jinye Peng, Ziyu Guan, and Jianping Fan. 2018. Dual regularized multi-view non-negative matrix factorization for clustering. Neurocomputing 294(2018), 1–11.Google ScholarCross Ref
- Feiping Nie, Guohao Cai, Jing Li, and Xuelong Li. 2017. Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Transactions on Image Processing 27, 3 (2017), 1501–1511.Google ScholarCross Ref
- Timo Ojala, Matti Pietikäinen, and Topi Mäenpää. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence7 (2002), 971–987.Google ScholarDigital Library
- Weihua Ou, Shujian Yu, Gai Li, Jian Lu, Kesheng Zhang, and Gang Xie. 2016. Multi-view non-negative matrix factorization by patch alignment framework with view consistency. Neurocomputing 204(2016), 116–124.Google ScholarDigital Library
- Nishant Rai, Sumit Negi, Santanu Chaudhury, and Om Deshmukh. 2016. Partial multi-view clustering using graph regularized NMF. In Proceedings of the International Conference on Pattern Recognition (ICPR). 2192–2197.Google ScholarCross Ref
- Farial Shahnaz, Michael W Berry, V Paul Pauca, and Robert J Plemmons. 2006. Document clustering using non-negative matrix factorization. Information Processing and Management 42, 2 (2006), 373–386.Google ScholarCross Ref
- Weixiang Shao, Lifang He, and S Yu Philip. 2015. Multiple incomplete views clustering via weighted nonnegative matrix factorization with L2, 1 regularization. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 318–334.Google ScholarDigital Library
- Jing Wang, Feng Tian, Hongchuan Yu, Chang Hong Liu, Kun Zhan, and Xiao Wang. 2017. Diverse non-negative matrix factorization for multiview data representation. IEEE Transactions on Cybernetics 48, 9 (2017), 2620–2632.Google ScholarCross Ref
- Jing Wang, Xiao Wang, Feng Tian, Chang Hong Liu, Hongchuan Yu, and Yanbei Liu. 2016. Adaptive multi-view semi-supervised nonnegative matrix factorization. In Proceedings of the International Conference on Neural Information Processing. Springer, 435–444.Google ScholarDigital Library
- Senhong Wang, Jiangzhong Cao, Fangyuan Lei, Qingyun Dai, Shangsong Liang, and Bingo Wing-Kuen Ling. 2021. Semi-supervised multi-view clustering with weighted anchor graph embedding. Computational Intelligence and Neuroscience 2021 (2021).Google Scholar
- Xiaobo Wang, Xiaojie Guo, Zhen Lei, Changqing Zhang, and Stan Z Li. 2017. Exclusivity-consistency regularized multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 923–931.Google ScholarCross Ref
- Xiumei Wang, Tianzhen Zhang, and Xinbo Gao. 2019. Multiview clustering based on non-negative matrix factorization and pairwise measurements. IEEE Transactions on Cybernetics 49, 9 (2019), 3333–3346.Google ScholarCross Ref
- Chang Xu, Dacheng Tao, and Chao Xu. 2013. A survey on multi-view learning. CoRR abs/1304.5634(2013).Google Scholar
- Jinglin Xu, Junwei Han, and Feiping Nie. 2016. Discriminatively embedded k-means for multi-view clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5356–5364.Google ScholarCross Ref
- Wei Xu, Xin Liu, and Yihong Gong. 2003. Document clustering based on non-negative matrix factorization. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. ACM, 267–273.Google ScholarDigital Library
- Xiaojun Yang, Weizhong Yu, Rong Wang, Guohao Zhang, and Feiping Nie. 2020. Fast spectral clustering learning with hierarchical bipartite graph for large-scale data. Pattern Recognition Letters 130 (2020), 345 – 352.Google ScholarDigital Library
- Yan Yang and Hao Wang. 2018. Multi-view clustering: a survey. Big Data Mining and Analytics 1, 2 (2018), 83–107.Google ScholarCross Ref
- Zuyuan Yang, Naiyao Liang, Wei Yan, Zhenni Li, and Shengli Xie. 2020. Uniform distribution non-negative matrix factorization for multiview clustering. IEEE Transactions on Cybernetics 51, 6 (2020), 3249–3262.Google ScholarCross Ref
- Zuyuan Yang, Yong Xiang, Kan Xie, and Yue Lai. 2016. Adaptive method for nonsmooth nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 28, 4(2016), 948–960.Google ScholarCross Ref
- Zuyuan Yang, Huimin Zhang, Naiyao Liang, Zhenni Li, and Weijun Sun. 2023. Semi-supervised multi-view clustering by label relaxation based non-negative matrix factorization. The Visual Computer 39, 4 (2023), 1409–1422.Google ScholarDigital Library
- Shan Zeng, Xiuying Wang, Hui Cui, Chaojie Zheng, and David Feng. 2017. A unified collaborative multikernel fuzzy clustering for multiview data. IEEE Transactions on Fuzzy Systems 26, 3 (2017), 1671–1687.Google ScholarCross Ref
- Hongyuan Zha, Xiaofeng He, Chris Ding, Ming Gu, and Horst D Simon. 2002. Spectral relaxation for k-means clustering. In Proceedings of the Advances in Neural Information Processing Systems. 1057–1064.Google Scholar
- Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4279–4287.Google ScholarCross Ref
- DongPing Zhang, YiHao Luo, YuYuan Yu, QiBin Zhao, and GuoXu Zhou. 2022. Semi-supervised multi-view clustering with dual hypergraph regularized partially shared non-negative matrix factorization. Science China Technological Sciences 65, 6 (2022), 1349–1365.Google ScholarCross Ref
- Zheng Zhang, Li Liu, Fumin Shen, Heng Tao Shen, and Ling Shao. 2018. Binary multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 7(2018), 1774–1782.Google ScholarCross Ref
- Handong Zhao, Zhengming Ding, and Yun Fu. 2017. Multi-view clustering via deep matrix factorization. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- Jing Zhao, Xijiong Xie, Xin Xu, and Shiliang Sun. 2017. Multi-view learning overview: Recent progress and new challenges. Information Fusion 38(2017), 43 – 54.Google ScholarDigital Library
- Wei Zhao, Cai Xu, Ziyu Guan, and Ying Liu. 2020. Multiview concept learning via deep matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 32, 2(2020), 814–825.Google ScholarCross Ref
- X. Zhu, S. Zhang, Y. Li, J. Zhang, L. Yang, and Y. Fang. 2019. Low-rank sparse subspace for spectral clustering. IEEE Transactions on Knowledge and Data Engineering 31, 8(2019), 1532–1543.Google ScholarDigital Library
- Linlin Zong, Xianchao Zhang, Long Zhao, Hong Yu, and Qianli Zhao. 2017. Multi-view clustering via multi-manifold regularized non-negative matrix factorization. Neural Networks 88(2017), 74–89.Google ScholarCross Ref
Index Terms
- Semi-supervised Multi-view Clustering based on Nonnegative Matrix Factorization with Fusion Regularization
Index terms have been assigned to the content through auto-classification.
Recommendations
Semi-supervised multi-view clustering based on constrained nonnegative matrix factorization
AbstractMost existing clustering approaches address multi-view clustering problems by graph regularized nonnegative matrix factorization to obtain the new representation of each view separately. And then, the additional regularization term is ...
Non-negative matrix factorization for semi-supervised data clustering
Traditional clustering algorithms are inapplicable to many real-world problems where limited knowledge from domain experts is available. Incorporating the domain knowledge can guide a clustering algorithm, consequently improving the quality of ...
Self-supervised semi-supervised nonnegative matrix factorization for data clustering
Highlights- A self-supervised semi-supervised NMF method is proposed for data clustering.
- ...
AbstractSemi-supervised nonnegative matrix factorization exploits the strengths of matrix factorization in successfully learning part-based representation and is also able to achieve high learning performance when facing a scarcity of labeled ...
Comments