Skip to main content
SpringerLink
Log in

Recursive universum linear discriminant analysis

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

Universum linear discriminant analysis was recently proposed to improve linear discriminant analysis by incorporating Universum information. However, it obtains each of the discriminant directions by just using samples from two classes, while other class samples are considered as Universum. This not only leads to the ignoring of some discriminant information, but also restricts its number of extracted features to at most \(0.5k(k-1)\), where k is the number of classes. To fully explore discriminant information from all classes, this paper studies a novel Universum linear discriminant analysis by considering a unified model that simultaneously uses all classes. Compared to the existing Universum linear discriminant analysis, all Universum information is fully utilized in the proposed model when obtaining each discriminant direction, where the Universum can be self-constructed as well can be given advanced of any types. The constrained “concave-convex" procedure can be used to solve the proposed method, which makes the algorithm convergent to a local minimum. By employing a recursive technique, arbitrary number of discriminant directions can be obtained. Experimental results on real-world benchmark datasets and image datasets illustrate the advantages of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data availability statement

The data that support the findings of this study are openly available in [https://archive.ics.uci.edu/ml/datasets.php], and IMM face and Indian females datasets are available on request from the authors.

References

  1. Hastie, T., Tibshirani, R., Friedman, JH.: et al. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. springer, New York (2009)

  2. Moore, B.C.: Principal component analysis in linear systems: controllability, observability, and model reduction. IEEE Trans. Auto. Control 26(1), 17–32 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  3. Jolliffe, I.T., Cadima, J.: Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  4. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugenics 7(2), 179–188 (1936)

    Article  Google Scholar 

  5. Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, New York (1991)

    MATH  Google Scholar 

  6. Vapnik, V.: Statistical Learning Theory, Wiley-Interscience, (1998)

  7. Vapnik, V.: Estimation of Dependencies Based on Empirical Data (Information Science and Statistics). Springer-Verlag, New York (2006)

    Book  MATH  Google Scholar 

  8. Zhang, D., Wang, J., Si, L.: Document clustering with Universum. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, PP. 873–882 (2011)

  9. Shen, C., Wang, P., Shen, F., et al.: \(\cal{U}\)Boost: Boosting with the Universum. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 825–832 (2011)

    Article  Google Scholar 

  10. Liu, C.L., Hsaio, W.H., Lee, C.H., et al.: Semi-supervised text classification with Universum learning. IEEE Trans. Cyber. 46(2), 462–473 (2015)

    Article  Google Scholar 

  11. Richhariya, B., Tanveer, M., Rashid, A.H., et al.: Diagnosis of Alzheimer’s disease using Universum support vector machine based recursive feature elimination (USVM-RFE). Biomed. Signal Process. Control 59, 101903 (2020)

    Article  Google Scholar 

  12. Weston, J., Collobert, R., Sinz, F.: et al. Inference with the Universum. In: Proceedings of the 23rd International Conference on Machine Learning, PP. 1009–1016 (2006)

  13. Qi, Z., Tian, Y., Shi, Y.: Twin support vector machine with Universum data. Neural Netw. 36, 112–119 (2012)

    Article  MATH  Google Scholar 

  14. Dhar, S., Cherkassky, V.: Development and evaluation of cost-sensitive Universum-SVM. IEEE Trans. Cyber. 45(4), 806–818 (2014)

    Article  Google Scholar 

  15. Richhariya, B., Tanveer, M.: A reduced Universum twin support vector machine for class imbalance learning. Pattern Recogn. 102, 107150 (2020)

    Article  Google Scholar 

  16. Moosaei, H., Bazikar, F., Ketabchi, S., et al.: Universum parametric-margin \(\nu \)-support vector machine for classification using the difference of convex functions algorithm. Appl. Intell. 52(3), 2634–2654 (2022)

    Article  Google Scholar 

  17. Dhar, S., Cherkassky, V., Shah, M.: Multiclass learning from contradictions. Adv. Neural Inf. Process. Syst., P. 32 (2019)

  18. Wang, Z., Zhu, Y., Liu, W., et al.: Multi-view learning with Universum. Knowl. Based Syst. 70, 376–391 (2014)

    Article  Google Scholar 

  19. Zhang, D., Wang, J., Wang, F.: et al. Semi-supervised classification with Universum. In: Proceedings of the 2008 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, pp. 323–333 (2008)

  20. Xiao, Y., Wen, J., Liu, B.: A new multi-task learning method with Universum data. Appl. Intell. 51(6), 3421–3434 (2021)

    Article  Google Scholar 

  21. Xiao, Y., Feng, J., Liu, B.: A new transductive learning method with universum data. Appl. Intell. 51(8), 5571–5583 (2021)

    Article  Google Scholar 

  22. Chapelle, O., Agarwal, A., Sinz, F.: et al. An analysis of inference with the Universum. Adv. Neural Inf. Process. Syst., p. 20 (2007)

  23. Chen, S., Zhang, C.: Selecting informative Universum sample for semi-supervised learning. In: Twenty-First International Joint Conference on Artificial Intelligence (2009)

  24. Cherkassky, V., Dhar, S., Dai, W.: Practical conditions for effectiveness of the Universum learning. IEEE Trans. Neural Netw. 22(8), 1241–1255 (2011)

    Article  Google Scholar 

  25. Chen, X., Ma, D.: Universum principal component analysis. n: International Conference on Information Technology and Management Engineering, pp. 236–241 (2014)

  26. Qiu, J., Zhang, Y., Pan, Z.: et al. A novel semi-supervised approach for feature extraction. In: 2016 International Joint Conference on Neural Networks (IJCNN). IEEE , pp. 3765–3770 (2016)

  27. Chen, X., Yin, H., Jiang, F., et al.: Multi-view dimensionality reduction based on Universum learning. Neurocomputing 275, 2279–2286 (2018)

    Article  Google Scholar 

  28. Chen, X.H., Chen, S.C., Xue, H.: Universum linear discriminant analysis. Electronics Lett. 48(22), 1407–1409 (2012)

    Article  Google Scholar 

  29. Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15(4), 915–936 (2003)

    Article  MATH  Google Scholar 

  30. Heisele, B., Poggio, T., Pontil, M.: Face detecton in still gray images (2000). publications.ai.mit.edu,ai-publications/1500-1999/AIM-1687.ps.Z

  31. Xiang, C., Fan, X.A., Lee, T.H.: Face recognition using recursive Fisher linear discriminant. IEEE Trans. Image Process. 15(8), 2097–2105 (2006)

    Article  Google Scholar 

  32. Stegmann, M.B., Ersboll, B.K., Larsen, R.: FAME-a flexible appearance modeling environment. IEEE Trans. Med. Imag. 22(10), 1319–1331 (2003)

    Article  Google Scholar 

  33. Nordstrøm M.M., Larsen M., Sierakowski J. et al.: The IMM face database-an annotated dataset of 240 face images. 2004, Technical University of Denmark, DTU Informatics, Building 321, Technical Report

  34. Jain, V., Mukherjee, A.: The Indian face database. http://vis-www.cs.umass.edu/ vidit/IndianFaceDatabase/ (2002)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 62066012 and 12271131), and the Hainan Provincial Natural Science Foundation of China (Nos. 620QN234 and 120RC449).

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Hainan University, Haikou, 570228, People’s Republic of China

    Chun-Na Li & Yuan-Hai Shao

  2. Hainan University, Haikou, 570228, People’s Republic of China

    Jiakou Liu

  3. NHC Key Laboratory of Male Reproduction and Genetics, Guangdong Provincial Reproductive Science Institute (Guangdong Provincial Fertility Hospital), Guangzhou, 510600, People’s Republic of China

    Yanhui Meng

Authors
  1. Chun-Na Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiakou Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanhui Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan-Hai Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan-Hai Shao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in the paper “Recursive Universum linear discriminant analysis”.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, CN., Liu, J., Meng, Y. et al. Recursive universum linear discriminant analysis. Optim Lett (2023). https://doi.org/10.1007/s11590-023-02067-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11590-023-02067-9

Keywords

Search

Navigation