Skip to main content
SpringerLink
Log in

Transfer Learning-Based Class Imbalance-Aware Shoulder Implant Classification from X-Ray Images

  • Research Article
  • Published:
Journal of Bionic Engineering Aims and scope Submit manuscript

Abstract

Total shoulder arthroplasty is a standard restorative procedure practiced by orthopedists to diagnose shoulder arthritis in which a prosthesis replaces the whole joint or a part of the joint. It is often challenging for doctors to identify the exact model and manufacturer of the prosthesis when it is unknown. This paper proposes a transfer learning-based class imbalance-aware prosthesis detection method to detect the implant’s manufacturer automatically from shoulder X-ray images. The framework of the method proposes a novel training approach and a new set of batch-normalization, dropout, and fully convolutional layers in the head network. It employs cyclical learning rates and weighting-based loss calculation mechanism. These modifications aid in faster convergence, avoid local-minima stagnation, and remove the training bias caused by imbalanced dataset. The proposed method is evaluated using seven well-known pre-trained models of VGGNet, ResNet, and DenseNet families. Experimentation is performed on a shoulder implant benchmark dataset consisting of 597 shoulder X-ray images. The proposed method improves the classification performance of all pre-trained models by 10–12%. The DenseNet-201-based variant has achieved the highest classification accuracy of 89.5%, which is 10% higher than existing methods. Further, to validate and generalize the proposed method, the existing baseline dataset is supplemented to six classes, including samples of two more implant manufacturers. Experimental results have shown average accuracy of 86.7% for the extended dataset and show the preeminence of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The data that support the findings of this study are publically available in UCI Machine Learning Repository and the website of Washington University. (https://archive.ics.uci.edu/ml/datasets/Shoulder+Implant+X-Ray+Manufacturer+Classification) (http://faculty.washington.edu/alexbert/Shoulder/CommonUSShoulderProstheses.htm).

References

  1. Hunder, G. G. (2002). Mayo clinic on arthritis paperback (G. G. Hunder (ed.); 2nd ed.). Mayo Foundation for Medical.

  2. Sanchez-Sotelo, J. (2007). (i) Shoulder arthroplasty for osteoarthritis and rheumatoid arthritis. Current Orthopaedics, 21(6), 405–414. https://doi.org/10.1016/j.cuor.2007.11.002

    Article  Google Scholar 

  3. Millett, P. J., Gobezie, R., & Boykin, R. E. (2008). Shoulder osteoarthritis: diagnosis and management. American Family Physician, 78(5), 605–611.

    Google Scholar 

  4. Cofield, R. H. (1984). Total shoulder arthroplasty with the Neer prosthesis. The Journal of Bone and Joint Surgery, 66(6), 899–906. https://doi.org/10.2106/00004623-198466060-00010

    Article  Google Scholar 

  5. Wicha, M., Tomczyk-Warunek, A., Jarecki, J., & Dubiel, A. (2020). Total shoulder arthroplasty, an overview, indications and prosthetic options. Wiadomosci Lekarskie (Warsaw, Poland : 1960), 73(9(cz. 1)), 1870–1873.

    Article  Google Scholar 

  6. Bohsali, K. I. (2006). Complications of total shoulder arthroplasty. The Journal of Bone and Joint Surgery (American), 88(10), 2279–2292. https://doi.org/10.2106/JBJS.F.00125

    Article  Google Scholar 

  7. Matsen, F. A., Clinton, J., Lynch, J., Bertelsen, A., & Richardson, M. L. (2008). Glenoid component failure in total shoulder arthroplasty. The Journal of Bone and Joint Surgery-American, 90(4), 885–896. https://doi.org/10.2106/JBJS.G.01263

    Article  Google Scholar 

  8. Papadonikolakis, A., Neradilek, M. B., & Matsen, F. A. (2013). Failure of the glenoid component in anatomic total shoulder arthroplasty. The Journal of Bone & Joint Surgery, 95(24), 2205–2212. https://doi.org/10.2106/JBJS.L.00552

    Article  Google Scholar 

  9. Dy, C. J., Bozic, K. J., Padgett, D. E., Pan, T. J., Marx, R. G., & Lyman, S. (2014). Is changing hospitals for revision total joint arthroplasty associated with more complications? Clinical Orthopaedics & Related Research, 472(7), 2006–2015. https://doi.org/10.1007/s11999-014-3515-z

    Article  Google Scholar 

  10. Branovacki, G. (2008). Ortho atlas: Hip arthroplasty US femoral implants 1938–2008. Ortho Atlas Publishing.

    Google Scholar 

  11. Wilson, N. A., Jehn, M., York, S., & Davis, C. M. (2014). Revision total hip and knee arthroplasty implant identification: Implications for use of unique device identification 2012 AAHKS member survey results. The Journal of Arthroplasty, 29(2), 251–255. https://doi.org/10.1016/j.arth.2013.06.027

    Article  Google Scholar 

  12. Yi, P. H., Kim, T. K., Wei, J., Li, X., Hager, G. D., Sair, H. I., & Fritz, J. (2020). Automated detection and classification of shoulder arthroplasty models using deep learning. Skeletal Radiology, 49(10), 1623–1632. https://doi.org/10.1007/s00256-020-03463-3

    Article  Google Scholar 

  13. Urban, G., Porhemmat, S., Stark, M., Feeley, B., Okada, K., & Baldi, P. (2020). Classifying shoulder implants in X-ray images using deep learning. Computational and Structural Biotechnology Journal, 18, 967–972. https://doi.org/10.1016/j.csbj.2020.04.005

    Article  Google Scholar 

  14. Sultan, H., Owais, M., Park, C., Mahmood, T., Haider, A., & Park, K. R. (2021). Artificial intelligence-based recognition of different types of shoulder implants in X-ray scans based on dense residual ensemble-network for personalized medicine. Journal of Personalized Medicine, 11(6), 482. https://doi.org/10.3390/jpm11060482

    Article  Google Scholar 

  15. Lundervold, A. S., & Lundervold, A. (2019). An overview of deep learning in medical imaging focusing on MRI. Zeitschrift Für Medizinische Physik, 29(2), 102–127. https://doi.org/10.1016/j.zemedi.2018.11.002

    Article  Google Scholar 

  16. Kim, M., Yan, C., Yang, D., Wang, Q., Ma, J., & Wu, G. (2020). Deep learning in biomedical image analysis. In D. D. Feng (Ed.), Biomedical Information Technology (2nd ed., pp. 239–263). Elsevier.

    Chapter  Google Scholar 

  17. Cai, L., Gao, J., & Zhao, D. (2020). A review of the application of deep learning in medical image classification and segmentation. Annals of Translational Medicine, 8(11), 713–727. https://doi.org/10.21037/atm.2020.02.44

    Article  Google Scholar 

  18. Su, Q., Wang, F., Chen, D., Chen, G., Li, C., & Wei, L. (2022). Deep convolutional neural networks with ensemble learning and transfer learning for automated detection of gastrointestinal diseases. Computers in Biology and Medicine, 150, 106054. https://doi.org/10.1016/j.compbiomed.2022.106054

    Article  Google Scholar 

  19. Marín, R., & Chang, V. (2021). Impact of transfer learning for human sperm segmentation using deep learning. Computers in Biology and Medicine, 136, 104687. https://doi.org/10.1016/j.compbiomed.2021.104687

    Article  Google Scholar 

  20. Zou, J., Zhang, X., Zhang, Y., Li, J., & Jin, Z. (2022). Prediction on the medial knee contact force in patients with knee valgus using transfer learning approaches: Application to rehabilitation gaits. Computers in Biology and Medicine, 150, 106099. https://doi.org/10.1016/j.compbiomed.2022.106099

    Article  Google Scholar 

  21. Zhan, Q., Wang, L., Ren, L., & Huang, X. (2022). A novel heterogeneous transfer learning method based on data stitching for the sequential coding brain computer interface. Computers in Biology and Medicine, 151, 106220. https://doi.org/10.1016/j.compbiomed.2022.106220

    Article  Google Scholar 

  22. Garg, S., Kumar, S., & Muhuri, P. K. (2022). A novel approach for COVID-19 Infection forecasting based on multi-source deep transfer learning. Computers in Biology and Medicine, 149, 105915. https://doi.org/10.1016/j.compbiomed.2022.105915

    Article  Google Scholar 

  23. Stark, M. B. C. G. (2018). Automatic detection and segmentation of shoulder implants in x-ray images, The California State University. https://scholarworks.calstate.edu/concern/theses/79407z98n?sequence=1. Accessed 19 Jan 2023.

  24. Zhou, M., & Mo, S. (2021). Shoulder Implant x-ray manufacturer classification: exploring with vision transformer. ARXIV: Electrical engineering and systems science [Eess.IV] Preprint. ArXiv:2104.07667

  25. Karaci, A. (2021). X-ışını görüntülerinden omuz implantlarının tespiti ve sınıflandırılması: YOLO ve önceden eğitilmiş evrişimsel sinir ağı tabanlı bir yaklaşım. Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi, 37(1), 283–294. https://doi.org/10.17341/gazimmfd.888202

    Article  Google Scholar 

  26. Vo, M. T., Vo, A. H., & Le, T. (2022). A robust framework for shoulder implant X-ray image classification. Data Technologies and Applications, 56(3), 447–460. https://doi.org/10.1108/DTA-08-2021-0210

    Article  Google Scholar 

  27. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, Li. (2009). ImageNet: A large-scale hierarchical image database. Proceedings of IEEE Conference Computer Vision and Pattern Recognition (CVPR) (pp. 248–255). Miami, FL, USA: IEEE.

    Google Scholar 

  28. University of Washington: Common US Shoulder Prostheses. (n.d.). Retrieved January 3, 2021, from http://faculty.washington.edu/alexbert/Shoulder/CommonUSShoulderProstheses.htm

  29. Bredow, J., Wenk, B., Westphal, R., Wahl, F., Budde, S., Eysel, P., & Oppermann, J. (2014). Software-based matching of x-ray images and 3D models of knee prostheses. Technology and Health Care, 22(6), 895–900. https://doi.org/10.3233/THC-140858

    Article  Google Scholar 

  30. Li, L., Wang, X., Meng, Q., Chen, C., Sun, J., & Yu, H. (2022). Intelligent knee prostheses: A systematic review of control strategies. Journal of Bionic Engineering, 19(5), 1242–1260. https://doi.org/10.1007/s42235-022-00169-1

    Article  Google Scholar 

  31. Sun, Y., Tang, H., Tang, Y., Zheng, J., Dong, D., Chen, X., Liu, F., Bai, L., Ge, W., Xin, L., Pu, H., Peng, Y., & Luo, J. (2021). Review of recent progress in robotic knee prosthesis related techniques: Structure, actuation and control. Journal of Bionic Engineering, 18(4), 764–785. https://doi.org/10.1007/s42235-021-0065-4

    Article  Google Scholar 

  32. Malathy, C., Sharma, U., Mayuri Naidu, C., & Uma Pratheebha, U. (2016). A new approach for recognition of implant in knee by template matching. Indian Journal of Science and Technology. https://doi.org/10.17485/ijst/2016/v9i37/102081

    Article  Google Scholar 

  33. Borjali, A., Chen, A. F., Muratoglu, O. K., Morid, M. A., & Varadarajan, K. M. (2020). Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network. Journal of Orthopaedic Research, 38(7), 1465–1471. https://doi.org/10.1002/jor.24617

    Article  Google Scholar 

  34. Hough, P. V. C. (1962). Method and means for recognizing complex pattern. U.S. Patent No. 3069654 issued December 18, 1962.

  35. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324

    Article  Google Scholar 

  36. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451

    Article  MathSciNet  Google Scholar 

  37. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27. https://doi.org/10.1109/TIT.1967.1053964

    Article  Google Scholar 

  38. Defazio, A., Bach, F., & Lacoste-Julien, S. (2014). SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. Procedings of Advances in Neural Information Processing Systems 27 (NIPS). Montreal, Quebec, Canada, 2014. ArXiv Preprint ArXiv:1407.0202

  39. Kim, E., Corte-Real, M., & Baloch, Z. (2016). A deep semantic mobile application for thyroid cytopathology. Proceedings of SPIE Medical Imaging Conference PACS and Imaging Informatics (p. 97890). San Diego, CA, United States: Next Generation and Innovations.

    Google Scholar 

  40. Antony, J., McGuinness, K., O’Connor, N. E., & Moran, K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, 2016, 1195–1200. https://doi.org/10.1109/ICPR.2016.7899799

  41. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., Kim, R., Raman, R., Nelson, P. C., Mega, J. L., & Webster, D. R. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402. https://doi.org/10.1001/jama.2016.17216

    Article  Google Scholar 

  42. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056

    Article  Google Scholar 

  43. Jiang, Z., Zhang, H., Wang, Y., & Ko, S.-B. (2018). Retinal blood vessel segmentation using fully convolutional network with transfer learning. Computerized Medical Imaging and Graphics, 68, 1–15. https://doi.org/10.1016/j.compmedimag.2018.04.005

    Article  Google Scholar 

  44. Ahn, E., Kumar, A., Kim, J., Li, C., Feng, D., & Fulham, M. (2016). X-ray image classification using domain transferred convolutional neural networks and local sparse spatial pyramid. 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), 855–858. https://doi.org/10.1109/ISBI.2016.7493400

  45. Wang, J., Ding, H., Bidgoli, F. A., Zhou, B., Iribarren, C., Molloi, S., & Baldi, P. (2017). Detecting cardiovascular disease from mammograms with deep learning. IEEE Transactions on Medical Imaging, 36(5), 1172–1181. https://doi.org/10.1109/TMI.2017.2655486

    Article  Google Scholar 

  46. Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014). How transferable are features in deep neural networks? Procedings of Advances in Neural Information Processing Systems (NIPS), Montreal, Quebec, Canada, 2014, 3320–3328. https://doi.org/10.48550/arXiv.1411.1792

  47. Bozinovski, S. (2020). Reminder of the first paper on transfer learning in neural networks, 1976. Informatica. https://doi.org/10.31449/inf.v44i3.2828

    Article  Google Scholar 

  48. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929–1958.

    MathSciNet  Google Scholar 

  49. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2014. ArXiv Preprint ArXiv:1409.1556

  50. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  51. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. Procedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 770–778. https://doi.org/10.1109/CVPR.2016.90

  52. Ioffe, S., & Szegedy, C. (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning (ICML), Volume 37, Lille, France, 2015, 448–456. ArXiv Preprint ArXiv:1502.03167

  53. Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 2010, 807–814

  54. Maas, A. L., Hannun, A. Y., & Ng, A. Y. (2013). Rectifier nonlinearities improve neural network acoustic models. In ICML Workshop on Deep Learning for Audio, Speech and Language Processing, 30(1), 3

  55. Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. Q. (2016). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, 4700–4708. https://doi.org/10.1109/CVPR.2017.243

  56. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. https://doi.org/10.1007/s10710-017-9314-z

    Article  Google Scholar 

  57. Garbin, C., Zhu, X., & Marques, O. (2020). Dropout vs. batch normalization: an empirical study of their impact to deep learning. Multimedia Tools and Applications, 79(19–20), 12777–12815. https://doi.org/10.1007/s11042-019-08453-9

    Article  Google Scholar 

  58. Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016). Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Transactions on Medical Imaging, 35(5), 1299–1312. https://doi.org/10.1109/TMI.2016.2535302

    Article  Google Scholar 

  59. Smith, L. N. (2017). Cyclical learning rates for training neural networks. Procedings of IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 2017, 464–472. ArXiv Preprint ArXiv:1506.01186

  60. Smith, L. N. (2018). A disciplined approach to neural network hyper-parameters: Part 1--learning rate, batch size, momentum, and weight decay. ArXiv Preprint ArXiv:1803.09820

  61. Moody, J., Hanson, S., Krogh, A., & Hertz, J. A. (1992). A simple weight decay can improve generalization. Advances in Neural Information Processing Systems, 4, 950–957. https://doi.org/10.5555/2986916.2987033

    Article  Google Scholar 

  62. King, G., & Zeng, L. (2001). Logistic regression in rare events data. Political Analysis, 9(2), 137–163. https://doi.org/10.1093/oxfordjournals.pan.a004868

    Article  Google Scholar 

  63. Loshchilov, I., & Hutter, F. (2017). Sgdr: Stochastic gradient descent with warm restarts. Procedings of 5th International Conference on Learning Representations (ICLR), Toulon, France, 2017, ArXiv Preprint ArXiv:1608.03983

  64. Okada, K., Stark, M. B., & Feeley, B. (2020). Shoulder implant X-ray manufacturer classification data set. https://archive.ics.uci.edu/ml/datasets/Shoulder+Implant+X-Ray+Manufacturer+Classification. Accessed 15 Dec 2022.

Download references

Acknowledgements

The authors appreciate Washington University for making the shoulder X-ray images available on their website for research purposes. The authors also want to express sincere thanks to Google LLC for making their Colaboratory Service available with free resources to students and researchers.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Sant Longowal Institute of Engineering and Technology, Sangrur, 148106, Punjab, India

    Marut Jindal & Birmohan Singh

Authors
  1. Marut Jindal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Birmohan Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Birmohan Singh.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval and Human Rights

All procedures performed in this study does not involve human participation. The X-ray images used in this study are available in online databases without any identity of the patients.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jindal, M., Singh, B. Transfer Learning-Based Class Imbalance-Aware Shoulder Implant Classification from X-Ray Images. J Bionic Eng 21, 892–912 (2024). https://doi.org/10.1007/s42235-023-00477-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42235-023-00477-0

Keywords

Search

Navigation