Abstract
One of the main challenges, since the advancement of convolutional neural networks is how to connect the extracted feature map to the final classification layer. VGG models used two sets of fully connected layers for the classification part of their architectures, which significantly increased the number of models’ weights. ResNet and the next deep convolutional models used the global average pooling layer to compress the feature map and feed it to the classification layer. Although using the GAP layer reduces the computational cost, but also causes losing spatial resolution of the feature map, which results in decreasing learning efficiency. In this paper, we aim to tackle this problem by replacing the GAP layer with a new architecture called Wise-SrNet. It is inspired by the depthwise convolutional idea and is designed for processing spatial resolution while not increasing computational cost. We have evaluated our method using three different datasets they are Intel Image Classification Challenge, MIT Indoors Scenes, and a part of the ImageNet dataset. We investigated the implementation of our architecture on several models of the Inception, ResNet, and DenseNet families. Applying our architecture has revealed a significant effect on increasing convergence speed and accuracy. Our experiments on images with 224224 resolution increased the Top-1 accuracy between 2 to 8% on different datasets and models. Running our models on 512512 resolution images of the MIT Indoors Scenes dataset showed a notable result of improving the Top-1 accuracy within 3 to 26%. We will also demonstrate the GAP layer’s disadvantage when the input images are large and the number of classes is not few. In this circumstance, our proposed architecture can do a great help in enhancing classification results. The code is shared at https://github.com/mr7495/image-classification-spatial.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Code Availability
All the main codes of our paper have been shared online for public use at https://github.com/mr7495/image-classification-spatial
Notes
This dataset is shared at https://www.kaggle.com/mohammadrahimzadeh/imagenet-70classes
This dataset is shared at https://www.kaggle.com/puneet6060/intel-image-classification
This dataset is shared at https://www.kaggle.com/itsahmad/indoor-scenes-cvpr-2019
References
Intel image classification challenge. https://www.kaggle.com/puneet6060/intel-image-classification
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, and Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems, Software available from tensorflow.org
Card D, Zhang M, Smith NA (2019) Deep weighted averaging classifiers. In: Proceedings of the conference on fairness, accountability, and transparency
Chollet F (2017) Xception: deep learning with depthwise separable convolutions
Chollet F and Others. keras, 2015
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255
He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition
He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks
Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2018) Densely connected convolutional networks
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25:1097–1105
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400
Peeples J, Xu W, Zare A (2021) Histogram layers for texture analysis. IEEE Trans Artif Intell 3(4):541–552
Qiu S (2018) Global weighted average pooling bridges pixel-level localization and image-level classification
Quattoni A, Torralba A (2009) Recognizing indoor scenes. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 413–420
Rahimzadeh M, Mohammadi MR (2021) Roct-net: A new ensemble deep convolutional model with improved spatial resolution learning for detecting common diseases from retinal oct images. In: 2021 11th international conference on computer engineering and knowledge (ICCKE), IEEE, pp 85–91
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2019) Grad-cam: visual explanations from deep networks via gradient-based localization. Int J Comput Vision 128(2):336–359
Seong H, Hyun J, Kim E (2020) Fosnet: an end-to-end trainable deep neural network for scene recognition. IEEE Access 8:82066–82077
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition
Sobti P, Nayyar A, Nagrath P et al (2021) Ensemv3x: a novel ensembled deep learning architecture for multi-label scene classification. PeerJ Comput Sci 7:e557
Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-resnet and the impact of residual connections on learning
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Tan M, Le Q (2020) Efficientnet: Rethinking model scaling for convolutional neural networks
Zoph B, Le QV (2017) Neural architecture search with reinforcement learning
Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rahimzadeh, M., Parvin, S., Askari, A. et al. Wise-SrNet: a novel architecture for enhancing image classification by learning spatial resolution of feature maps. Pattern Anal Applic 27, 30 (2024). https://doi.org/10.1007/s10044-024-01211-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10044-024-01211-0