Skip to main content
SpringerLink
Log in

DKH: a hybridized approach for image inpainting using Bayes probabilistic-based image fusion

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Image inpainting is the process of removing the unwanted objects from the image or it refers to the restoration of the original image. Despite the fact that there are various ways for image inpainting, these traditional approaches have some limitations in terms of data loss, which the proposed methodology should be able to address. This paper introduces a hybrid image inpainting method, termed DKH, which is the combination of deep learning, KNN, and biharmonic functions. Three phases make up the proposed DKH technique. The creation of the residual image, which takes place in the first phase, is accomplished using a Deep Convolutional Neural Network (Deep CNN) that was trained using the whale-monarch butterfly optimization algorithm. The second phase is the formation of patches and generation of the reconstructed image using the neighbour searching phenomenon named K-nearest neighbours (KNN), where the patch with the shortest distance is chosen during patch extraction using the Bhattacharya distance. In the third phase, the harmonic image is a reconstruction using biharmonal technique. Finally, using the Bayes probabilistic-based fusion method, the results of the three steps of image inpainting are combined. The performance of the image inpainting based on the proposed DKH is evaluated in terms of PSNR, SDME, SSIM, and accuracy. The developed image inpainting method achieves the PSNR of 35.63 dB, maximal SDME of 95.48 dB, the maximal SSIM of 0.960, and the maximal accuracy of 0.921 using Corel-10 k dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets analysed during the current study are available in “http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx”.

Abbreviations

KNN:

K-nearest neighbours

Deep CNN:

Deep convolutional neural network

PDE:

Partial differential equation

MARR:

Momentum adaptive and rank revealing

TSLRA:

Two-stage low-rank approximation

MRF:

Markov random field

TGV:

Total generalized variation

MC:

Matrix completion

FC:

Fully connected

SDME:

Sustainable decision making exercise

PSNR:

Peak signal-to-noise ratio

SSIM:

Structural similarity index measure

DWT:

Discrete wavelet transform

Content-based-CVA:

Content-based-cognitive visual attention

DNN:

Deep neural network

Whale-MBO:

Whale-monarch butterfly optimization

MBO:

Monarch butterfly optimization

WOA:

Whale optimization algorithm

References

  • Alexander, R., Alexey, S., Vladimir, I., Alexandr, A.K.: Deep convolutional neural networks for breast cancer histology image analysis. In: International conference image analysis and recognition ICIAR, pp. 737–744. Springer, Cham (2018)

    Google Scholar 

  • Brendt, W.: Inpainting by joint optimization of linear combinations of exemplars. IEEE Signal Process. Lett. 18(1), 75–78 (2011)

    Article  Google Scholar 

  • Chinmayee, H.R., Anupama, A., Bhagyashree, P., Patil, M.R.: Image Inpainting using exemplar based technique with improvised data term. In: 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS), pp. 162–166 (2018)

  • Christine, G., Meur, O.L.: Image inpainting. IEEE Signal Process. Mag. 31(1), 127–144 (2014)

    Article  Google Scholar 

  • Corel-10k and GHIM-10k Datasets Taken (2018) http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx. Accessed Aug 2018

  • Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  • Damelin, S.B., Hoang, N.S.: On surface completion and image inpainting by biharmonic functions: numerical aspects. Int. J. Math. Math. Sci. (2018). https://doi.org/10.1155/2018/3950312

    Article  MathSciNet  MATH  Google Scholar 

  • Desai, M., Ganatra, A.: Survey on gap filling in satellite images and inpainting algorithm. Int. J. Comput. Theory Eng. 4(3), 341–345 (2012)

    Article  Google Scholar 

  • Erkan, U., Serdar, E., Dang, N.H.T.: An iterative image inpainting method based on similarity of pixels values. In: Proceedings of 6th International Conference on Electrical and Electronics Engineering, pp. 107–111 (2019)

  • Fawzi, A., Samulowitz, H., Turaga, D. and Frossard, P., Image inpainting through neural networks hallucinations. In: Image, Video, and Multidimensional Signal Processing, pp. 1–5 (2016)

  • Ghorai, M., Mandal, S., Chanda, B.: A group-based image inpainting using patch refinement in MRF framework. IEEE Trans. Image Process. 27(2), 556–567 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  • Guo, Q., Gao, S., Zhang, X., Yin, Y., Zhang, C.: Patch-based image inpainting via two-stage low rank approximation. IEEE Trans. vis. Comput. Graph. 24(6), 2023–2036 (2018)

    Article  Google Scholar 

  • Janani, V.D., Haribaabu, V.: Terminating the spamming links and privacy guaranteed search logs. In: proceedings of 2013 International Conference on Information Communication and Embedded Systems (ICICES), pp. 394–397 (2013)

  • Jiahui, Y., Zhe, L., Jimei, Y., Xiaohui, S., Xin, L., Thomas, S.H.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505–5514 (2018)

  • Jiao, L., Hao, W., Haodi, W., Rongfang, B.: Multi-scale semantic image inpainting with residual learning and GAN. Neurocomputing 331, 199–212 (2019)

    Article  Google Scholar 

  • Jin, D., Bai, X.: Patch-sparsity-based image inpainting through facet deduced directional derivative. IEEE Trans. Circ. Syst. Video Technol. (2018). https://doi.org/10.1109/TCSVT.2018.2839351

    Article  Google Scholar 

  • Jixiang, C., Zhidan, L.: Markov Random Field-based image inpainting with direction structure distribution analysis for maintaining structure coherence. Signal Process. 154, 182–197 (2019)

    Article  Google Scholar 

  • Komal, S.M., Vaidya, M.B.: Image in painting techniques: a survey. IOSR J. Comput. Eng. (IOSRJCE) 5(4), 45–49 (2012)

    Article  Google Scholar 

  • Mahesh, M., Bhanodia, P.: Image inpainting techniques for removal of object. In: In International Conference on Information Communication and Embedded Systems, pp. 1–4 (2014)

  • Mirjalili, S., Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95, 51–67 (2016)

    Article  Google Scholar 

  • Newson, A., Almansa, A., Matthieu, F., Yann, G., Patrick, P.: Video inpainting of complex scenes. SIAM J. Imaging Sci. 7(4), 1993–2019 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Pal, G., Sharma, V.: Survey on different techniques in image inpainting. Int. J. Inf. Mov. 2(8), 119–127 (2017)

    Google Scholar 

  • Pandya, N., Bhailal, L.: A survey on image inpainting techniques. Int. J. Curr. Eng. Technol. 2(2), 1–18 (2013)

    Google Scholar 

  • Thanh, D.N.H., Surya Prasath, V.B., Sergey, D., Le Minh, H.: An adaptive image inpainting method based on euler’s elastica with adaptive parameters estimation and the discrete gradient method. Signal Process. 178, 107797 (2021)

    Article  Google Scholar 

  • Vahid, K.A., Farzin, Y.: Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis. Multimedia Tools Appl. 76(5), 7213–7234 (2017)

    Article  Google Scholar 

  • Wali, S., Zhang, H., Chang, H., Wu, C.: A new adaptive boosting total generalized variation (TGV) technique for image denoising and inpainting. J. vis. Commun. Image Represent. 59, 39–51 (2019)

    Article  Google Scholar 

  • Wang, G.G., Deb, S., Zhao, X., Cui, Z.: A new monarch butterfly optimization with an improved crossover operator. Oper. Res. 18, 731–755 (2016)

    Article  Google Scholar 

  • Yang, S., Liu, J., Fang, Y., Guo, Z.: Structure-guided image inpainting using homography transformation. IEEE Trans. Multimedia 20(12), 3252–3265 (2018)

    Article  Google Scholar 

  • Yu, Y., Fangneng, Z., Shijian, L., Jianxiong, P., Feiying, M., Xuansong, X., Chunyan, M.: WaveFill: a wavelet-based generation network for image inpainting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14114–14123 (2021)

  • Zheng, J., Qin, M., Yu, H., Wang, W.: An efficient truncated nuclear norm constrained matrix completion for image inpainting. In: Computer Graphics, pp. 97–106 (2018)

  • Zongben, X., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Tatyasaheb Kore Institute of Engineering and Technology, Kolhapur, Maharashtra, India

    Manjunath R. Hudagi

  2. Department of Computer Science and Engineering, Poojya Doddappa Appa College of Engineering, Kalaburagi, Karnataka, India

    Shridevi Soma

  3. Electronics & Telematics Engineering Dept., G. Narayanamma Institute of Technology and Science, Shaikpet, Hyderabad, India

    Rajkumar L. Biradar

Authors
  1. Manjunath R. Hudagi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shridevi Soma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajkumar L. Biradar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manjunath R. Hudagi.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hudagi, M.R., Soma, S. & Biradar, R.L. DKH: a hybridized approach for image inpainting using Bayes probabilistic-based image fusion. Int J Intell Robot Appl 7, 149–163 (2023). https://doi.org/10.1007/s41315-022-00267-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-022-00267-7

Keywords

Search

Navigation