Skip to main content
SpringerLink
Log in

Fast partition algorithm in depth map intra coding unit based on multi-deep convolution neural network

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The three-dimension high-efficiency video coding standard (3D-HEVC) finalized comes with a significant increase in complexity caused by the integration of depth map coding technology. This complexity is primarily triggered by the quad-tree partition of the Intra Coding Units (CU) in the depth map. A new technique utilizing deep learning is proposed, in this paper, to tackle the issue of excessive complexity aiming to predict efficiently the CU partition structure. The proposed method involves building a dataset of CU partition structure information for a depth map, creating a Multi-Deep Convolutional Neural Network (MD-CNN) model using this dataset, and then incorporating the model into the 3D-HEVC test platform. This approach reduces the 3D-HEVC video encoder complexity by 48.29% without affecting robustness, compression efficiency and video quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Tech, G., Chen, Y., Müller, K., Ohm, J.-R., Vetro, A., Wang, Y.-K.: Overview of the multiview and 3d extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016)

    Article  Google Scholar 

  2. Sullivan, G.J., Ohm, J.-R., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (hevc) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  3. Sanchez, G., Agostini, L., Marcon, C.: Algorithms for efficient and fast 3d-hevc depth map encoding (2020). https://doi.org/10.1007/978-3-030-25927-3

  4. Bakkouri, S., Elyousfi, A.: An adaptive cu size decision algorithm based on gradient boosting machines for 3d-hevc inter-coding. Multimed. Tools Appl., 1–19 (2023)

  5. Zhang, Z., Yu, L., Qian, J., Wang, H.: Learning-based fast depth inter coding for 3d-hevc via xgboost. In: 2022 Data Compression Conference (DCC), pp. 43–52. IEEE (2022)

  6. Song, W., Dai, P., Zhang, Q.: Content-adaptive mode decision for low complexity 3d-hevc. Multimedi. Tools Appl., 1–16 (2023)

  7. Lee, J.Y., Kang, M., Park, S.-h.: Fast depth intra mode decision using machine learning in 3d-hevc. Available at SSRN 4197680

  8. Chiang, J.-C., Peng, K.-K., Wu, C.-C., Deng, C.-Y., Lie, W.-N.: Fast intra mode decision and fast cu size decision for depth video coding in 3d-hevc. Signal Process.: Image Commun. 71, 13–23 (2019)

    Google Scholar 

  9. Zuo, J., Chen, J., Zeng, H., Cai, C., Ma, K.-K.: Bi-layer texture discriminant fast depth intra coding for 3d-hevc. IEEE Access 7, 34265–34274 (2019)

    Article  Google Scholar 

  10. Li, T., Wang, H., Chen, Y., Yu, L.: Fast depth intra coding based on spatial correlation and rate distortion cost in 3d-hevc. Signal Process.: Image Commun. 80, 115668 (2020)

    Google Scholar 

  11. Li, T., Yu, L., Wang, S., Wang, H.: Simplified depth intra coding based on texture feature and spatial correlation in 3d-hevc. In: 2018 Data Compression Conference, pp. 421–421. IEEE (2018)

  12. Wang, C., Feng, G., Cai, C., Han, X.: Fast cu size decision algorithm for depth map intra-coding in 3d-hevc. Commun. Technol. 50(4), 655–661 (2017)

    Google Scholar 

  13. Hamout, H., Elyousfi, A.: Fast 3d-hevc pu size decision algorithm for depth map intra-video coding. J. Real-Time Image Proc. 17(5), 1285–1299 (2020)

    Article  Google Scholar 

  14. Hamout, H., Elyousfi, A.: A computation complexity reduction of the size decision algorithm in 3d-HEVC depth map intracoding. Adv. Multimed. 2022, 3507201 (2022). https://doi.org/10.1155/2022/3507201

    Article  Google Scholar 

  15. Saldanha, M., Sanchez, G., Marcon, C., Agostini, L.: Fast 3d-hevc depth map encoding using machine learning. IEEE Trans. Circuits Syst. Video Technol. 30(3), 850–861 (2019)

    Article  Google Scholar 

  16. Saldanha, M., Sanchez, G., Marcon, C., Agostini, L.: Fast 3d-hevc depth maps intra-frame prediction using data mining. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1738–1742. IEEE (2018)

  17. Liu, C., Jia, K., Liu, P., Sun, Z.: Fast depth intra coding based on layer-classification and cnn for 3d-hevc. In: 2020 Data Compression Conference (DCC), pp. 381–381. IEEE (2020)

  18. Fu, C.-H., Chen, H., Chan, Y.-L., Tsang, S.-H., Hong, H., Zhu, X.: Fast depth intra coding based on decision tree in 3d-hevc. IEEE Access 7, 173138–173147 (2019)

    Article  Google Scholar 

  19. Liu, C., Jia, K., Liu, P.: Fast depth intra coding based on depth edge classification network in 3d-hevc. IEEE Trans. Broadcast. 68(1), 97–109 (2021)

    Article  MathSciNet  Google Scholar 

  20. Zhang, R., Jia, K., Yu, Y., Liu, P., Sun, Z.: Fast 3d-hevc inter coding using data mining and machine learning. IET Image Proc. 16(11), 3067–3084 (2022)

    Article  Google Scholar 

  21. Xu, M., Li, T., Wang, Z., Deng, X., Yang, R., Guan, Z.: Reducing complexity of hevc: a deep learning approach. IEEE Trans. Image Process. 27(10), 5044–5059 (2018)

    Article  ADS  MathSciNet  Google Scholar 

  22. Imen, W., Amna, M., Fatma, B., Ezahra, S.F., Masmoudi, N.: Fast hevc intra-cu decision partition algorithm with modified lenet-5 and alexnet. SIViP 16(7), 1811–1819 (2022)

    Article  Google Scholar 

  23. Li, T., Xu, M., Tang, R., Chen, Y., Xing, Q.: Deepqtmt: a deep learning approach for fast qtmt-based cu partition of intra-mode vvc. IEEE Trans. Image Process. 30, 5377–5390 (2021)

    Article  ADS  PubMed  Google Scholar 

  24. Amna, M., Imen, W., Fatma Ezahra, S.: Fast multi-type tree partitioning for versatile video coding using machine learning. SIViP 17(1), 67–74 (2023)

    Article  Google Scholar 

  25. Dang-Nguyen, D.-T., Pasquini, C., Conotter, V., Boato, G.: Raise: A raw images dataset for digital image forensics. In: Proceedings of the 6th ACM Multimedia Systems Conference, pp. 219–224 (2015)

  26. ITU/ISO/IEC: HEVC HM reference software. [Online; accessed 28-April-2023] (2017). https://vcgit.hhi.fraunhofer.de/jvet/HM/-/tree/HM-16.18?ref_type=tags

  27. ITU/ISO/IEC: 3D-HEVC HTM reference software. [Online; accessed 28-April-2023] (2017). https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/branches/HTM-16.3-fixes/

  28. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D.G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., Zheng, X.: Tensorflow: A system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265–283 (2016)

Download references

Author information

Authors and Affiliations

  1. Electronics and Microelectronics Laboratory, Faculty of Sciences of Monastir, Environment Street, Monastir, 5019, Tunisia

    Nacir Omran, Amna Maraoui & Belgacem Hamdi

  2. Electronic and Information Technology Laboratory, University of Sfax, Sfax, Tunisia

    Imen Werda

Authors
  1. Nacir Omran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amna Maraoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Imen Werda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Belgacem Hamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NO and AM wrote the manuscript and prepared figures. All authors reviewed the manuscript.

Corresponding author

Correspondence to Nacir Omran.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Omran, N., Maraoui, A., Werda, I. et al. Fast partition algorithm in depth map intra coding unit based on multi-deep convolution neural network. J Real-Time Image Proc 21, 23 (2024). https://doi.org/10.1007/s11554-023-01404-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01404-6

Keywords

Search

Navigation