Skip to main content
SpringerLink
Log in

Integration of local and global features for image retargeting quality assessment

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper presents a new objective retargeting quality assessment method based on the integration of global and local features by a Gaussian regression model. In the proposed method, first, a graph-based visual saliency algorithm and deep learning model are employed to extract an importance map of the input image. At the same time, the SIFT-Flow method is used for estimating the displacement of different pixels in the retargeted image. Then, the importance map and the displacement results are used for computing four features, called local aspect ratio similarity, local geometric distortion, global geometric distortion, and global preserved information, from the images. Finally, a Gaussian process regression model is employed to integrate these features and compute the final criterion for retargeting quality assessment. The proposed method was tested on the images of the MIT-RetargetMe and CUHK datasets and the results demonstrated its excellent performance compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

To evaluate the proposed method, we have used two popular databases those links are below:MIT RetargetMe:https://people.csail.mit.edu/mrub/retargetme/CUHK:https://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/database.html

References

  1. Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. ACM Trans. Graph. 27 (2008)

  2. Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. SIGGRAPH 26 (2007)

  3. Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing 151–158 (2009)

  4. Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on 1, 1–6 (2007)

  5. Krähenbühl, P., Lang, M., Sorkine-Hornung, A., Gross, M.: A system for retargeting of streaming video. ACM Trans. Graph. 28 (2009)

  6. Wang, Y.-S., Tai, C.-L., Sorkine, O., Lee, T.-Y.: Optimized scale-and-stretch for image resizing. ACM Trans. Graph. (TOG) 27, 118 (2008)

    Article  Google Scholar 

  7. Rubinstein, M., Shamir, A., Avidan, S.: Multi-operator media retargeting. ACM Trans. Graph. 28, 1–11 (2009)

    Article  Google Scholar 

  8. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  9. Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1 – 8 (2008)

  10. Rubinstein, M., Gutiérrez, D., Sorkine, O., Shamir, A.: A comparative study of image retargeting. ACM Trans. Graph. 29, 160 (2010)

    Article  Google Scholar 

  11. Kasutani, E., Yamada, A.: The mpeg-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval 1, 674 – 677 vol.1 (2001)

  12. Manjunath, B., Ohm, J., Vasudevan, V., Yamada, A.: Color and texture descriptors. IEEE Trans. Circuits Syst. Video Technol. 11, 703–715 (2001)

    Article  Google Scholar 

  13. Liu, C., Yuen, J., Torralba, A.: Sift flow: Dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33, 978–94 (2011)

    Article  Google Scholar 

  14. Boltz, S., Nielsen, F., Soatto, S.: Earth mover distance on superpixels. Proceedings / ICIP ... International Conference on Image Processing 4597 – 4600 (2010)

  15. Hsu, C.-C., Lin, C.-W., Fang, Y., Lin, W.: Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J. Sel. Topics Signal Process. 8, 377–389 (2014)

    Article  Google Scholar 

  16. Fang, Y., et al.: Objective quality assessment for image retargeting based on structural similarity. IEEE J. Emerg. Sel. Topics Circuits Syst. 4, 95–105 (2014)

    Article  Google Scholar 

  17. Zhang, Y., Fang, Y., Lin, W., Zhang, X., Li, L.: Backward registration based aspect ratio similarity (ars) for image retargeting quality assessment. IEEE Trans. Image Process. 25, 1–1 (2016)

    Article  Google Scholar 

  18. Liang, Y., Liu, Y.-J., Gutiérrez, D.: Objective quality prediction of image retargeting algorithms. IEEE Trans. Visual. Comput. Graph. 23 (2016)

  19. Zhang, Y., Lin, W., Li, Q., Cheng, W., Zhang, X.: Multiple-level feature based measure for retargeted image quality. IEEE Trans. Image Process. PP, 1–1 (2017)

  20. Fu, Z., Shao, F., Qiuping, J., Fua, R., Ho, Y.-S.: Quality assessment of retargeted images using hand-crafted and deep-learned features. IEEE Access PP, 1–1 (2018)

  21. Zhang, Y., Ngan, K., Ma, L., Li, H.: Objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection. IEEE Trans. Image Process. PP, 1–1 (2017)

  22. Shao, F., Fu, Z., Qiuping, J., Jiang, G., Ho, Y.-S.: Transformation-aware similarity measurement for image retargeting quality assessment via bidirectional rewarping. IEEE Trans. Syst. Man Cybern. Syst. PP, 1–15 (2019)

  23. Absetan, A., Fathi, A.: Integration of deep learned and handcrafted features for image retargeting quality assessment. Cybern. Syst. 54, 673–696 (2023)

    Article  Google Scholar 

  24. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural Inform. Process. Syst 19, 545–552 (2006)

    Google Scholar 

  25. Cui, Y., Jiang, G., Yu, M., Chen, Y., Ho, Y.-S.: Stitched wide field of view light field image quality assessment: Benchmark database and objective metric. IEEE Trans. Multim. 1–15 (2023)

  26. Fang, Z., et al.: Blind 3d-synthesized image quality measurement by analysis of local and global statistical properties. IEEE Trans. Instrum. Meas. 72, 1–15 (2023)

    Google Scholar 

  27. Zheng, S., et al.: Conditional random fields as recurrent neural networks 1529–1537 (2015)

  28. Ma, L., Lin, W., Deng, C., Ngan, K.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J. Sel. Topics Signal Process. 6, 626–639 (2012)

    Article  Google Scholar 

  29. Dong, W., Zhou, N., Paul, J.-C., Zhang, X.: Optimized image resizing using seam carving and scaling. ACM Trans. Graph. 28 (2009)

  30. Karni, Z., Freedman, D., Gotsman, C.: Energy-based image deformation. Comput. Graph. Forum 28, 1257–1268 (2009)

    Article  Google Scholar 

  31. Lin, W., Kuo, C.-C.J.: Perceptual visual quality metrics: a survey. J. Visual Commun. Image Rep. 22, 297–312 (2011)

    Article  Google Scholar 

  32. Jiang, B., Yang, J., Meng, Q., Li, B., Lu, W.A.: deep evaluator for image retargeting quality by geometrical and contextual interaction. IEEE Trans. Cybern. PP, 1–13 (2018)

  33. Peng, Z., Qiuping, J., Shao, F., Gao, W., Lin, W.: Lggd+: Image retargeting quality assessment by measuring local and global geometric distortions. IEEE Trans. Circuits Syst. Video Technol. PP, 1–1 (2021)

  34. Sheikh, H., Sabir, M., Bovik, A.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. Publicat. IEEE Signal Process. Soc. 15, 3440–51 (2006)

    Article  Google Scholar 

  35. Kendall, M.A.: New measure of rank correlation. Biometrika 30 (1938)

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Author notes

  1. Ahmad Absetan have contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Engineering and Information Technology, Razi University, Bagh Abrisham, Kermanshah, Kermanshah, 6714414971, Iran

    Ahmad Absetan & Abdolhossein Fathi

Authors
  1. Ahmad Absetan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdolhossein Fathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdolhossein Fathi.

Ethics declarations

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Absetan, A., Fathi, A. Integration of local and global features for image retargeting quality assessment. SIViP 18, 3577–3586 (2024). https://doi.org/10.1007/s11760-024-03022-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03022-6

Keywords

Search

Navigation