Skip to main content
SpringerLink
Log in

Learning Local Contrast for Crisp Edge Detection

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In recent years, the accuracy of edge detection on several benchmarks has been significantly improved by deep learning based methods. However, the prediction of deep neural networks is usually blurry and needs further post-processing including non-maximum suppression and morphological thinning. In this paper, we demonstrate that the blurry effect arises from the binary cross-entropy loss, and crisp edges could be obtained directly from deep convolutional neural networks. We propose to learn edge maps as the representation of local contrast with a novel local contrast loss. The local contrast is optimized in a stochastic way to focus on specific edge directions. Experiments show that the edge detection network trained with local contrast loss achieves a high accuracy comparable to previous methods and dramatically improves the crispness. We also present several applications of the crisp edges, including image completion, image retrieval, sketch generation, and video stylization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Kittler J. On the accuracy of the sobel edge detector. Image and Vision Computing, 1983, 1(1): 37–42. https://doi.org/10.1016/0262-8856(83)90006-9.

    Article  Google Scholar 

  2. Canny J. A computational approach to edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 1986, PAMI-8(6): 679–698. https://doi.org/10.1109/TPAMI.1986.4767851.

  3. Konishi S, Yuille A L, Coughlan J M, Zhu S C. Statistical edge detection: Learning and evaluating edge cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 2003, 25(1): 57–74. https://doi.org/10.1109/TPAMI.2003.1159946.

  4. Martin D R, Fowlkes C C, Malik J. Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 2004, 26(5): 530–549. https://doi.org/10.1109/TPAMI.2004.1273918.

  5. Paris S. Edge-preserving smoothing and mean-shift segmentation of video streams. In Proc. the 10th European Conference on Computer Vision: Part II, Oct. 2008, pp.460–473. https://doi.org/10.1007/978-3-540-88688-4_34.

  6. Ren X F, Bo L F. Discriminatively trained sparse code gradients for contour detection. In Proc. the 25th International Conference on Neural Information Processing Systems, Dec. 2012, pp.584–592. https://doi.org/10.5555/2999134.2999200.

  7. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arxiv: 1409.1556, 2015. https://doi.org/10.48550/arXiv.1409.1556.

  8. He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.770–778. https://doi.org/10.1109/CVPR.2016.90.

  9. Girshick R. Fast R-CNN. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.1440–1448. https://doi.org/10.1109/ICCV.2015.169.

  10. Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.

  11. Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Analysis and Machine Intelligence, 2018, 40(4): 834–848. https://doi.org/10.1109/TPAMI.2017.2699184.

  12. He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In Proc. the 2017 IEEE International Conference on Computer Vision, Oct. 2017, pp.2980–2988. https://doi.org/10.1109/ICCV.2017.322.

  13. Xie S N, Tu Z W. Holistically-nested edge detection. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.1395–1403. https://doi.org/10.1109/ICCV.2015.164.

  14. Maninis K K, Pont-Tuset J, Arbeláez P, Van Gool L. Convolutional oriented boundaries. In Proc. the 14th European Conference on Computer Vision, Sept. 2016, pp.580–596. https://doi.org/10.1007/978-3-319-46448-0_35.

  15. Liu Y, Cheng M M, Hu X W, Wang K, Bai X. Richer convolutional features for edge detection. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.5872–5881. https://doi.org/10.1109/CVPR.2017.622.

  16. He J Z, Zhang S L, Yang M, Shan Y H, Huang T J. Bi-directional cascade network for perceptual edge detection. In Proc. the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2019, pp.3823–3832. https://doi.org/10.1109/CVPR.2019.00395.

  17. Arbelaez P, Maire M, Fowlkes C, Malik J. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence, 2011, 33(5): 898–916. https://doi.org/10.1109/TPAMI.2010.161.

  18. Silberman N, Hoiem D, Kohli P, Fergus R. Indoor segmentation and support inference from RGBD images. In Proc. the 12th European Conference on Computer Vision, Oct. 2012, pp.746–760. https://doi.org/10.1007/978-3-642-33715-4_54.

  19. Mély D A, Kim J, McGill M, Guo Y L, Serre T. A systematic comparison between visual cues for boundary detection. Vision Research, 2016, 120: 93–107. https://doi.org/10.1016/j.visres.2015.11.007.

    Article  Google Scholar 

  20. Huan L X, Xue N, Zheng X W, He W, Gong J Y, Xia G S. Unmixing convolutional features for crisp edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 2022, 44(10): 6602–6609. https://doi.org/10.1109/TPAMI.2021.3084197.

  21. Wang Y P, Zhao X, Huang K Q. Deep crisp boundaries. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.1724–1732. https://doi.org/10.1109/CVPR.2017.187.

  22. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Proc. the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015, pp.234–241. https://doi.org/10.1007/978-3-319-24574-4_28.

  23. Deng R X, Shen C H, Liu S J, Wang H B, Liu X R. Learning to predict crisp boundaries. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.570–586. https://doi.org/10.1007/978-3-030-01231-1_35.

  24. Wang M, Fang X N, Yang G W, Shamir A, Hu S M. Prominent structures for video analysis and editing. IEEE Trans. Visualization and Computer Graphics, 2021, 27(7): 3305–3317. https://doi.org/10.1109/TVCG.2020.2970045.

    Article  Google Scholar 

  25. Nazeri K, Ng E, Joseph T, Qureshi F Z, Ebrahimi M. EdgeConnect: Generative image inpainting with adversarial edge learning. arXiv: 1901.00212, 2019. https://arxiv.org/abs/1901.00212, May 2023.

  26. Lu P, Huang G, Lin H Y, Yang W M, Guo G D, Fu Y W. Domain-aware se network for sketch-based image retrieval with multiplicative Euclidean margin softmax. In Proc. the 29th ACM International Conference on Multimedia, Oct. 2021, pp.3418–3426. https://doi.org/10.1145/3474085.3475499.

  27. Li M T, Lin Z, Mech R, Yumer E, Ramanan D. Photosketching: Inferring contour drawings from images. In Proc. the 2019 IEEE Winter Conference on Applications of Computer Vision, Jan. 2019, pp.1403–1412. https://doi.org/10.1109/WACV.2019.00154.

  28. Winnemöller H, Olsen S C, Gooch B. Real-time video abstraction. ACM Trans. Graphics, 2006, 25(3): 1221–1226. https://doi.org/10.1145/1141911.1142018.

    Article  Google Scholar 

  29. Torre V, Poggio T A. On edge detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 1986, PAMI- 8(2): 147–163. https://doi.org/10.1109/TPAMI.1986.4767769.

  30. Deng J, Dong W, Socher R, Li L J, Li K, Li F F. ImageNet: A large-scale hierarchical image database. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2009, pp.248–255. https://doi.org/10.1109/CVPR.2009.5206848.

  31. Liu J J, Hou Q B, Cheng M M. Dynamic feature integration for simultaneous detection of salient object, edge, and skeleton. IEEE Trans. Image Processing, 2020, 29: 8652–8667. https://doi.org/10.1109/TIP.2020.3017352.

    Article  MATH  Google Scholar 

  32. Soria X, Riba E, Sappa A. Dense extreme inception network: Towards a robust CNN model for edge detection. In Proc. the 2020 IEEE Winter Conference on Applications of Computer Vision, Mar. 2020, pp.1923–1932. https://doi.org/10.1109/WACV45572.2020.9093290.

  33. Mittal M, Verma A, Kaur I, Kaur B, Sharma M, Goyal L M, Roy S, Kim T H. An efficient edge detection approach to provide better edge connectivity for image analysis. IEEE Access, 2019, 7: 33240–33255. https://doi.org/10.1109/ACCESS.2019.2902579.

    Article  Google Scholar 

  34. Hallman S, Fowlkes C C. Oriented edge forests for boundary detection. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.1732–1740. https://doi.org/10.1109/CVPR.2015.7298782.

  35. Ganin Y, Lempitsky V. N4-fields: Neural network nearest neighbor fields for image transforms. In Proc. the 2014 Asian Conference on Computer Vision, Apr. 2014, pp.536–551. https://doi.org/10.1007/978-3-319-16808-1_36.

  36. Shen W, Wang X G, Wang Y, Bai X, Zhang Z J. Deep-contour: A deep convolutional feature learned by positive-sharing loss for contour detection. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.3982–3991. https://doi.org/10.1109/CVPR.2015.7299024.

  37. Bertasius G, Shi J B, Torresani L. High-for-low and low- for-high: Efficient boundary detection from deep object features and its applications to high-level vision. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.504–512. https://doi.org/10.1109/ICCV.2015.65.

  38. Yang J M, Price B, Cohen S, Lee H, Yang M H. Object contour detection with a fully convolutional encoder-decoder network. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.193–202. https://doi.org/10.1109/CVPR.2016.28.

  39. Kokkinos I. Pushing the boundaries of boundary detection using deep learning. arXiv: 1511.07386, 2015. https://arxiv.org/pdf/1511.07386.pdf, May 2023.

  40. Xu D, Ouyang W L, Alameda-Pineda X, Ricci E, Wang X G, Sebe N. Learning deep structured multi-scale features using attention-gated CRFs for contour prediction. In Proc. the 31st International Conference on Neural Information Processing Systems, Dec. 2017, pp.3964–3973. https://doi.org/10.5555/3294996.3295152.

  41. Liao Y, Fu S P, Lu X Q, Zhang C C, Tang Z. Deep-learning-based object-level contour detection with CCG and CRF optimization. In Proc. the 2017 IEEE International Conference on Multimedia and Expo, Jul. 2017, pp.859– 864. https://doi.org/10.1109/ICME.2017.8019358.

  42. Liu Z W, Luo P, Wang X G, Tang X O. Deep learning face attributes in the wild. In Proc. the 2015 IEEE International Conference on Computer Vision, Dec. 2015, pp.3730–3738. https://doi.org/10.1109/ICCV.2015.425.

  43. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Processing, 2004, 13(4): 600–612. https://doi.org/10.1109/TIP.2003.819861.

    Article  Google Scholar 

  44. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Proc. the 31st International Conference on Neural Information Processing Systems, Dec. 2017, pp.6629–6640. https://doi.org/10.5555/3295222.3295408.

  45. Eitz M, Hays J, Alexa M. How do humans sketch objects? ACM Trans. Graphics, 2012, 31(4): Article No. 44. https://doi.org/10.1145/2185520.2185540.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Xiao-Nan Fang & Song-Hai Zhang

Authors
  1. Xiao-Nan Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song-Hai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Song-Hai Zhang.

Supplementary Information

ESM 1

(PDF 122 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, XN., Zhang, SH. Learning Local Contrast for Crisp Edge Detection. J. Comput. Sci. Technol. 38, 554–566 (2023). https://doi.org/10.1007/s11390-023-3101-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-023-3101-5

Keywords

Search

Navigation