Skip to main content
SpringerLink
Log in

A detection method for occluded and overlapped apples under close-range targets

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Accurate and rapid identification and location of apples contributes to speeding up automation harvesting. However, in unstructured orchard environments, it is common for apples to be overlapped and occluded by branches and leaves, which interferes with apple identification and localization. In order to quickly reconstruct the fruits under overlapping and occlusion conditions, an adaptive radius selection strategy based on random sample consensus algorithm (ARSS-RANSAC) was proposed. Firstly, the edge of apple in the image was obtained by using image preprocessing method. Secondly, an adaptive radius selection strategy was proposed, which is based on fruit shape characteristics. The fruit initial radius was obtained through horizontal or vertical scanning. Then, combined with RANSAC algorithm to select effective contour points by the determined radius, and the circle center coordinates were obtained. Finally, fitting the circle according to the selected valid contour and achieving the recognition and localization of overlapped and occluded apples. 175 apple images with different overlaps and branches and leaves occlusion were applied to verify the effectiveness of algorithm. The evaluation indicators of overlap rate, average false-positive rate, average false-negative rate, and average segmentation error of ARSS-RANSAC were improved compared with the classical Hough transform method. The detection time of a single image was less than 50 ms, which can meet requirements of real-time target detection. The experimental results show that the ARSS-RANSAC algorithm can quickly and accurately identify and locate occluded and overlapped apples and is expected to be applied to harvesting robots of apple and other round fruits.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Starowicz M, Achrem-Achremowicz B, Piskula MK, Zielinski H (2020) Phenolic compounds from apples: reviewing their occurrence absorption, bioavailability, processing, and antioxidant activity—a review. Polish J Food Nutr Sci 70:321–336

    Article  CAS  Google Scholar 

  2. Zhang Z, Wang YJ, Zhang ZH, Li DP, Wu ZZ, Bai R, Meng GF (2019) Ergonomic and efficiency analysis of conventional apple harvest process. Int J Agric Biol Eng 12:210–217

    Google Scholar 

  3. Wang D, Song H, He D (2017) Research advance on vision system of apple picking robot. Trans Chin Soc Agric Eng 33:59–69

    Google Scholar 

  4. Bac CW, van Henten EJ, Hemming J, Edan Y (2014) Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. J Field Robot 31:888–911. https://doi.org/10.1002/rob.21525

    Article  Google Scholar 

  5. Xu Y, Li Y, Song H, He D (2015) Segmentation method of overlapped double apples based on Snake model and corner detectors. Trans Chin Soc Agric Eng 31:196–203

    Google Scholar 

  6. Wang D, Xu Y, Song H, He D, Zhang H (2015) Fusion of K-means and Ncut algorithm to realize segmentation and reconstruction of two overlapped apples without blocking by branches and leaves. Trans Chin Soc Agric Eng 31:227–234

    CAS  Google Scholar 

  7. Yang CH, Hu YC, Huang L, Liu S, Liu YP (2017) IEEE, Overlapped fruit recognition for citrus harvesting robot in natural scenes. In: 2017 2nd international conference on robotics and automation engineering (ICRAE), pp 398–402

  8. Fu LS, Tola E, Al-Mallahi A, Li R, Cui YJ (2019) A novel image processing algorithm to separate linearly clustered kiwifruits. Biosyst Eng 183:184–195. https://doi.org/10.1016/j.biosystemseng.2019.04.024

    Article  Google Scholar 

  9. Lv JD, Zhao DA, Ji W, Ding SH (2016) Recognition of apple fruit in natural environment. Optik 127:1354–1362. https://doi.org/10.1016/j.ijleo.2015.10.177

    Article  Google Scholar 

  10. Lin GC, Tang YC, Zou XJ, Cheng JB, Xiong JT (2020) Fruit detection in natural environment using partial shape matching and probabilistic Hough transform. Precis Agric 21:160–177. https://doi.org/10.1007/s11119-019-09662-w

    Article  Google Scholar 

  11. Jiao YH, Luo R, Li QW, Deng XB, Yin X, Ruan CZ, Jia WK (2020) Detection and localization of overlapped fruits application in an apple harvesting robot. Electronics. https://doi.org/10.3390/electronics9061023

    Article  Google Scholar 

  12. Chen JQ, Wu JH, Wang ZK, Qiang H, Cai GW, Tan CZ, Zhao CY (2021) Detecting ripe fruits under natural occlusion and illumination conditions. Comput Electron Agr. https://doi.org/10.1016/j.compag.2021.106450

    Article  Google Scholar 

  13. Behera SK, Mishra N, Sethy PK, Rath AK (2018) IEEE, On-Tree detection and counting of apple using color thresholding and CHT. In: 7th IEEE international conference on communication and signal processing (IEEE ICCSP), pp 224–228

  14. Gander W, Golub GH, Strebel R (1994) Least-squares fitting of circles and ellipses. BIT 34:558–578. https://doi.org/10.1007/BF01934268

    Article  MathSciNet  Google Scholar 

  15. Moura L, Kitney R (1991) A direct method for least-squares circle fitting. Comput Phys Commun 64:57–63. https://doi.org/10.1016/0010-4655(91)90049-Q

    Article  ADS  MathSciNet  Google Scholar 

  16. Chernov N, Lesort C (2005) Least squares fitting of circles. J Math Imaging Vis 23:239–252. https://doi.org/10.1007/s10851-005-0482-8

    Article  MathSciNet  Google Scholar 

  17. Duda RO, Hart PE (1972) Use of hough transformation to detect lines and curves in pictures. Commun ACM 15:11–000. https://doi.org/10.1145/361237.361242

    Article  Google Scholar 

  18. Yuen HK, Princen J, Illingworth J, Kittler J (1990) Comparative-study of hough transform methods for circle finding. Image Vision Comput 8:71–77. https://doi.org/10.1016/0262-8856(90)90059-E

    Article  Google Scholar 

  19. Xu L, Oja E, Kultanen P (1990) A new curve detection method - randomized hough transform (rht), Pattern Recogn. Lett 11:331–338. https://doi.org/10.1016/0167-8655(90)90042-Z

    Article  Google Scholar 

  20. Ukwuoma C, Zhiguang Q, Bin Heyat MB, Ali L, Almaspoor Z, Monday H (2022) Recent advancements in fruit detection and classification using deep learning techniques. Math Probl Eng. https://doi.org/10.1155/2022/9210947

    Article  Google Scholar 

  21. Tang YC, Chen MY, Wang CL, Luo LF, Li JH, Lian GP, Zou XJ (2020) Recognition and localization methods for vision-based fruit picking robots: a review. Front Plant Sci. https://doi.org/10.3389/fpls.2020.00510

    Article  PubMed  PubMed Central  Google Scholar 

  22. Koirala A, Walsh KB, Wang ZL, McCarthy C (2019) Deep learning—method overview and review of use for fruit detection and yield estimation. Comput Electron Agr 162:219–234. https://doi.org/10.1016/j.compag.2019.04.017

    Article  Google Scholar 

  23. Prakash AJ, Prakasam P (2023) An intelligent fruits classification in precision agriculture using bilinear pooling convolutional neural networks. Visual Comput. https://doi.org/10.1007/s00371-022-02443-z

    Article  Google Scholar 

  24. Yu Y, Zhang KL, Yang L, Zhang DX (2019) Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput Electron Agr. https://doi.org/10.1016/j.compag.2019.06.001

    Article  Google Scholar 

  25. Tian YN, Yang GD, Wang Z, Wang H, Li E, Liang ZZ (2019) Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput Electron Agr 157:417–426. https://doi.org/10.1016/j.compag.2019.01.012

    Article  ADS  Google Scholar 

  26. Wang DD, He DJ (2021) Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning. Biosyst Eng 210:271–281. https://doi.org/10.1016/j.biosystemseng.2021.08.015

    Article  CAS  Google Scholar 

  27. Junos MH, Khairuddin A, Thannirmalai S, Dahari M (2022) Automatic detection of oil palm fruits from UAV images using an improved YOLO model. Visual Comput 38:2341–2355. https://doi.org/10.1007/s00371-021-02116-3

    Article  Google Scholar 

  28. Zheng C, Chen PF, Pang J, Yang XF, Chen CX, Tu SQ, Xue YJ (2021) A mango picking vision algorithm on instance segmentation and key point detection from RGB images in an open orchard. Biosyst Eng 206:32–54. https://doi.org/10.1016/j.biosystemseng.2021.03.012

    Article  CAS  Google Scholar 

  29. Yang F, Tang W, Liang Y (2018) A novel track initialization algorithm based on random sample consensus in dense clutter. Int J Adv Robot Syst. https://doi.org/10.1177/1729881418812632

    Article  Google Scholar 

  30. Dai W et al (2021) Improved random sampling consensus algorithm using local pixel matching. J National Univ Defense Technol 43(4):38–43

    Google Scholar 

  31. Wu Y et al (2015) A novel point-matching algorithm based on fast sample consensus for image registration. IEEE Geosci Remote Sens Lett 12(1):43–47. https://doi.org/10.1109/LGRS.2014.2325970

    Article  ADS  Google Scholar 

  32. Fischler MA, Bolles RC (1981) Random sample consensus—a paradigm for model-fitting with applications to image-analysis and automated cartography. Commun ACM 24:381–395. https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  33. Torr P, Zisserman A (2000) A new robust estimator with application to estimating image geometry. Comput Vis Image Und 78:138–156. https://doi.org/10.1006/cviu.1999.0832

    Article  Google Scholar 

  34. Otsu N (1979) Threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62–66. https://doi.org/10.1109/TSMC.1979.4310076

    Article  Google Scholar 

Download references

Acknowledgements

We thank Forestry and Fruit Tree Research Institute of Beijing Academy of Agriculture and Forestry Sciences for their help in image acquisition. The research was supported by the National Natural Science Foundation of China (Grant No. 31772064).

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China

    Yuhui Yuan, Hubin Liu, Zengrong Yang, Jianhua Zheng, Junhui Li & Longlian Zhao

  2. Key Laboratory of Modern Precision Agriculture System Integration Research, Ministry of Education, Beijing, 100083, China

    Yuhui Yuan, Hubin Liu, Zengrong Yang, Jianhua Zheng, Junhui Li & Longlian Zhao

Authors
  1. Yuhui Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hubin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zengrong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhua Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junhui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Longlian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Longlian Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, Y., Liu, H., Yang, Z. et al. A detection method for occluded and overlapped apples under close-range targets. Pattern Anal Applic 27, 12 (2024). https://doi.org/10.1007/s10044-024-01222-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01222-x

Keywords

Search

Navigation