Skip to main content
SpringerLink
Log in

An efficient center-based method for real-time pig posture recognition and tracking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Detecting, tracking, and recognizing the posture of individual pigs are the primary computer vision tasks of many camera-based decision support tools for precise livestock monitoring. Recently, the use of deep learning approaches in computer vision, particularly convolutional neural networks (CNNs), has led to performance never before possible. However, the heavy constraints of pig environments such as instinctive grouping and similar animal appearance limit the effectiveness of state-of-the-art approaches popular in other application fields. To tackle these problems, we propose a fully anchor-free center-based CNN framework that detects pigs, classifies their postures, and tracks them throughout an image sequence. Input images are first fed into two sub-networks that detect pig posture and generate a global appearance map from which local appearance vectors corresponding to the detected pig centers can be extracted. Next, an assignment strategy uses spatial and appearance metrics to associate each pig detected in the frame with one of the tracked pig trajectories in a video sequence. As shown by the experiments, our real-time method significantly outperforms state-of-the-art approaches for pig detection, posture recognition, and tracking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

Confidential data.

Code availability

Confidential.

References

  1. Godfray HCJ, Aveyard P, Garnett T, Hall JW, Key TJ, Lorimer J, Pierrehumbert RT, Scarborough P, Springmann M, Jebb SA (2018) Meat consumption, health, and the environment. Science 361(e6399):5324

    Article  Google Scholar 

  2. Vranken E, Berckmans D (2017) Precision livestock farming for pigs. Anim Front 7(e1):32–37

    Article  Google Scholar 

  3. Yang Q, Xiao D (2020) A review of video-based pig behavior recognition. Appl Anim Behav Sci 233:105146

    Article  Google Scholar 

  4. Peden RS, Turner SP, Boyle LA, Camerlink I (2018) The translation of animal welfare research into practice: The case of mixing aggression between pigs. Appl Anim Behav Sci 204:1–9

    Article  Google Scholar 

  5. Ahmed S, Mun H-S, Yoe H, Yang C-J (2015) Monitoring of behavior using a video-recording system for recognition of salmonella infection in experimentally infected growing pigs. Animal 9(e1):115–121

    Article  Google Scholar 

  6. Zhang Z, Wang H, Liu T, Wang Y, Zhang H, Yuan F, Yang X, Xu S, Meng Y (2021) Accurate detection method of pig’s temperature based on non-point source thermal infrared image. CAAI Transactions on Intelligence Technology 6(e3):312–323

    Article  Google Scholar 

  7. Li J, Green-Miller AR, Hu X, Lucic A, Mohan MM, Dilger RN, Condotta IC, Aldridge B, Hart JM, Ahuja N (2022) Barriers to computer vision applications in pig production facilities. Comput Electron Agric 200:107227

    Article  Google Scholar 

  8. Cang Y, He H, Qiao Y (2019) An intelligent pig weights estimate method based on deep learning in sow stall environments. IEEE Access 7:164867–164875

    Article  Google Scholar 

  9. Yang Q, Xiao D, Lin S (2018) Feeding behavior recognition for group-housed pigs with the faster r-cnn. Comput Electron Agric 155:453–460

    Article  Google Scholar 

  10. Hansen MF, Smith ML, Smith LN, Salter MG, Baxter EM, Farish M, Grieve B (2018) Towards on-farm pig face recognition using convolutional neural networks. Comput Ind 98:145–152

    Article  Google Scholar 

  11. Zande L, Guzhva O, Rodenburg TB et al (2021) Individual detection and tracking of group housed pigs in their home pen using computer vision. Front Anim Sci 2:10

    Google Scholar 

  12. Zhang L, Gray H, Ye X, Collins L, Allinson N (2019) Automatic individual pig detection and tracking in pig farms. Sensors 19(e5):1188

    Article  Google Scholar 

  13. Cowton J, Kyriazakis I, Bacardit J (2019) Automated individual pig localisation, tracking and behaviour metric extraction using deep learning. IEEE Access 7:108049–108060

    Article  Google Scholar 

  14. Bergamini L, Pini S, Simoni A et al (2021) Extracting accurate long-term behavior changes from a large pig dataset. In: 16th International joint conference on computer vision, imaging and computer graphics theory and applications, VISIGRAPP 2021, pp 524–533. SciTePress

  15. Shao H, Pu J, Mu J (2021) Pig-posture recognition based on computer vision: Dataset and exploration. Animals 11(e5):1295

    Article  Google Scholar 

  16. Riekert M, Klein A, Adrion F, Hoffmann C, Gallmann E (2020) Automatically detecting pig position and posture by 2d camera imaging and deep learning. Comput Electron Agric 174:105391

    Article  Google Scholar 

  17. Zou Z, Chen K, Shi Z, Guo Y, Ye J (2023) Object detection in 20 years: A survey. Proc IEEE 111(e3):257–276

    Article  Google Scholar 

  18. Mattina M, Benzinou A, Nasreddine K, Richard F (2022) An efficient anchor-free method for pig detection. IET Image Proc 17(e2):613–626

    Google Scholar 

  19. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767

  20. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inform Process Syst 28

  21. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: Computer vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, Proceedings, Part I 14, Springer, pp 21–37. Accessed 11–14 Oct 2016

  22. Dai J, Li Y, He K, Sun J (2016) R-fcn: Object detection via region-based fully convolutional networks. Adv Neural Inform Process Syst29

  23. Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-nms–improving object detection with one line of code. In: Proceedings of the IEEE international conference on computer vision, pp 5561–5569

  24. D’Eath RB, Jack M, Futro A, Talbot D, Zhu Q, Barclay D, Baxter EM (2018) Automatic early warning of tail biting in pigs: 3d cameras can detect lowered tail posture before an outbreak. PloS one 13(e4):0194524

  25. Chen C, Zhu W, Ma C, Guo Y, Huang W, Ruan C (2017) Image motion feature extraction for recognition of aggressive behaviors among group-housed pigs. Comput Electron Agri 142:380–387

  26. Viazzi S, Ismayilova G, Oczak M, Sonoda LT, Fels M, Guarino M, Vranken E, Hartung J, Bahr C, Berckmans D (2014) Image feature extraction for classification of aggressive interactions among pigs. Comput Electron Agric 104:57–62

  27. Nasirahmadi A, Hensel O, Edwards S, Sturm B (2017) A new approach for categorizing pig lying behaviour based on a delaunay triangulation method. Animal 11(e1):131–139

  28. Zhang Y, Wang C, Wang X, Zeng W, Liu W (2021) Fairmot: On the fairness of detection and re-identification in multiple object tracking. Int J Comput Vision 129(e11):3069–3087

    Article  Google Scholar 

  29. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International conference on image processing (ICIP), IEEE, pp 3645–3649

  30. Voigtlaender P, Krause M, Osep A, Luiten J, Sekar BBG, Geiger A, Leibe B (2019) Mots: Multi-object tracking and segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7942–7951

  31. Dang TL, Nguyen GT, Cao T (2020) Object tracking using improved deep sort yolov3 architecture. ICIC Express Letters 14(e10):961–969

    Google Scholar 

  32. Guo QY, Orsini C JE, de Vlieg J, Bijma P, de With PHN (2023) Enhanced camera-based individual pig detection and tracking for smart pig farms. Comput Electron Agric 211:108009

  33. Tu S, Zeng Q, Liang Y, Liu X, Huang L, Weng S, Huang Q (2022) Automated behavior recognition and tracking of group-housed pigs with an improved deepsort method. Agriculture 12(11)

  34. Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. In: 2016 IEEE International conference on image processing (ICIP), IEEE, pp 3464–3468

  35. Kuhn HW (1955) The hungarian method for the assignment problem. Naval Research Logistics Quarterly 2(e1–2):83–97

    Article  MathSciNet  Google Scholar 

  36. Lu Z, Rathod V, Votel R, Huang J (2020) Retinatrack: Online single stage joint detection and tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14668–14678

  37. Pang J, Qiu L, Li X, Chen H, Li Q, Darrell T, Yu F (2021) Quasi-dense similarity learning for multiple object tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 164–173

  38. Gan H, Ou M, Zhao F, Xu C, Li S, Chen C, Xue Y (2021) Automated piglet tracking using a single convolutional neural network. Biosys Eng 205:48–63

    Article  Google Scholar 

  39. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  40. Ciaparrone G, Sánchez FL, Tabik S, Troiano L, Tagliaferri R, Herrera F (2020) Deep learning in video multi-object tracking: A survey. Neurocomputing 381:61–88

    Article  Google Scholar 

  41. Jiang P, Ergu D, Liu F, Cai Y, Ma B (2022) A review of yolo algorithm developments. Procedia Computer Science 199:1066–1073

    Article  Google Scholar 

  42. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: European conference on computer vision, Springer, pp 213–229

  43. Zhu X, Su W, Lu L, Li B, Wang X, Dai J (2020) Deformable detr: Deformable transformers for end-to-end object detection. In: International conference on learning representations

  44. Sun P, Zhang R, Jiang Y, Kong T, Xu C, Zhan W, Tomizuka M, Li L, Yuan Z, Wang C et al (2021) Sparse r-cnn: End-to-end object detection with learnable proposals. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14454–14463

  45. Ge Z, Liu S, Wang F, Li Z, Sun J (2021) Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430

  46. Liu W, Hasan I, Liao S (2020) Center and scale prediction: A box-free approach for pedestrianand face detection. In: Computer vision and pattern recognition

  47. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25

  48. Padilla R, Passos WL, Dias TL, Netto SL, Da Silva EA (2021) A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics 10(e3):279

    Article  Google Scholar 

  49. Wang Z, Zheng L, Liu Y, Li Y, Wang S (2020) Towards real-time multi-object tracking. In: European conference on computer vision, Springer, pp 107–122

  50. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  51. Jabbar H, Khan RZ (2015) Methods to avoid over-fitting and under-fitting in supervised machine learning (comparative study). Comput Sci Commun Instrum Devices 70

  52. Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J et al (2019) Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155

  53. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(e1):43–76

  54. Kim H-I, Park R-H (2018) Residual lstm attention network for object tracking. IEEE Signal Process Lett 25(e7):1029–1033

    Article  Google Scholar 

Download references

Funding

This work is supported by the French National Association of Research and Technology (ANRT) with the Grant CIFRE 2018/1906.

Author information

Authors and Affiliations

  1. ENIB, UMR CNRS 6285 LabSTICC, 29238, Brest, France

    Morann Mattina, Abdesslam Benzinou & Kamal Nasreddine

  2. Cooperl Innovation S.A.S, 7 rue de la Jeannaie, 22400, Lamballe, France

    Morann Mattina & Francis Richard

Authors
  1. Morann Mattina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdesslam Benzinou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kamal Nasreddine
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francis Richard
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Morann Mattina: Conceptualization, data curation, formal analysis, investigation, methodology, resources, software, validation, visualization, writing - original draft, writing - review and editing. Abdesslam Benzinou: Conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, resources, software, supervision, validation, visualization, writing - original draft, writing - review and editing. Kamal Nasreddine: Conceptualization, data curation, formal analysis, investigation, methodology, resources, software, supervision, validation, visualization, writing - original draft, writing - review and editing. Francis Richard: Conceptualization, data curation, formal analysis, investigation, methodology, resources, software, supervision, validation, visualization, writing - original draft, writing - review and editing

Corresponding author

Correspondence to Abdesslam Benzinou.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mattina, M., Benzinou, A., Nasreddine, K. et al. An efficient center-based method for real-time pig posture recognition and tracking. Appl Intell 54, 5183–5196 (2024). https://doi.org/10.1007/s10489-024-05439-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05439-5

Keywords

Search

Navigation