Skip to main content
SpringerLink
Log in

An embedded FPGA-SoC framework and its usage in moving object tracking application

  • Published:
Design Automation for Embedded Systems Aims and scope Submit manuscript

Abstract

Moving object tracking is a computation-intensive operation that requires accelerating hardware solution. In this work, a high-performance design for mean shift based moving object tracking algorithm and its FPGA implementation is done. Here, associated circuits are utilized as intellectual-property cores to implement an embedded system-on-a-chip (SoC) framework for real-time moving object tracking application. Real-time video with \(640 \times 480\) resolution at 60 frames per second is captured and buffered in SDRAM, and processing is performed on the temporal frames. In the implemented FPGA-SoC framework, PowerPC processor embedded inside the FPGA device is used for the platform configuration, IPs control, and running the application program. The design require 30.08% slices, 35.14% BRAMs, and 43.75% DSP48E slices of Xilinx Virtex-5 xc5vfx70t FPGA device on the ML-507 platform. The computed power is 48.3 mW.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability Statement

The manuscript has no associated data.

References

  1. Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577

    Article  Google Scholar 

  2. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  3. Yilmaz A, Javed O, and Shah M (2006) Object tracking: A survey. ACM computing surveys (CSUR), 38(4):13–es

  4. Sreenivasan Koduri K, Mandyam S, Alireza K (1993) Automated vision system for inspection of IC pads and bonds. IEEE Trans Compon Hybrids Manuf Technol 16(3):333–338

    Article  Google Scholar 

  5. Yoshida H, Nappi Janne (2001) Three-dimensional computer-aided diagnosis scheme for detection of colonic polyps. IEEE Trans Med Imaging 20(12):1261–1274

    Article  Google Scholar 

  6. Kotmel R, Soltesz P, Wondka A, and Perkins R. Methods of endobronchial diagnosis using imaging, March 20 2003. US Patent App. 10/241,974

  7. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans Syst Man Cybern Part C (Appl Rev) 34(3):334–352

    Article  Google Scholar 

  8. Li NJ, Chuang CF, Wei YT, Wang WJ, and Chen HC (2012) A video surveillance system for people detection and number estimation. In 2012 international conference on fuzzy theory and its applications (iFUZZY2012), pages 249–253, Taichung, Taiwan, 16–18. IEEE

  9. Burger W, Burge MJ (2010) Principles of digital image processing: core algorithms. Springer Science & Business Media, Berlin

    MATH  Google Scholar 

  10. Dai X, Khorram S (1999) A feature-based image registration algorithm using improved chain-code representation combined with invariant moments. IEEE Trans Geosci Remote Sens 37(5):2351–2362

    Article  Google Scholar 

  11. Kato H and Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In Proceedings 2nd IEEE and ACM international workshop on augmented reality (IWAR’99), pages 85–94, San Francisco, CA, USA, 20–21 . IEEE

  12. Sharp CS, Shakernia O, and Sastry SS (2001) A vision system for landing an unmanned aerial vehicle. In Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164), volume 2, pages 1720–1727, Seoul, South Korea, 21–26 . IEEE

  13. Saripalli S, Montgomery JF, and Sukhatme GS (2002) Vision-based autonomous landing of an unmanned aerial vehicle. In Proceedings 2002 IEEE international conference on robotics and automation (Cat. No. 02CH37292), volume 3, pages 2799–2804, Washington, DC, USA, 11–15 . IEEE

  14. Yilmaz Al, Li X, Shah M (2004) Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans Pattern Anal Mach Intell 26(11):1531–1536

    Article  Google Scholar 

  15. Baker S, Kanade T et al (2005) Shape-from-silhouette across time part ii: applications to human modeling and markerless motion tracking. Int J Comput Vis 63(3):225–245

    Article  Google Scholar 

  16. Yang F, Paindavoine M (2003) Implementation of an RBF neural network on embedded systems: real-time face tracking and identity verification. IEEE Trans Neural Netw 14(5):1162–1175

    Article  Google Scholar 

  17. Shi J et al (1994) Good features to track. In 1994 proceedings of ieee conference on computer vision and pattern recognition, pages 593–600, Seattle, WA, USA, 21–23 . IEEE

  18. Zhong Y, Jain AK (2000) Object localization using color, texture and shape. Pattern Recogn 33(4):671–684

    Article  Google Scholar 

  19. Cho M, Kwak S, Laptev I, Schmid C, and Ponce J (2015) Unsupervised object discovery and localization in images and videos. In 2015 12th international conference on ubiquitous robots and ambient intelligence (URAI), pages 292–293, Goyang, South Korea, 28–30 . IEEE

  20. Nummiaro K, Koller-Meier E, Van Gool L (2003) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110

    Article  Google Scholar 

  21. Yang C, Duraiswami R, and Davis L (2005) Fast multiple object tracking via a hierarchical particle filter. In 10th ieee international conference on computer vision (ICCV’05) Volume 1, volume 1, pages 212–219, Beijing, China, 17–21 . IEEE

  22. Mikic I, Krucinski S, James T (1998) Segmentation and tracking in echocardiographic sequences: active contours guided by optical flow estimates. IEEE Trans Med Imaging 17(2):274–284

    Article  Google Scholar 

  23. Black MJ, Jepson AD (1998) Eigentracking: robust matching and tracking of articulated objects using a view-based representation. Int J Comput Vis 26(1):63–84

    Article  Google Scholar 

  24. Allen JG, Xu RYD, Jin JS, et al (2004) Object tracking using camshift algorithm and multiple quantized feature spaces. In ACM international conference proceeding series, volume 100, pages 3–7, Sydney, Australia, 1–1 . Citeseer

  25. Wang Z, Yang X, Yi X, Songyu Y (2009) Camshift guided particle filter for visual tracking. Pattern Recogn Lett 30(4):407–413

    Article  Google Scholar 

  26. Lampert CH, Blaschko MB, Hofmann T (2009) Efficient subwindow search: a branch and bound framework for object localization. IEEE Trans Pattern Anal Mach Intell 31(12):2129–2142

    Article  Google Scholar 

  27. Pandey M and Lazebnik S (2011) Scene recognition and weakly supervised object localization with deformable part-based models. In 2011 international conference on computer vision, pages 1307–1314, Barcelona, Spain, 6–13 IEEE

  28. Comaniciu D, Ramesh V, and Meer P (2000) Real-time tracking of non-rigid objects using mean shift. In proceedings IEEE conference on computer vision and pattern recognition. CVPR 2000 (Cat. No. PR00662), volume 2, pages 142–149, Hilton Head Island, SC, USA, 15-15 . IEEE

  29. Krebs S, Duraisamy B, and Flohr F (2017) A survey on leveraging deep neural networks for object tracking. In 2017 IEEE 20th international conference on intelligent transportation systems (ITSC), pages 411–418, Yokohama, Japan, 16–19 . IEEE

  30. Chandan G, Jain A, Jain H, et al (2018) Real time object detection and tracking using deep learning and OpenCV. In 2018 international conference on inventive research in computing applications (ICIRCA), pages 1305–1308, Coimbatore, India, 11–12. IEEE

  31. Craciun S, Kirchgessner R, George AD, Lam H, Principe JC (2018) A real-time, power-efficient architecture for mean-shift image segmentation. J Real-Time Image Process 14(2):379–394

    Article  Google Scholar 

  32. Pandey JG, Karmakar A, Shekhar C, and Gurunarayanan S (2015) An embedded framework for accurate object localization using center of gravity measure with mean shift procedure. In 2015 19th international symposium on vlsi design and test, pages 1–6, Ahmedabad, India, 26–29 . IEEE

  33. Cheng Y (1995) Mean shift, mode seeking, and clustering. IEEE Trans Pattern Anal Mach Intell 17(8):790–799

    Article  Google Scholar 

  34. Demirović D (2019) An implementation of the mean shift algorithm. Image Process Line 9:251–268

    Article  MathSciNet  Google Scholar 

  35. Ali U, Malik MB (2010) Hardware/software co-design of a real-time kernel based tracking system. J Syst Arch 56(8):317–326

    Article  Google Scholar 

  36. Patel H, Singhadia A (2014) Object tracking system using mean shift algorithm and implementation on FPGA. Int J Eng Trends Technol 18(6):293–296

    Article  Google Scholar 

  37. Sajjanar S, Mankani SK, Dongrekar PR, Kumar NS, and Aradhya HVR (2016) Implementation of real time moving object detection and tracking on FPGA for video surveillance applications. In 2016 IEEE distributed computing, VLSI, electrical circuits and robotics (DISCOVER), pages 289–295, Mangalore, India, 13-14 .IEEE

  38. Chen X, Xu J, and Yu Z (2017) A fast and energy efficient FPGA-based system for real-time object tracking. In 2017 Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), pages 965–968. IEEE

  39. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Article  MathSciNet  Google Scholar 

  40. Pandey JG, Purushottam S, Karmakar A, Shekhar C (2012) Platform-based extensible hardware-software video streaming module for a smart camera system. Int J Model Optimiz 2(4):482

    Article  Google Scholar 

  41. Pandey JG, Karmakar A (2019) Unsupervised image thresholding: hardware architecture and its usage for FPGA-SoC platform. Int J Electron 106(3):455–476

    Article  Google Scholar 

  42. Stefano LDi and A Bulgarelli (1999) A simple and efficient connected components labeling algorithm. In proceedings 10th international conference on image analysis and processing, pages 322–327, Venice, Italy, 27–29 . IEEE

  43. Pandey JG, Karmakar A, Mishra AK, Shekhar C, and Gurunarayanan S (2014) Implementation of an improved connected component labeling algorithm using FPGA-based platform. In 2014 international conference on signal processing and communications (SPCOM), pages 1–6, Bangalore, India, 22–25. IEEE

  44. Madisetti V, Arpnikanondt C (2006) A platform-centric approach to system-on-chip (SOC) Design. Springer Science & Business Media, Berlin

    Google Scholar 

  45. Densmore D, Passerone R (2006) A platform-based taxonomy for ESL design. IEEE Design Test Comput 23(5):359–374

    Article  Google Scholar 

  46. Sass R, Schmidt AG (2010) Embedded systems design with platform FPGAs: principles and practices. Morgan Kaufmann

  47. Xilinx (2020) Ml505/ml506/ml507 evaluation platform

  48. Martin G, Chang H (2012) Winning the SoC revolution: experiences in real design. Springer Science & Business Media, Berlin

    Google Scholar 

  49. Trieu DBK and Maruyama T (2011) An implementation of the mean shift filter on FPGA. In 2011 21st international conference on field programmable logic and applications, pages 219–224, Chania, Greece, 5–7 . IEEE

  50. Tehreem A, Khawaja SG, Khan AM, Akram MU, Khan SA (2019) Multiprocessor architecture for real-time applications using mean shift clustering. J Real-Time Image Process 16(6):2233–2246

    Article  Google Scholar 

  51. Trieu DBK, Maruyama T (2015) Real-time color image segmentation based on mean shift algorithm using an FPGA. J Real-Time Image Process 10(2):345–356

    Article  Google Scholar 

  52. Gorry B, Chen Z, Hammond K, Wallace A, Michaelson G (2007) Using mean-shift tracking algorithms for real-time tracking of moving images on an autonomous vehicle testbed platform. Proc World Acad Sci Eng Technol 25:1307–6884

    Google Scholar 

  53. Pandey JG, Karmakar A, Shekhar C, and Gurunarayanan S (2015) An FPGA-based architecture for local similarity measure for image/video processing applications. In 2015 28th international conference on VLSI design, pages 339–344, Bangalore, India, 3–7 . IEEE

  54. Kim H, Nam BG, Sohn JH, Woo JH, Yoo HJ (2006) A 231-mhz, 2.18-mw 32-bit logarithmic arithmetic unit for fixed-point 3-D graphics system. IEEE J Solid-State Circuits 41(11):2373–2381

Download references

Acknowledgements

The author extends sincere gratitude to the Director, CSIR - Central Electronics Engineering Research Institute (CEERI), Pilani, India, and Ministry of Electronics and Information Technology, Govt. of India for providing necessary resources through special manpower development program for chips-to-system design (SMDP-C2SD) project.

Author information

Authors and Affiliations

  1. Integrated Systems Laboratory, CSIR - Central Electronics Engineering Research Institute, Pilani, Rajasthan, 333-031, India

    Jai Gopal Pandey

Authors
  1. Jai Gopal Pandey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jai Gopal Pandey.

Ethics declarations

Conflict of interest

The author has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pandey, J.G. An embedded FPGA-SoC framework and its usage in moving object tracking application. Des Autom Embed Syst 25, 213–236 (2021). https://doi.org/10.1007/s10617-021-09252-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10617-021-09252-y

Keywords

Search

Navigation