Skip to main content
SpringerLink
Log in

Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter

  • Special Section: Computer Vision Empowering Earthquake Engineering and Engineering Vibration
  • Published:
Earthquake Engineering and Engineering Vibration Aims and scope Submit manuscript

Abstract

Computer vision (CV)-based techniques have been widely used in the field of structural health monitoring (SHM) owing to ease of installation and cost-effectiveness for displacement measurement. This paper introduces computer vision based method for robust displacement measurement under occlusion by incorporating random sample consensus (RANSAC). The proposed method uses the Kanade-Lucas-Tomasi (KLT) tracker to extract feature points for tracking, and these feature points are filtered through RANSAC to remove points that are noisy or occluded. With the filtered feature points, the proposed method incorporates Kalman filter to estimate acceleration from velocity and displacement extracted by the KLT. For validation, numerical simulation and experimental validation are conducted. In the simulation, performance of the proposed RANSAC filtering was validated to extract correct displacement out of group of displacements that includes dummy displacement with noise or bias. In the experiment, both RANSAC filtering and acceleration measurement were validated by partially occluding the target for tracking attached on the structure. The results demonstrated that the proposed method successfully measures displacement and estimates acceleration as compared to a reference displacement sensor and accelerometer, even under occluded conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arcasoy CC and Ouyang G (1997), “Analytical Solution of α-β-Γ Tracking Filter with a Noisy Jerk as Correlated Target Maneuver Model,” IEEE Transactions on Aerospace and Electronic Systems, 33: 347–353.

    Article  Google Scholar 

  • Anandan P (1985), “Computing Dense Displacement Fields with Confidence Measures in Scenes Containing Occlusion,” Intelligent Robots and Computer Vision, 521:184–194.

    Article  Google Scholar 

  • Cho S, Sim SH, Park JW and Lee J (2014), “Extension of Indirect Displacement Estimation Method Using Acceleration and Strain to Various Types of Beam Structures,” Smart Structures and Systems, 14: 699–718.

    Article  Google Scholar 

  • Cho S, Yun CB and Sim SH (2015), “Displacement Estimation of Bridge Structures Using Data Fusion of Acceleration and Strain Measurement Incorporating Finite Element Model,” Smart Structures and Systems, 15: 645–663.

    Article  Google Scholar 

  • Choi HS, Cheung JH, Kim SH and Ahn JH (2011), “Structural Dynamic Displacement Vision System Using Digital Image Processing,” NDT & E International, 44: 597–608.

    Article  Google Scholar 

  • Cimellaro GP (2007), “Simultaneous Stiffness-Damping Optimization of Structures with Respect to Acceleration, Displacement and Base Shear,” Engineering Structures, 29: 2853–2870.

    Article  Google Scholar 

  • Dong CZ and Catbas FN (2019), “A Non-Target Structural Displacement Measurement Method Using Advanced Feature Matching Strategy,” Advances in Structural Engineering, 22: 3461–3472.

    Article  Google Scholar 

  • Edison A and Jiji C (2017), “Optical Acceleration for Motion Description in Videos,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 39–47.

  • Farnebäck G (2003), “Two-Frame Motion Estimation Based on Polynomial Expansion,” Scandinavian Conference on Image Analysis, 363–370.

  • Feng D and Feng MQ (2016), “Vision-Based Multipoint Displacement Measurement for Structural Health Monitoring,” Structural Control and Health Monitoring, 23: 876–890.

    Article  Google Scholar 

  • Feng D and Feng MQ (2018), “Computer Vision for SHM of Civil Infrastructure: From Dynamic Response Measurement to Damage Detection–A Review,” Engineering Structures, 156: 105–117.

    Article  Google Scholar 

  • Feng D, Feng MQ, Ozer E and Fukuda Y (2015), “A Vision-Based Sensor for Noncontact Structural Displacement Measurement,” Sensors, 15: 16557–16575.

    Article  Google Scholar 

  • Fukuda Y, Feng MQ, Narita Y, Kaneko Si and Tanaka T (2013), “Vision-Based Displacement Sensor for Monitoring Dynamic Response Using Robust Object Search Algorithm,” IEEE Sensors Journal, 13: 4725–4732.

    Article  Google Scholar 

  • Gillijns S and De Moor B (2007a), “Unbiased Minimum-Variance Input and State Estimation for Linear Discrete-Time Systems,” Automatica, 43: 111–116.

    Article  Google Scholar 

  • Gillijns S and De Moor B (2007b), “Unbiased Minimum-Variance Input and State Estimation for Linear Discrete-Time Systems with Direct Feedthrough,” Automatica, 43: 934–937.

    Article  Google Scholar 

  • Goldenstein S and Vogler C (2006), “When Occlusions are Outliers,” 2006 Conference on Computer Vision and Pattern Recognition Workshop. pp. 98.

  • Gomez F, Park JW and Spencer BF (2018), “Reference-Free Structural Dynamic Displacement Estimation Method,” Structural Control and Health Monitoring, 25: e2209.

    Article  Google Scholar 

  • Hartmann C, Wang J, Opristescu D and Volk W (2018), “Implementation and Evaluation of Optical Flow Methods for Two-Dimensional Deformation Measurement in Comparison to Digital Image Correlation,” Optics and Lasers in Engineering, 107: 127–141.

    Article  Google Scholar 

  • Horn BK and Schunck BG (1981), “Determining Optical Flow,” Artificial Intelligence, 17: 185–203.

    Article  Google Scholar 

  • Jabar F, Farokhi S and Sheikh U (2015), “Object Tracking Using SIFT and KLT Tracker for UAV-Based Applications,” in IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), IEEE, pp. 65–68.

  • Jeong JH and Jo H (2022), “Real-Time Generic Target Tracking for Structural Displacement Monitoring Under Environmental Uncertainties via Deep Learning,” Structural Control and Health Monitoring, 29: e2902.

    Article  Google Scholar 

  • Khuc T and Catbas FN (2018), “Computer Vision-Based Displacement and Vibration Monitoring Without Using Physical Target on Structures,” in Bridge Design, Assessment and Monitoring, Routledge, 89–100.

  • Lee J, Lee KC, Jeong S, Lee YJ and Sim SH (2020), “Long-Term Displacement Measurement of Full-Scale Bridges Using Camera Ego-Motion Compensation,” Mechanical Systems and Signal Processing, 140: 106651.

    Article  Google Scholar 

  • Lee J, Lee KC, Lee S, Lee YJ and Sim SH (2019), “Long-Term Displacement Measurement of Bridges Using a LiDAR System,” Structural Control and Health Monitoring, 26: e2428.

    Article  Google Scholar 

  • Lee JJ and Shinozuka M (2006), “A Vision-Based System for Remote Sensing of Bridge Displacement,” NDT & E International, 39: 425–431.

    Article  Google Scholar 

  • Lee S, Kim H and Sim SH (2022), “Nontarget-Based Displacement Measurement Using LiDAR and Camera,” Automation in Construction, 142: 104493.

    Article  Google Scholar 

  • Lee S, Lee J, Park JW and Sim SH (2020), “Nontarget-Based Measurement of 6-DOF Structural Displacement Using Combined RGB Color and Depth Information,” IEEE/ASME Transactions on Mechatronics, 26: 1358–1368.

    Article  Google Scholar 

  • Lucas BD and Kanade T (1981), “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proceedings of Imaging Understanding Workshop, pp. 121–130, Vancouver.

  • Ma Z, Choi J, Liu P and Sohn H (2022a), “Structural Displacement Estimation by Fusing Vision Camera and Accelerometer Using Hybrid Computer Vision Algorithm and Adaptive Multi-Rate Kalman Filter,” Automation in Construction, 140: 104338.

    Article  Google Scholar 

  • Ma Z, Choi J and Sohn H (2022b), “Real-Time Structural Displacement Estimation by Fusing Asynchronous Acceleration and Computer Vision Measurements,” Computer-Aided Civil and Infrastructure Engineering, 37: 688–703.

    Article  Google Scholar 

  • Merry R, Van de Molengraft M and Steinbuch M (2010), “Velocity and Acceleration Estimation for Optical Incremental Encoders,” Mechatronics, 20: 20–26.

    Article  Google Scholar 

  • Morlier J and Michon G (2010), “Virtual Vibration Measurement Using KLT Motion Tracking Algorithm,” Journal of Dynamic Systems, Measurement, and Control, 132(1): 011003–011011.

    Article  Google Scholar 

  • Narazaki Y, Gomez F, Hoskere V, Smith MD and Spencer BF (2021), “Efficient Development of Vision-Based Dense Three-Dimensional Displacement Measurement Algorithms Using Physics-Based Graphics Models,” Structural Health Monitoring, 20: 1841–1863.

    Article  Google Scholar 

  • Park JW, Lee JJ, Jung HJ and Myung H (2010), “Vision-Based Displacement Measurement Method for High-Rise Building Structures Using Partitioning Approach,” NDT & E International, 43: 642–647.

    Article  Google Scholar 

  • Park JW, Moon DS, Yoon H, Gomez F, Spencer BF and Kim JR (2018), “Visual-Inertial Displacement Sensing Using Data Fusion of Vision-Based Displacement with Acceleration,” Structural Control and Health Monitoring, 25: e2122.

    Article  Google Scholar 

  • Park JW, Sim SH, Jung HJ and Spencer BF (2013), “Development of a Wireless Displacement Measurement System Using Acceleration Responses,” Sensors, 13: 8377–8392.

    Article  Google Scholar 

  • Park KT, Kim SH, Park HS and Lee KW (2005), “The Determination of Bridge Displacement Using Measured Acceleration”, Engineering Structures, 27: 371–378.

    Article  Google Scholar 

  • Rofooei FR and Tadjbakhsh IG (1993), “Optimal Control of Structures with Acceleration, Velocity, and Displacement Feedback,” Journal of Engineering Mechanics, 119: 1993–2010.

    Article  Google Scholar 

  • Sarwar MZ and Park JW (2020), “Bridge Displacement Estimation Using a Co-Located Acceleration and Strain,” Sensors, 20: 1109.

    Article  Google Scholar 

  • Sharifi M, Seif M and Hadi M (2013), “A Comparison Between Numerical Differentiation and Kalman Filtering for a Leo Satellite Velocity Determination,” Artificial Satellites, 48: 103–110.

    Article  Google Scholar 

  • Shi J (1994), “Good Features to Track”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 593–600.

  • Strecha C, Fransens R and Gool LV (2004), “A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection,” International Workshop on Statistical Methods in Video Processing, pp. 71–82.

  • Tomasi C and Kanade T (1991), “Detection and Tracking of Point,” International Journal of Computer Vision, 9: 137–154.

    Article  Google Scholar 

  • Wang SW and Wan SM (2012), “Velocity and Acceleration Computations by Single-Dimensional Kalman Filter with Adaptive Noise Variance,” Przeglad Elektrotechniczny, 88: 283–287.

    Google Scholar 

  • Waterfall P, Macdonald JH and McCormick N (2012), “Targetless Precision Monitoring of Road and Rail Bridges Using Video Cameras,” 6th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2012, pp. 3976–3983.

  • Welch G and Bishop G (1995), “An Introduction to the Kalman Filter,” University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.

    Google Scholar 

  • Won J, Park JW, Park J, Shin J and Park M (2021), “Development of a Reference-Free Indirect Bridge Displacement Sensing System”, Sensors, 21: 5647.

    Article  Google Scholar 

  • Won J, Park JW, Park K, Yoon H and Moon DS (2019), “Non-Target Structural Displacement Measurement Using Reference Frame-Based Deepflow,” Sensors, 19: 2992.

    Article  Google Scholar 

  • Xu Y and Brownjohn JM (2018), “Review of MachineVision Based Methodologies for Displacement Measurement in Civil Structures,” Journal of Civil Structural Health Monitoring, 8: 91–110.

    Article  Google Scholar 

  • Ye X, Yi TH, Dong C and Liu T (2016) “Vision-Based Structural Displacement Measurement: System Performance Evaluation and Influence Factor Analysis,” Measurement, 88: 372–384.

    Article  Google Scholar 

  • Yoon H, Elanwar H, Choi H, Golparvar-Fard M and Spencer BF (2016), “Target-Free Approach for Vision-Based Structural System Identification Using Consumer-Grade Cameras,” Structural Control and Health Monitoring, 23: 1405–1416.

    Article  Google Scholar 

  • Yoon H, Hoskere V, Park JW and Spencer BF (2017), “Cross-Correlation-Based Structural System Identification Using Unmanned Aerial Vehicles,” Sensors, 17: 2075.

    Article  Google Scholar 

  • Zhu J, Lu Z and Zhang C (2021), “A Marker-Free Method for Structural Dynamic Displacement Measurement Based on Optical Flow,” Structure and Infrastructure Engineering, 18: 84–96.

    Article  Google Scholar 

Download references

Acknowledgement

This research was conducted with the support of the “National R&D Project for Smart Construction Technology (RS-2020-KA156887)” funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure, and Transport and managed by the Korea Expressway Corporation and National Research Foundation of Korea (NRF) Grant (NRF-2021R1A6A3A13046053) and the Chung-Ang University Research grants in 2022.

Author information

Authors and Affiliations

  1. Department of Civil and Environmental Engineering, Chung-Ang University, Dongjak, Seoul, 06974, Korea

    Jongbin Won & Jong-Woong Park

  2. Department of Smart Cities, Chung-Ang University, Dongjak, Seoul, 06974, Korea

    Jong-Woong Park & Min-Hyuk Song

  3. R&D Division, Tantan Safety co., Ltd, Seongdong, Seoul, 04798, Korea

    Youn-Sik Kim

  4. Department of Civil and Environmental Engineering, University of Hawaii at Manoa, Honolulu, 96822, USA

    Dosoo Moon

Authors
  1. Jongbin Won
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jong-Woong Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min-Hyuk Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Youn-Sik Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dosoo Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jong-Woong Park.

Additional information

Supported by: National R&D Project for Smart Construction Technology (RS-2020-KA156887) funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure, and Transport and managed by the Korea Expressway Corporation and National Research Foundation of Korea (NRF) Grant (NRF-2021R1A6A3A13046053) and the Chung-Ang University Research grants in 2022

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Won, J., Park, JW., Song, MH. et al. Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter. Earthq. Eng. Eng. Vib. 22, 347–358 (2023). https://doi.org/10.1007/s11803-023-2173-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11803-023-2173-0

Keywords

Search

Navigation