Skip to main content
SpringerLink
Log in

A review and performance comparison of visual servoing controls

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Visual servoing is a technique for robot control, in which visual feedback is used in a closed-loop control to improve the accuracy and performance of robot systems. The control tasks in visual servoing are defined to control the robot using visual features extracted from the image. There are various problems when applying Visual servoing such as local minima, singularity, and visibility of feature points. These problems can be solved by using different features or using different control schemes. This paper provides a review of visual servoing for robot manipulators and conducts comparisons of five visual servoing approaches. First, the general theory of visual servoing and five different schemes for the comparison and evaluation are presented. Next, the behaviors of the Visual servoing system depending on the selection of visual features are also presented in detail. In addition, the enhancement and combination schemes are also presented, which are a combination of visual servoing with various control techniques that help increase the robustness of visual servoing. To overcome some issues of the image-based visual servoing scheme, different methods to approximate the interaction matrix are presented. After that, the five visual servoing schemes are simulated on Matlab for performance comparison and evaluation. To conduct the assessment, the simulations are implemented with typical control tasks that are translational movements and rotation around the X, Y and Z axes. The evaluations are conducted with the varying motion parameters and the varying effects of noise in the image. The results of the criteria are visually displayed as 3D charts, from which the reviews and comparisons of schemes are drawn. In addition, the paper also evaluates the schemes when performing general movements. General tasks are simulated by using the PUMA 560 robot. Each task has its own purpose to show issues in some schemes as well as how others overcome them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  • Abhilash, T.V., Ashok, S.: Visual Servoing of a Switched System with Supervisory Control. In: International Journal of Scientific & Engineering Research, vol. 7, No. 1. (2016)

  • Ahmadi, B., Zakeri, E., Xie, W.F.: Optimal image-based task-sequence/path planning and robust hybrid vision/force control of industrial robots. IEEE Access 10, 26347–26368 (2022)

    Google Scholar 

  • Alatartsev, S., Stellmacher, S., Ortmeier, F.: Robotic task sequencing problem: a survey. J Intell Robot Syst 80, 279–298 (2015)

    Google Scholar 

  • Andreff, N., Espiau, B., Horaud, R.: Visual servoing from lines. In: IEEE International Conference on Robotics and Automation, pp. 2070–2075. (2000)

  • Cervera, E., Pobil, A., Berry, F., Martinet, P.: Improving image-based visual servoing with three-dimensional features. Int. J. Robot. Res. 22(10–11), 821–840 (2003)

    Google Scholar 

  • Chaumette, F.: A first step toward visual servoing using image moments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 378 -383. (2002)

  • Chaumette, F.: Image moments: a general and useful set of features for visual servoing. IEEE Trans. Rob. 20(4), 713–723 (2004)

    Google Scholar 

  • Chaumette, F., Hutchinson, S., Corke, P.: Visual servoing. In: Siciliano, B., Khatib, O. (eds.) Handbook of robotics, 2nd edn., pp. 841–866. Springer (2016)

    Google Scholar 

  • Chwa, D.: Integral-sliding-mode-observer-based structure and motion estimation of a single object in general motion using a monocular dynamic camera. IEEE Access 8, 14207–14222 (2020)

    Google Scholar 

  • Collewet, C., Marchand, E.: Colorimetry-based visual servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5438–5443. (2009a)

  • Collewet C., Marchand, E.: Photometry-based visual servoing using light reflexion models. In: IEEE International Conference on Robotics and Automation, pp. 701–706. (2009b)

  • Collewet, C., Marchand, E., Chaumette, F.: Visual servoing set free from image processing. In: IEEE International Conference on Robotics and Automation, pp. 81–86. (2008)

  • Cong, V.D., Hanh, L.D: Evaluate Control Laws Related to Interaction Matrix for Image-Based Visual Servoing. In: 2019 6th NAFOSTED Conference on Information and Computer Science (NICS), pp. 454–459. Hanoi, Vietnam (2019)

  • Cong, V.D., Hanh, L.D.: Combination of two visual servoing techniques in contour following task. In: 2021 International Conference on System Science and Engineering (ICSSE), pp. 382–386. (2021)

  • Cong, V.D., Hanh, L.D.: A new decoupled control law for image-based visual servoing control of robot manipulators. Int. J. Intell. Robot. Appl. (2022). https://doi.org/10.1007/s41315-022-00223-5

    Article  Google Scholar 

  • Corke, P.I., Hutchinson, S.A.: A new partitioned approach to image-based visual servo control. IEEE Trans. Robot. Autom. 17(4), 507–515 (2001)

    Google Scholar 

  • Corke, P.I., Hutchinson, S.A.: A new hybrid image-based visual servo control scheme. In: Proceedings of the 39th IEEE Conference on Decision and Control, vol. 3, pp. 2521–2526. (2000)

  • Corke, P.I., Spindler, F., Chaumette, F.: Combining Cartesian and polar coordinates in IBVS. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5962–5967. (2009)

  • Deguchi, K.: A direct interpretation of dynamic images with camera and object motions for vision guided robot control. Int. J. Comput. Vision 37(1), 7–20 (2000)

    MATH  Google Scholar 

  • Dejun, G., Kam, K.L.: Image-based estimation, planning, and control for high-speed flying through multiple openings. Int. J. Robot. Res. 39(9), 1122–1137 (2020)

    Google Scholar 

  • Dementhon, D., Davis, L.S.: Model-based object pose in 25 lines of code. Int. J. Comput. Vision 15(2), 123–141 (1995)

    Google Scholar 

  • Deng, L., Janabi-Sharifi, F., Wilson, W.J.: Hybrid strategies for image constraints avoidance in visual servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 348-353. (2002)

  • Deng, L., Wilson, W.J., Janabi-Sharifi, F.: Dynamic performance of the position-based visual servoing method in the Cartesian and image spaces. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 510-515. (2003)

  • Diyaley, S., Aditya, A., Chakraborty, S.: Optimization of the multi-hole drilling path sequence for concentric circular patterns. Opsearch 57(3), 746–764 (2020)

    MATH  Google Scholar 

  • Dong, J., Zhang, J.: A new image-based visual servoing method with velocity direction control. J. Frankl. Inst. 357, 3993–4007 (2020)

    MathSciNet  MATH  Google Scholar 

  • Espiau, B.: Effect of camera calibration errors on visual servoing in robotics. In: Experimental Robotics III: The 3rd Int. Symp. on Experimental Robotics, pp. 187–193. (1993)

  • Feddema, J.T., Lee, C.S.G., Mitchell, O.R.: Weighted selection of image features for resolved rate visual feedback control. IEEE Trans. Robot. Autom. 7(1), 31–47 (1991)

    Google Scholar 

  • Gans, N.R., Hutchinson, S.A.: Stable visual servoing through hybrid switched-system control. IEEE Trans. Rob. 23(3), 530–540 (2007)

    Google Scholar 

  • Gans, N., Hutchinson, S., Corke, P.: Performance tests for visual servo control systems, with application to partitioned approaches to visual servo control. Int. J. Robot. Res. 22(10/11), 955–981 (2003)

    Google Scholar 

  • Gans, N., Hutchinson, S.: A switching approach to visual servo control. In: 2002 IEEE International Symposiumon Intelligent Control, pp. 770–776. (2002)

  • Ghasemi, A., Li, P., Xie, W.F., Tian, W.: Enhanced switch image-based visual servoing dealing with featuresloss. Electronics 8(8), 1–20 (2019)

    Google Scholar 

  • Ghasemi, A., Li, P., Xie, W.: Adaptive switch image-based visual servoing for industrial robots. Int. J. Control Autom. Syst. 18, 1324–1334 (2020)

    Google Scholar 

  • Hanh, L.D., Cong, V.D.: Implement contour following task of objects with unknown geometric models by using combination of two visual servoing techniques. Int. J. Comput. vis. Robot. 12(5), 464 (2022)

    Google Scholar 

  • Hashimoto, K.: Visual Servoing - Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback. World Scientific, Singapore (1993)

    Google Scholar 

  • Hosoda, K., Sakamato, K., Asada, M.: Trajectory generation for obstacle avoidance of uncalibrated stereo visual servoing without 3-D reconstruction. IEEE/RSJ Int. Conf. Intell. Robots Syst. 3, 29–34 (1995)

    Google Scholar 

  • Hu, G., Gans, N., Fitz-Coy, N., Dixon, W.: Adaptive homography-based visual servo tracking control via a quaternion formulation. IEEE Trans. Control Syst. Technol. 18(1), 128–135 (2010)

    Google Scholar 

  • Janabi-Sharifi, F., Wilson, W.J.: Automatic selection of image features for visual servoing. IEEE Trans. Robot. Autom. 13(6), 890–903 (1997)

    Google Scholar 

  • Kallem, V., Dewan, M., Swensen, J.P., Hager, G.D., Cowan, N.J.: Kernel based visual servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1975–1980. (2007)

  • Kelly, R., Moreno, J., Campa, R.: Visual servoing of planar robots via velocity fields. In IEEE Conf. Decis. Control (CDC) 4, 4028–4033 (2004)

    Google Scholar 

  • Kelly, R., Bugarin, E., Sanchez, V.: Image-based visual control of nonholonomic mobile robots via velocity fields: Case of partially calibrated inclined camera. In: IEEE Conference on Decision and Control, pp. 3071–3076. (2006)

  • Keshmiri, M., Xie, W.F., Ghasemi, A.: Visual servoing using an optimized trajectory planning technique for a 4 dofs robotic manipulator. Int. J. Control Autom. Syst. 15(3), 1362–1373 (2017)

    Google Scholar 

  • Kurtser, P., Edan, Y.: Planning the sequence of tasks for harvesting robots. Robot. Autom. Syst. 131, 103591 (2020)

    Google Scholar 

  • Kyrki, V., Kragic, D., Christensen, H.I.: New shortest-path approaches to visual servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 349–354, Sendai (2004)

  • Li, S., Ghasemi, A., Xie, W., Gao, Y.: An enhanced ibvs controller of a 6 dof manipulator using hybrid pd-smc method”. Int. J. Control Autom. Syst. 16, 844–855 (2018)

    Google Scholar 

  • Liu, H., Zhu, W., Dong, H., Ke, Y.: Hybrid visual servoing for rivet-in-hole insertion based on super-twisting sliding mode control. Int. J. Control Autom. Syst. 18, 2145–2156 (2020)

    Google Scholar 

  • Malis, E.: Visual servoing invariant to changes in camera-intrinsic parameters”. IEEE Trans. Robot. Autom. 20(1), 72–81 (2004)

    Google Scholar 

  • Malis, E., Mezouar, Y., Rives, P.: Robustness of image-based visual servoing with a calibrated camera in the presence of uncertainties in the three-dimensional structure. IEEE Trans. Robot. 26(1), 112–120 (2010)

    Google Scholar 

  • Malis, E., Chaumette, F., Boudet, S.: 2–1/2 D visual servoing. IEEE Trans. Robot. Autom. 15(2), 238–250 (1999)

    Google Scholar 

  • Mansard, N., Chaumette, F.: Tasks sequencing for visual servoing. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS’04), Sendai, Japan (2004)

  • Mansard, N., Chaumette, F.: A new redundancy formalism for avoidance in visual servoing. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, (2005)

  • Mansard, N., Chaumette, F.: Task sequencing for high level sensor-based control. IEEE Trans. Robot. 23, 60–72 (2007)

    Google Scholar 

  • Marchand, E., Chaumette, F.: Feature tracking for visual servoing purposes. Robot. Auton. Syst. 52(1), 53–70 (2005)

    Google Scholar 

  • Martinet, P., Gallice, J., Khadraoui, D.: Vision Based Control Law using 3D Visual Features. In: Committees, Econometrica, pp. 497–502. (1996)

  • Mezouar, Y., Chaumette, F.: Path planning for robust image-based control. IEEE Trans. Robot. Autom. 18, 534–549 (2002)

    Google Scholar 

  • Nayar, S.K., Nene, S.A., Murase, H.: Subspace methods for robot vision. IEEE Trans. Robot. Autom. 12(5), 750–758 (1996)

    Google Scholar 

  • Nematollahi, E., Janabi-Sharifi, F.: Generalizations to control laws of image-based visual servoing. Int. J. Optomechatronics 3, 167–186 (2009)

    Google Scholar 

  • Norouzi-Ghazbi, S., Janabi-Sharifi, F.: A switching image-based visual servoing method for cooperative continuum robots. J Intell Robot Syst 103, 42 (2021)

    Google Scholar 

  • Pages, J., Collewet, C., Chaumette, F., Salvi, J.: Optimizing plane-to-plane positioning tasks by image-based visual servoing and structured light. IEEE Trans. Rob. 22(5), 1000–1010 (2006)

    Google Scholar 

  • Ren, X., Li, H., Li, Y.: Image-based visual servoing control of robot manipulators using hybrid algorithm with feature constraints. IEEE Access 8, 223495–223508 (2020)

    Google Scholar 

  • Sanderson, A.C., Weiss, L.E.: Image-based visual servo control using relational graph error signals. In: IEEE International Conference on Cybernetics and Society, pp. 1074–1077. (1980)

  • Shi, J., Tomasi, C: Good features to track. In: IEEE Conference on Computer Society, pp 593–600. (1994)

  • Shirai, Y., Inoue, H.: Guiding a robot by visual feedback in assembly tasks. Pattern Recogn 5, 99–108 (1973)

    Google Scholar 

  • Shu, T., Gharaaty, S., Xie, W.F., Joubair, A., Bonev, I.: Dynamic path tracking of industrial robots with high accuracy using photogrammetry sensor. IEEE/ASME Trans. Mechatron. 23(3), 1159–1170 (2018)

    Google Scholar 

  • Tahri, O., Chaumette, F.: Complex objects pose estimation based on image moment invariants. In: IEEE International Conference on Robotics and Automation, pp. 436–441. (2005)

  • Tahri, O., Chaumette, F.: Application of moment invariants to visual servoing. In: IEEE International Conference on Robotics and Automation, vol. 3, pp. 4276–4281. (2003)

  • Tahri O., Chaumette, F.: Image moments: generic descriptors for decoupled image-based visual servo. In: IEEE International Conference on Robotics and Automation, vol. 2, pp. 1185–1190. (2004)

  • Tahri, O., Chaumette, F.: Point-based and region-based image moments for visual servoing of planar objects. IEEE Trans. Rob. 21(6), 1116–1127 (2005)

    Google Scholar 

  • Wang, H., Yang, B., Liu, Y., Chen, W., Liang, X., Pfeifer, R.: Visual servoing of soft robot manipulator in constrained environments with an adaptive controller. IEEE/ASME Trans. Mechatron. 22(1), 41–50 (2017)

    Google Scholar 

  • Wang, J., Wilson, W.J.: 3D relative position and orientation estimation using Kalman filter for robot control. In: IEEE International Conference on Robotics and Automation, pp. 2638–2645. (1992)

  • Wilson, W.J., Williams Hulls, C.C., Bell, G.S.: Relative end-effector control using cartesian position based visual servoing. IEEE Trans. Robot. Autom. 12(5), 684–696 (1996)

    Google Scholar 

  • Wu, J., Jin, Z., Liu, A., Yu, J., Yang, F.: A survey Of learning-based control of robotic visual servoing systems. J. Franklin Inst. 359(1), 556–577 (2022)

    MATH  Google Scholar 

  • Xiaolin, R., Hongwen, L.: Uncalibrated image-based visual servoing control with maximum correntropy kalman filter. IFAC-PapersOnLine 53(5), 560 (2020)

    Google Scholar 

  • Xu, X., Hu, Y., Zhai, J.M., Li, L.Z., Guo, P.S.: A novel non-collision trajectory planning algorithm based on velocity potential field for robotic manipulator. Int. J. Adv. Robot. Syst. 15(4), 1–13 (2018)

    Google Scholar 

  • Zanne, P., Morel, G., Piestan, F.: Robust vision-based 3D trajectory tracking using sliding mode control. In IEEE Int. Conf. Robot. Autom. 3, 2088–2093 (2000)

    Google Scholar 

  • Zhao, Y., Xie, W.F., Liu, S.: Image-based visual servoing using improved image moments in 6-DOF robot systems. Int. J. Control Autom. Syst. 11(3), 586–596 (2013)

    Google Scholar 

  • Zhao, Y.M., Lin, Y., Xi, F., Guo, S., Ouyang, P.: Switch-based sliding mode control for position-based visual servoing of robotic riveting system. J. Manuf. Sci. Eng. Trans. ASME. 139(4), 1–11 (2017)

    Google Scholar 

  • Zhao, T., Li, H., Dian, S.: Multi-robot path planning based on improved artificial potential field and fuzzy inference system1. J. Intell. Fuzzy Syst. 39(5), 7621–7637 (2020)

    Google Scholar 

  • Zhao, X., Emami, M.R., Zhang, S.: Image-based control for rendezvous and synchronization with a tumbling space debris. Acta Astronaut. 179, 56–68 (2021)

    Google Scholar 

  • Zhong, X., Zhong, X., Hu, H., Peng, X.: Adaptive neuro-filtering based visual servo control of a robotic manipulator. IEEE Access 7, 76891–76901 (2019)

    Google Scholar 

Download references

Acknowledgements

We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study.

Author information

Authors and Affiliations

  1. Industrial Maintenance Training Center, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Vo Duy Cong

  2. Department of Mechatronics, Faculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Le Duc Hanh

  3. Vietnam National University Ho Chi Minh City, Linh Trung Ward, Thu Duc City, Ho Chi Minh City, Vietnam

    Vo Duy Cong & Le Duc Hanh

Authors
  1. Vo Duy Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Le Duc Hanh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vo Duy Cong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cong, V.D., Hanh, L.D. A review and performance comparison of visual servoing controls. Int J Intell Robot Appl 7, 65–90 (2023). https://doi.org/10.1007/s41315-023-00270-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-023-00270-6

Keywords

Search

Navigation