Skip to main content
SpringerLink
Log in

Bots in Software Development: A Systematic Literature Review and Thematic Analysis

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Modern Software Engineering thrives with innovative tools that aid developers in creating better software grounded on quality standards. Software bots are an emerging and exciting trend in this regard, supporting numerous software development activities. As an emerging trend, few studies describe and analyze different bots in software development. This research presents a systematic literature review covering the state of the art of applied and proposed bots for software development. Our study spans literature from 2003 to 2022, with 82 different bots applied in software development activities, covering 83 primary studies. We found four bot archetypes: chatbots which focus on direct communication with developers to aid them, analysis bots that display helpful information in different tasks, repair bots for resolving software defects, and development bots that combine aspects of other bot technologies to provide a service to the developer. The primary benefits of using bots are increasing software quality, providing useful information to developers, and saving time through the partial or total automation of development activities. However, drawbacks are reported, including limited effectiveness in task completion, high coupling to third-party technologies, and some prejudice from developers toward bots and their contributions. We discovered that including Bots in software development is a promising field of research in software engineering that has yet to be fully explored.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.

REFERENCES

  1. Nagaria, B. and Hall, T., How software developers mitigate their errors when developing code, IEEE Trans. Software Eng., 2022, vol. 48, no. 6.

  2. Suta, P., Lan, X., Wu, B., Mongkolnam, P., and Chan, J., An overview of machine learning in chatbots, Int. J. Mech. Eng. Rob. Res., 2020, vol. 9, no. 4, pp. 502–510.

    Google Scholar 

  3. Rainey, S.K., Brown, B., and Kirk, D.B., Bots, natural language processing, and machine learning, Tax Exec., 2017, vol. 69, p. 39.

    Google Scholar 

  4. Lebeuf, C.R., A taxonomy of software bots: Towards a deeper understanding of software bot characteristics, Ph.D. Thesis, 2018.

  5. Lebeuf, C., Zagalsky, A., Foucault, M., and Storey, M.-A., Defining and classifying software bots: a faceted taxonomy, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 1–6.

  6. Erlenhov, L., de Oliveira Neto, F.G., Scandariato, R., and Leitner, P., Current and future bots in software development, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 7–11.

  7. Orgeolet, L., Foulquier, N., Misery, L., Redou, P., Pers, J.-O., Devauchelle-Pensec, V., and Saraux, A., Can artificial intelligence replace manual search for systematic literature? Review on cutaneous manifestations in primary Sjogren’s syndrome, Rheumatology, 2020, vol. 59, no. 4, pp. 811–819.

    Article  Google Scholar 

  8. Ciupe, A., Meza, S., and Orza, B., Systematic assessment of interactive instructional technologies in higher engineering education, in Proc. Int. Conf. on Interactive Collaborative Learning, Springer, 2020, pp. 797–804.

  9. Jung, H., Lee, J., and Park, C., Deriving design principles for educational chatbots from empirical studies on human-chatbot interaction, J. Digital Contents Soc., 2020, vol. 21, pp. 487–493.

    Article  Google Scholar 

  10. Bhirud, N., Tataale, S., Randive, S., and Nahar, S., A literature review on chatbots in healthcare domain, Int. J. Sci. Technol. Res., 2019, vol. 8, no. 7, pp. 225–231.

    Google Scholar 

  11. Perez, J.Q., Daradoumis, T., and Puig, J.M.M., Rediscovering the use of chatbots in education: A systematic literature review, Comput. Appl. Eng. Educ., 2020, vol. 28, no. 6, pp. 1549–1565.

    Article  Google Scholar 

  12. Syvanen, S. and Valentini, C., Conversational agents in on-line organization–stakeholder interactions: A state-of-the-art analysis and implications for further research, J. Commun. Manag., 2020, vol. 24, no. 4.

  13. Borsci, S., Malizia, A., Schmettow, M., Van Der Velde, F., Tariverdiyeva, G., Balaji, D., and Chamberlain, A., The chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents, Pers. Ubiquitous Comput., 2022, vol. 26, no. 1, pp. 95–119.

    Article  Google Scholar 

  14. Feldt, R., de Oliveira Neto, F.G., and Torkar, R., Ways of applying artificial intelligence in software engineering, Proc. 6th IEEE/ACM Int. Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE), Gothenburg, 2018, pp. 35–41.

  15. Erlenhov, L., Neto, F.G.D.O., and Leitner, P., An empirical study of bots in software development: characteristics and challenges from a practitioner’s perspective, in Proc. 28th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering, Association for Computing Machinery, 2020, pp. 445–455.

  16. Santhanam, S., Hecking, T., Schreiber, A., and Wagner, S., Bots in software engineering: A systematic mapping study, Peer J. Comput. Sci., 2022, vol. 8, p. e866.

    Article  Google Scholar 

  17. Lebeuf, C., Zagalsky, A., Foucault, M., and Storey, M.-A., Defining and classifying software bots: A faceted taxonomy, Proc. IEEE/ACM 1st Int. Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 1–6.

  18. Kitchenham, B.A., Budgen, D., and Brereton, P., Evidence-Based Software Engineering and Systematic Reviews, CRC Press, 2015, vol. 4.

    Book  Google Scholar 

  19. Zhang, H., Babar, M.A., and Tell, P., Identifying relevant studies in software engineering, Inf. Software Technol., 2011, vol. 53, no. 6, pp. 625–637. https://doi.org/10.1016/j.infsof.2010.12.010

    Article  Google Scholar 

  20. Zenodo: Bots and their Uses in Software Development: A Systematic Mapping Study [Data Set]. https://doi.org/10.5281/zenodo.7872403

  21. Wohlin, C., Guidelines for snowballing in systematic literature studies and a replication in software engineering, Proc. 18th Int. Conf. on Evaluation and Assessment in Software Engineering, London, 2014.

  22. Dybå, T. and Dingsoyr, T., Empirical studies of agile software development: A systematic review, Inf. Software Technol., 2008, vol. 50, no. 9–10, pp. 833–859.

    Article  Google Scholar 

  23. Cruzes, D. and Dybå, T., Recommended Steps for Thematic Synthesis in Software Engineering, Trondheim: Dept. of Computer and Information Science, 2011.

    Book  Google Scholar 

  24. Matthies, C., Dobrigkeit, F., and Hesse, G., An additional set of (automated) eyes: Chatbots for agile retrospectives, Proc. 1st Int. IEEE/ACM Workshop on Bots in Software Engineering (BotSE), Montreal, 2019, pp. 34–37.

  25. Balachandran, V., Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation, Proc. 35th Int. Conf. on Software Engineering (ICSE), San Francisco, 2013, pp. 931–940.

  26. McConnell, S., Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers, Addison-Wesley, 2004.

    Google Scholar 

Download references

Funding

This work was supported by ongoing institutional funding. No additional grants to carry out or direct this particular research were obtained

Author information

Authors and Affiliations

  1. School of Statistics and Informatics, Universidad Veracruzana, Veracruz, Xalapa, México

    R. Moguel-Sánchez, C. S. Sergio Martínez-Palacios, J. O. Ocharán-Hernández, X. Limón & A. J. Sánchez-García

Authors
  1. R. Moguel-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. S. Sergio Martínez-Palacios
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. O. Ocharán-Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. X. Limón
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. J. Sánchez-García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to R. Moguel-Sánchez, C. S. Sergio Martínez-Palacios, J. O. Ocharán-Hernández, X. Limón or A. J. Sánchez-García.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s Note.

Pleiades Publishing remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moguel-Sánchez, R., Martínez-Palacios, C.S., Ocharán-Hernández, J.O. et al. Bots in Software Development: A Systematic Literature Review and Thematic Analysis. Program Comput Soft 49, 712–734 (2023). https://doi.org/10.1134/S0361768823080145

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768823080145

Keywords:

Search

Navigation