Abstract
Debugging is a core skill required by programmers, yet we know little about how to effectively teach the process of debugging. The challenges of learning debugging are compounded for novices who lack experience and are still learning the tools they need to program effectively. In this work, we report a case study in which we used a think-aloud protocol to gain insight into the behaviour of three students engaged in debugging tasks. Our qualitative analysis reveals a variety of helpful practices and barriers that limit the effectiveness of debugging. We observe that comprehension, evidence-based activities, and workflow practices all contribute to novice debugging success. Lack of sustained effort, precision, and methodical processes negatively impact debugging effectiveness. We anticipate that understanding how students engage in debugging tasks will aid future work to address ineffective behaviours and promote effective debugging activities.
- [1] . 2005. An analysis of patterns of debugging among novice computer science students. In Proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE’05). ACM, New York, NY, 84–88.
DOI: Google ScholarDigital Library - [2] . 2007. The impact of improving debugging skill on programming ability. Innovation in Teaching and Learning in Information and Computer Sciences 6, 4 (2007), 72–87.
DOI: Google ScholarCross Ref - [3] . 2019. An analysis of using many small programs in CS1. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 585–591.
DOI: Google ScholarDigital Library - [4] . 2015. 37 million compilations: Investigating novice programming mistakes in large-scale student data. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE’15). ACM, New York, NY, 522–527.
DOI: Google ScholarDigital Library - [5] . 2021. Common logic errors for programming learners: A three-decade literature survey. In ASEE Virtual Annual Conference Content Access. ASEE Conferences, Virtual Conference. https://peer.asee.org/36814.Google ScholarCross Ref
- [6] . 2016. Categorizing compiler error messages with principal component analysis. In 12th China-Europe International Symposium on Software Engineering Education (CEISEE’16). https://researchrepository.ucd.ie/handle/10197/7889.Google Scholar
- [7] . 2005. Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the 1st International Workshop on Computing Education Research (ICER’05). ACM, New York, NY, 81–86.
DOI: Google ScholarDigital Library - [8] . 1993. Cognitive processes in software fault detection: A review and synthesis. International Journal of Human–Computer Interaction 5, 2 (1993), 189–206.
DOI: Google ScholarCross Ref - [9] . 2008. Mental models, consistency and programming aptitude. In Proceedings of the 10th Conference on Australasian Computing Education - Volume 78 (ACE’08). Australian Computer Society, Inc., AUS, 53–61.Google ScholarDigital Library
- [10] . 2020. Rubber ducks and double labs: Teaching debugging strategies in a geoscience class. In AGU Fall Meeting Abstracts, Vol. 2020. Article
ED011-04 , ED011-04 pages.Google Scholar - [11] . 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101.
DOI: arXiv: https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa.Google ScholarCross Ref - [12] . 2022. Conceptual and design thinking for thematic analysis. Qualitative Psychology 9, 1 (2022), 3–26.
DOI: Google ScholarCross Ref - [13] . 2019. Identifying and validating Java misconceptions toward a CS1 concept inventory. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’19). ACM, New York, NY, 23–29.
DOI: Google ScholarDigital Library - [14] . 2015. Its debug: Practical results. Journal of Computing Sciences in Colleges 30, 3 (
jan 2015), 9–15.Google ScholarDigital Library - [15] . 1989. Cognitive style and diagnostic skills of student programmers. Journal of Research on Computing in Education 21, 4 (1989), 411–420.Google ScholarDigital Library
- [16] . 2013. Novices’ debugging behaviors in VB programming. In 2013 Learning and Teaching in Computing and Engineering. 25–30.
DOI: Google ScholarDigital Library - [17] . 2004. Debugging: From novice to expert. ACM SIGCSE Bulletin 36, 1 (2004), 17–21.Google ScholarDigital Library
- [18] . 1993. Inside/outside: Teacher Research and Knowledge. Teachers College Press, New York.Google Scholar
- [19] . 2020. Dynamics of emotion, problem solving, and identity: Portraits of three girl coders. Computer Science Education 30, 3 (2020), 362–389.
DOI: Google ScholarCross Ref - [20] . 2020. Debugging as a context for collaborative reflection on problem-solving processes. In Deeper Learning, Communicative Competence, and Critical Thinking: Innovative, Research-Based Strategies for Development in 21st Century Classrooms, (Ed.). Routledge, 209–228.Google Scholar
- [21] . 2019. Debugging as a context for fostering reflection on critical thinking and emotion. In Deeper Learning, Dialogic Learning, and Critical Thinking. Routledge, 1–20.
DOI: Google ScholarCross Ref - [22] . 2012. All syntax errors are not equal. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE’12). ACM, New York, NY, 75–80.
DOI: Google ScholarDigital Library - [23] . 2009. Empirical study on college students’ debugging abilities in computer programming. In 2009 1st International Conference on Information Science and Engineering. 3319–3322.
DOI: Google ScholarDigital Library - [24] . 1989. Some difficulties of learning to program. In Studying the Novice Programmer, and (Eds.). Number 14. Lawrence Erlbaum Associates, 283–299. http://www.sussex.ac.uk/Users/bend/papers/diffsofprogramming.pdf.Google Scholar
- [25] . 1988. A review of automated debugging systems: Knowledge, strategies and techniques. In Proceedings of the 10th International Conference on Software Engineering. IEEE Computer Society Press, 162–171.Google ScholarDigital Library
- [26] . 2014. Application of program slicing technique to improve novice programming competency in spoken tutorial workshops. In IEEE 6th International Conference on Technology for Education (T4E’14). IEEE, 32–35.Google ScholarDigital Library
- [27] . 2018. Common logic errors made by novice programmers. In Proceedings of the 20th Australasian Computing Education Conference (ACE’18). ACM, New York, NY, 83–89.
DOI: Google ScholarDigital Library - [28] . 2008. Debugging: Finding, fixing and flailing, a multi-institutional study of novice debuggers. Computer Science Education 18, 2 (2008), 93–116.
DOI: arXiv:.Google ScholarCross Ref - [29] . 2010. Debugging from the student perspective. IEEE Transactions on Education 53, 3 (
Aug 2010), 390–396.DOI: Google ScholarDigital Library - [30] . 2020. Computing Curricula 2020: Paradigms for Global Computing Education. ACM, New York, NY.Google ScholarDigital Library
- [31] . 2008. Identifying important and difficult concepts in introductory computing courses using a delphi process. In Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education. 256–260.Google ScholarDigital Library
- [32] . 2009. Males’ and Females’ script debugging strategies. In End-User Development, , , , and (Eds.). Springer, Berlin, 205–224.Google ScholarCross Ref
- [33] . 2013. Online Python tutor: Embeddable web-based program visualization for Cs education. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). ACM, New York, NY, 579–584.
DOI: Google ScholarDigital Library - [34] . 2022. Synthesizing Research on Programmers’ Mental Models of Programs, Tasks and Concepts – a Systematic Literature Review. (2022).
DOI: Google ScholarCross Ref - [35] . 2013. Exploring the gender effect on cognitive processes in program debugging based on eye-movement analysis. In Proceedings of the 5th International Conference on Computer Supported Education. ResearchGate, 16–21.Google Scholar
- [36] . 2003. Identifying and correcting Java programming errors for introductory computer science students. SIGCSE Bulletin 35, 1 (
Jan 2003), 153–156.DOI: Google ScholarDigital Library - [37] . 2020. Pair debugging of electronic textiles projects: Analyzing think-aloud protocols for high school students’ strategies and practices while problem solving. In ICLS 2020: The 14th International Conference of the Learning Sciences. International Society of the Learning Sciences (ISLS’20). https://repository.isls.org//handle/1/6292.Google Scholar
- [38] . 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY.Google Scholar
- [39] . 2006. Learning to troubleshoot: A new theory-based design architecture. Educational Psychology Review 18 (2006), 77–114.
DOI: Google ScholarCross Ref - [40] . 2019. Rethinking debugging as productive failure for CS education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 169–170.
DOI: Google ScholarDigital Library - [41] . 1987. Debugging: An analysis of bug-location strategies. Human– Computer Interaction 3, 4 (
Dec. 1987), 351–399.DOI: Google ScholarDigital Library - [42] . 2018. What are they thinking? Eliciting student reasoning about troublesome concepts in introductory computer science. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling’18). ACM, New York, NY, Article
7 , 10 pages.DOI: Google ScholarDigital Library - [43] . 1986. Learning flow of control: Recursive and iterative procedures. Human– Computer Interaction 2, 2 (
Jun 1986), 135–166.DOI: Google ScholarDigital Library - [44] . 1987. Qualitative and quantitative methods: When stories converge. New Directions for Program Evaluation 1987, 35 (1987), 57–75.
DOI: arXiv: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ev.1459.Google ScholarCross Ref - [45] . 2015. Evaluation of programming competency using student error patterns. In 2015 International Conference on Learning and Teaching in Computing and Engineering. 34–41.
DOI: Google ScholarDigital Library - [46] . 2020. The CS-Ed Podcast. (2020). Retrieved March 17, 2020 from https://sites.duke.edu/csedpodcast/2020/01/07/episode-3-amy/.Google Scholar
- [47] . 2019. Teaching explicit programming strategies to adolescents. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 469–475.
DOI: Google ScholarDigital Library - [48] . 2020. Explicit programming strategies. Empirical Software Engineering 25, 4 (2020), 2416–2449.Google ScholarCross Ref
- [49] . 2013. How programmers debug, revisited: An information foraging theory perspective. IEEE Transactions on Software Engineering 39, 2 (2013), 197–215.
DOI: Google ScholarDigital Library - [50] . 1999. Debug it: A debugging practicing system. Computers & Education 32, 2 (1999), 165–179.Google ScholarDigital Library
- [51] . 2015. Teaching and engaging with debugging puzzles. Ph.D. Dissertation. University of Washington.Google Scholar
- [52] . 2012. The importance of students’ attention to program state: A case study of debugging behavior. In Proceedings of the 9th Annual International Conference on International Computing Education Research (ICER’12). ACM, New York, NY, 127–134.
DOI: Google ScholarDigital Library - [53] . 2019. Towards a framework for teaching debugging. In Proceedings of the 21st Australasian Computing Education Conference (ACE’19). ACM, New York, NY, 79–86.Google ScholarDigital Library
- [54] . 2016. Tracking students’ cognitive processes during program debugging–an eye-movement approach. IEEE Transactions on Education 59, 3 (
Aug 2016), 175–186.DOI: Google ScholarDigital Library - [55] . 2017. Understanding problem solving behavior of 6–8 graders in a debugging game. Computer Science Education 27, 1 (2017), 1–29.
DOI: arXiv:.Google ScholarCross Ref - [56] . 2016. Coderunner: A tool for assessing computer programming skills. ACM Inroads 7, 1 (
Feb 2016), 47–51.DOI: Google ScholarDigital Library - [57] . 2016. The role of self-regulation in programming problem solving process and success. In Proceedings of the 2016 ACM Conference on International Computing Education Research (ICER’16). ACM, New York, NY, 83–91.
DOI: Google ScholarDigital Library - [58] . 2021. Metacognition and self-regulation in programming education: Theories and exemplars of use. ACM Transactions on Computing Education (
Dec 2021).DOI: Just Accepted. Google ScholarDigital Library - [59] . 2020. Investigating novices’ in situ reflections on their programming process. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE’20). ACM, New York, NY, 149–155.
DOI: Google ScholarDigital Library - [60] . 2018. Ladebug: An online tool to help novice programmers improve their debugging skills. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’18). ACM, New York, NY, 159–164.
DOI: Google ScholarDigital Library - [61] . 2007. Investigating the viability of mental models held by novice programmers. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (SIGCSE’07). ACM, New York, NY, 499–503.
DOI: Google ScholarDigital Library - [62] . 2021. Inside the Mind of a CS Undergraduate TA: A Firsthand Account of Undergraduate Peer Tutoring in Computer Labs. ACM, New York, NY, 502–508. Google ScholarDigital Library
- [63] . 2016. Novice Programmer Errors - Analysis and Diagnostics. Ph.D. Dissertation. University of Kent. https://kar.kent.ac.uk/61340/.Google Scholar
- [64] . 2019. A new look at novice programmer errors. ACM Transactions on Computing Education 19, 4, Article
38 (jul 2019), 30 pages.DOI: Google ScholarDigital Library - [65] . 2008. Debugging: A review of the literature from an educational perspective. Computer Science Education 18, 2 (2008), 67–92.Google ScholarCross Ref
- [66] . 2004. Debugging by Thinking: A Multidisciplinary Approach. Elsevier Digital Press.Google Scholar
- [67] . 2017. RoboBUG: A serious game for learning debugging techniques. In Proceedings of the 2017 ACM Conference on International Computing Education Research. ACM, 93–100.Google ScholarDigital Library
- [68] . 2021. Growing mindsets: Debugging by design to promote students’ growth mindset practices in computer science class. In Proceedings of the 15th International Conference of the Learning Sciences (ICLS’21), , , and (Eds.). https://par.nsf.gov/biblio/10309425.Google Scholar
- [69] . 2010. Pair debugging: A transactive discourse analysis. In Proceedings of the 6th International Workshop on Computing Education Research (ICER’10). ACM, New York, NY, 51–58.
DOI: Google ScholarDigital Library - [70] . 2008. Debugging: The good, the bad, and the quirky – a qualitative analysis of novices’ strategies. In Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education (SIGCSE’08). ACM, New York, NY, 163–167.
DOI: Google ScholarDigital Library - [71] . 2017. The debugging mindset. Queue 15, 1 (
Feb. 2017), 71–90.DOI: Google ScholarDigital Library - [72] . 1986. Fragile knowledge and neglected strategies in novice programmers. In 1st Workshop on Empirical Studies of Programmers on Empirical Studies of Programmers. 213–229.Google Scholar
- [73] . 2018. Metacognitive difficulties faced by novice programmers in automated assessment tools. In Proceedings of the 2018 ACM Conference on International Computing Education Research (ICER’18). ACM, New York, NY, 41–50.
DOI: Google ScholarDigital Library - [74] . 2015. Frequency distribution of error messages. In Proceedings of the 6th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU’15). ACM, New York, NY, 1–8.
DOI: Google ScholarDigital Library - [75] . 2008. A theory-centric real-time assessment of programming. In 2008 8th IEEE International Conference on Advanced Learning Technologies. 139–143.
DOI: Google ScholarDigital Library - [76] . 2003. The Effect of Training Subjects in Self-explanation Strategies on Problem Solving Success in Computer Programming. Marquette University.Google ScholarDigital Library
- [77] . 2003. Learning and teaching programming: A review and discussion. Computer Science Education 13 (2003), 137–172.Google ScholarCross Ref
- [78] . 2005. Interactive, visual fault localization support for end-user programmers. Journal of Visual Languages & Computing 16, 1-2 (2005), 3–40.Google ScholarDigital Library
- [79] . 2000. Cognitive task analysis and innovation of training: The case of structured troubleshooting. Human Factors 42, 1 (2000), 75–86.
DOI: arXiv:.PMID: 10917147. Google ScholarCross Ref - [80] . 2014. On the domain-specificity of mindsets: The relationship between aptitude beliefs and programming practice. IEEE Transactions on Education 57, 3 (2014), 169–174.
DOI: Google ScholarDigital Library - [81] . 2021. Exploring the association between self-regulation of learning and programming learning: A multinational investigation. In 2021 IEEE Frontiers in Education Conference (FIE’21). 1–8.
DOI: Google ScholarDigital Library - [82] . 2018. Language choice in introductory programming courses at Australasian and UK universities. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 852–857.
DOI: Google ScholarDigital Library - [83] . 2019. The error landscape: Characterizing the mistakes of novice programmers. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 538–544.
DOI: Google ScholarDigital Library - [84] . 1986. Learning to program = learning to construct mechanisms and explanations. Communications of the ACM 29, 9 (
Sept. 1986), 850–858.DOI: Google ScholarDigital Library - [85] . 2007. Explaining debugging strategies to end-user programmers. In IEEE Symposium on Visual Languages and Human-Centric Computing, 2007 (VL/HCC’07). IEEE, 127–136.Google Scholar
- [86] . 2019. The Pragmatic Programmer: Your Journey to Mastery. Addison-Wesley Professional.Google Scholar
- [87] . 1994. The think aloud method: A practical guide to modelling cognitive processes.Google Scholar
- [88] . 2016. Identifying novice student programming misconceptions and errors from summative assessments. Journal of Educational Technology Systems 45, 1 (2016), 50–73.
DOI: arXiv:.Google ScholarCross Ref - [89] . 2020. G4D — A treasure hunt game for novice programmers to learn debugging. Smart Learning Environments 7, 1 (2020), 21.Google ScholarCross Ref
- [90] . 2021. A multi-modal investigation of self-regulation strategies adopted by first-year engineering students during programming tasks. In Proceedings of the 17th ACM Conference on International Computing Education Research (ICER’21). ACM, New York, NY, USA, 446–447.
DOI: Google ScholarDigital Library - [91] . 2021. Analysis of a process for introductory debugging. In Australasian Computing Education Conference (ACE’21). ACM, New York, NY, 11–20.
DOI: Google ScholarDigital Library - [92] . 2021. Novice reflections on debugging. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE’21). ACM, New York, NY, 73–79.
DOI: Google ScholarDigital Library - [93] . 2018. An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 344–349.
DOI: Google ScholarDigital Library - [94] . 2012. Analysis of experts’ and novices’ thinking process in program debugging. In Engaging Learners Through Emerging Technologies, , , , , and (Eds.). Springer, Berlin, 122–134.Google ScholarCross Ref
- [95] . 2009. Why Programs Fail: A Guide to Systematic Debugging. Elsevier.Google Scholar
Index Terms
- A Think-Aloud Study of Novice Debugging
Recommendations
What constitutes debugging? An exploratory study of debugging episodes
AbstractWhen debugging, developers engage in activities such as navigating, editing, testing, and inspecting code. Despite being the building blocks of debugging, little is known about how they constitute debugging. To address this gap, we introduce the ...
Debugging Debugging
COMPSACW '11: Proceedings of the 2011 IEEE 35th Annual Computer Software and Applications Conference WorkshopsWhen a program fails to accomplish its intended task, debugging is conducted to identify and remove any bugs. The debugging operation itself is not immune to flaws. Empirical evidence suggests many bugs are found after shipping, which calls into ...
Transparent mutable replay for multicore debugging and patch validation
ASPLOS '13We present Dora, a mutable record-replay system which allows a recorded execution of an application to be replayed with a modified version of the application. This feature, not available in previous record-replay systems, enables powerful new ...
Comments