Skip to main content
SpringerLink
Log in

Exploring when learners become aware of their knowledge gaps: Content analyses of learner discussions

  • Original Research
  • Published:
Instructional Science Aims and scope Submit manuscript

Abstract

This study investigates when and how awareness of knowledge gaps (AKG) manifests by observing the problem-solving phase of the educational approach known as problem-solving followed by instruction (PS-I). By comprehensively exploring cognitive and metacognitive process of learners during this phase and categorizing students’ judgements of knowledge structure in relation to AKG, it strengthens the underlying mechanisms of PS-I. With sixteen university students as participants, this study quantitatively and qualitatively analyzes conversations that take place during problem-solving activities. In the analysis, the authors suggest a total of ten cognitive and metacognitive events that occur and six judgements of knowledge structure in relation to AKG. The findings indicate that students spend most of their time solving the problem and seldom evaluate their thoughts; few express awareness of a knowledge gap. The authors discuss the relationships between the judgements of knowledge structure and consider when—and to what extent—students perceive their knowledge gaps. Lastly, the authors bring four learning behaviors (i.e., representing and reflecting on knowledge; recognizing and specifying knowledge gaps) with possible instructional strategies to promote each learning behavior.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The datasets used during the current study are available from the corresponding author on reasonable request.

References

  • Brand, C., Hartmann, C., Loibl, K., & Rummel, N. (2021). Observing or generating solution attempts in problem solving prior to instruction are the preparatory processes comparable. In E. Vries, Y. Hod, & J. Ahn (Eds.), Proceedings of the 15th international conference of the learning sciences - ICLS 2021 (pp. 115–122). International Society of the Learning Sciences.

    Google Scholar 

  • Bransford, J. D., & Stein, B. S. (1993). The IDEAL problem solver (2nd ed.). Freeman.

    Google Scholar 

  • Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology (pp. 161–238). Lawrence Erlbaum Associates.

    Google Scholar 

  • De Grave, W. S., Boshuizen, H. P. A., & Schmidt, H. G. (1996). Problem based learning: Cognitive and metacognitive processes during problem analysis. Instructional Science, 24(5), 321–341.

    Article  Google Scholar 

  • de Jong, T., & Ferguson-Hessler, M. G. (1996). Types and qualities of knowledge. Educational Psychologist, 31(2), 105–113.

    Article  Google Scholar 

  • Deng, Y., Zeng, Z., Jha, K., & Huang, D. (2022). Problem-based cybersecurity lab with knowledge graph as guidance. Journal of Artificial Intelligence and Technology, 2(2), 55–61.

    Google Scholar 

  • Dunlosky, J., & Nelson, T. (1992). Importance of the kind of cue for judgements of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20(4), 374–380.

    Article  Google Scholar 

  • Duschl, R., & Ellenbogen, K. (2009). Argumentation and epistemic criteria: Investigating learners’ reasons for reasons. Educación Química, 20(2), 111–118.

    Article  Google Scholar 

  • Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115.

    Article  Google Scholar 

  • Flynn, L. R., & Goldsmith, R. E. (1999). A short, reliable measure of subjective knowledge. Journal of business research, 46(1), 57–66.

  • Gadgil, S., Nokes-Malach, T. J., & Chi, M. T. (2012). Effectiveness of holistic mental model confrontation in driving conceptual change. Learning and Instruction, 22(1), 47–61.

    Article  Google Scholar 

  • Ge, X., & Land, S. (2003). Scaffolding students’ problem solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology Research and Development, 51(1), 21–38.

    Article  Google Scholar 

  • Gick, M. L. (1986). Problem-solving strategies. Educational Psychologist, 21, 99–120.

    Article  Google Scholar 

  • Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction, 39, 72–87.

    Article  Google Scholar 

  • Grigg, S. J., & Benson, L. C. (2014). A coding scheme for analysing problem-solving processes of first-year engineering students. European Journal of Engineering Education, 39(6), 617–635.

    Article  Google Scholar 

  • Große, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: Can this foster learning outcomes? Learning and Instruction, 17(6), 612–634.

    Article  Google Scholar 

  • Gruber, M. J., & Ranganath, C. (2019). How curiosity enhances hippocampus-dependent memory: The prediction, appraisal, curiosity, and exploration (PACE) framework. Trends in Cognitive Sciences, 23(12), 1014–1025.

    Article  Google Scholar 

  • Hart, H. T. (1965). Memory and the feeling-of knowing experience. Journal of Educational Psychology, 56, 208–216.

    Article  Google Scholar 

  • Hartmann, C., Rummel, N., & Bannert, M. (2022). Using HeuristicsMiner to analyze problem-solving processes: Exemplary use case of a productive-failure study. Journal of Learning Analytics, 9(2), 66–86.

    Article  Google Scholar 

  • Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266.

    Article  Google Scholar 

  • Hmelo-Silver, C. E., Kapur, M., & Hamstra, M. (2018). Learning through problem solving. In F. Fischer, C. Hmelo-Silver, S. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences (pp. 210–220). Routledge.

    Chapter  Google Scholar 

  • Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(1), 65–94.

    Article  Google Scholar 

  • Jordan, M. E., & McDaniel, R. R., Jr. (2014). Managing uncertainty during collaborative problem solving in elementary school teams: The role of peer influence in robotics engineering activity. Journal of the Learning Sciences, 23(4), 490–536.

    Article  Google Scholar 

  • Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424.

    Article  Google Scholar 

  • Kapur, M. (2015). The preparatory effects of problem solving versus problem posing on learning from instruction. Learning and Instruction, 39, 23–31.

    Article  Google Scholar 

  • Kapur, M. (2016). Examining productive failure, productive success, unproductive failure, and unproductive success in learning. Educational Psychologist, 51(2), 289–299.

    Article  Google Scholar 

  • Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. The Journal of the Learning Sciences, 21(1), 45–83.

    Article  Google Scholar 

  • Kapur, M., & Kinzer, C. K. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning, 4(1), 21–46.

    Article  Google Scholar 

  • Koriat, A. (1998). Metamemory: The feeling of knowing and its vagaries. Biological and cognitive aspectsIn M. Sabourin & F. Craik (Eds.), Advances in psychological science (Vol. 2, pp. 461–479). Psychology Press.

    Google Scholar 

  • Krippendorff, K. (1980). Validity in content analysis. In E. Mochmann (Ed.), Computerstrategien fur die Komunikationsanalyse (pp. 69–112). Campus.

    Google Scholar 

  • Lai, C. L., Hwang, G. J., & Tu, Y. H. (2018). The effects of computer-supported self-regulation in science inquiry on learning outcomes, learning processes, and self-efficacy. Educational Technology Research and Development, 66(4), 863–892.

    Article  Google Scholar 

  • Lee, J. (2021). Design and development research on “Knowledge Gap Tracker” prototype for supporting awareness of knowledge gap in productive failure [Unpublished Doctoral dissertation]. Hanyang University

  • Litman, J., Hutchins, T., & Russon, R. (2005). Epistemic curiosity, feeling-of-knowing, and exploratory behaviour. Cognition & Emotion, 19(4), 559–582.

    Article  Google Scholar 

  • Loibl, K., Roll, I., & Rummel, N. (2017). Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review, 29(4), 693–715.

    Article  Google Scholar 

  • Loibl, K., & Rummel, N. (2014). Knowing what you don’t know makes failure productive. Learning and Instruction, 34, 74–85.

    Article  Google Scholar 

  • Loibl, K., Tillema, M., Rummel, N., & van Gog, T. (2020). The effect of contrasting cases during problem solving prior to and after instruction. Instructional Science, 48, 1–22.

    Article  Google Scholar 

  • Mayring, P. (2004). Qualitative content analysis. A Companion to Qualitative Research, 1(2), 159–176.

    Google Scholar 

  • Metcalfe, J., & Finn, B. (2008). Evidence that judgments of learning are causally related to study choice. Psychonomic Bulletin & Review, 15(1), 174–179.

    Article  Google Scholar 

  • Nachtigall, V., Serova, K., & Rummel, N. (2020). When failure fails to be productive: Probing the effectiveness of productive failure for learning beyond STEM domains. Instructional Science, 48(6), 651–697.

    Article  Google Scholar 

  • Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The “delayed-JOL effect.” Psychological Science, 2(4), 267–271.

    Article  Google Scholar 

  • Newman, P. M., & DeCaro, M. S. (2019). Learning by exploring: How much guidance is optimal? Learning and Instruction, 62, 49–63.

    Article  Google Scholar 

  • Owen, E., & Sweller, J. (1985). What do students learn while solving mathematics problems? Journal of Educational Psychology, 77(3), 272.

    Article  Google Scholar 

  • Pluta, W. J., Chinn, C. A., & Duncan, R. G. (2011). Learners’ epistemic criteria for good scientific models. Journal of Research in Science Teaching, 48(5), 486–511.

    Article  Google Scholar 

  • Poole, M. S., & Holmes, M. E. (1995). Decision development in computer-assisted group decision making. Human Communication Research, 22(1), 90–127.

    Article  Google Scholar 

  • Pretz, J. E., Naples, A. J., & Sternberg, R. J. (2003). Recognizing, defining, and representing problems. The Psychology of Problem Solving. https://doi.org/10.1017/CBO9780511615771.002

    Article  Google Scholar 

  • Rawson, K. A., & Dunlosky, J. (2007). Improving students’ self-evaluation of learning for key concepts in textbook materials. European Journal of Cognitive Psychology, 19(4–5), 559–579.

    Article  Google Scholar 

  • Roll, I., Holmes, N. G., Day, J., & Bonn, D. (2012). Evaluating metacognitive scaffolding in guided invention activities. Instructional science, 40, 691–710.

  • Schank, R. C. (1999). Dynamic memory revisited. Cambridge University Press.

    Book  Google Scholar 

  • Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.

    Article  Google Scholar 

  • Smith, J. P., III., DiSessa, A. A., & Roschelle, J. (1994). Misconceptions reconceived: A constructivist analysis of knowledge in transition. The Journal of the Learning Sciences, 3(2), 115–163.

    Article  Google Scholar 

  • Sternberg, R. J. (1986). Intelligence applied: Understanding and increasing your intellectual skills. Harcourt Brace Jovanovich.

    Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  • Trninic, D., Sinha, T., & Kapur, M. (2022). Comparing the effectiveness of preparatory activities that help undergraduate students learn from instruction. Learning and Instruction, 82, 101688.

    Article  Google Scholar 

  • Van Lehn, V. (1988). Towards a theory of impasse-driven learning. In H. Mandl & A. Lesgold (Eds.), Learning issues for intelligent tutoring systems (pp. 31–32). Springer.

    Google Scholar 

  • VanLehn, K. (1999). Rule learning events in the acquisition of a complex skill: An evaluation of cascade. Journal of the Learning Sciences, 8(1), 71–125.

    Article  Google Scholar 

  • Westermann, K., & Rummel, N. (2012). Delaying instruction: Evidence from a study in a university relearning setting. Instructional Science, 40(4), 673–689.

    Article  Google Scholar 

  • Youssef, A., Ayres, P., & Sweller, J. (2012). Using general problem-solving strategies to generate ideas in order to solve geography problems. Applied Cognitive Psychology, 26(6), 872–877.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Korea Advanced Institute of Science & Technology (KAIST), 193 Munji-ro, Yuseong-gu, Daejeon, 34051, South Korea

    Jinju Lee

  2. Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul, 04763, South Korea

    Jinju Lee & Dongsik Kim

  3. Arizona State University, 1000 S Forest Mall, Tempe, AZ, 85281, USA

    Jongchan Park

Authors
  1. Jinju Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jongchan Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongsik Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jongchan Park.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J., Park, J. & Kim, D. Exploring when learners become aware of their knowledge gaps: Content analyses of learner discussions. Instr Sci 52, 171–205 (2024). https://doi.org/10.1007/s11251-023-09654-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11251-023-09654-4

Keywords

Search

Navigation