Skip to main content
SpringerLink
Log in

The impact of thinking-aloud peer assessment (TAPA) on the development of high order thinking skills (HOTS) in math e-learning

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Assessment plays a crucial role in the learning process and significantly impacts students’ understanding of tasks and their engagement. With the increasing popularity of e-learning, cognitive assessment methods are becoming more prevalent, and they can help develop students’ high-order thinking skills (HOTS). Thinking-Aloud Peer Assessment (TAPA) is one such method that enables students to provide feedback on their peers’ assignments while articulating their thinking processes. This study aims to investigate the impact of TAPA on HOTS in elementary school students. The four-week TAPA method was used to assess the students’ math lessons in e-learning. Pre- and post-tests using the Ricketts Critical Thinking Questionnaire and Cassidy and Long Problem-Solving Questionnaire were conducted to measure the method’s effect on the student’s HOTS. The questionnaires’ reliability was calculated using Cronbach’s alpha coefficient, resulting in scores of 0.84 and 0.81, respectively. Data analysis involved an analysis of variance test, revealing a significant difference between the experimental and control groups in all questionnaire components. The results suggest that the TAPA method can positively impact the development of students’ HOTS. Overall, the study highlights the potential of TAPA as a valuable tool for fostering HOTS. By addressing its limitations and building upon its strengths, educators can leverage this method to better equip students for their cognitive development in eLearning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aboagye, E., Yawson, J. A., & Appiah, K. N. (2021). COVID-19 and E-learning: The challenges of students in tertiary institutions. Social Education Research, 1–8.

  • Almarabeh, T. (2014). Students’ perceptions of E-learning at the University of Jordan. International Journal of Emerging Technologies in Learning, 9(3). https://doi.org/10.3991/ijet.v9i3.3347

  • Araghian Mojarad, F., Sanagoo, A., & Jouybari, L. (2018). Title: High order teaching: Necessity Approach. Nursing Education IJME, 18(0), 470–471. http://ijme.mui.ac.ir/article-1-4823-fa.html

    Google Scholar 

  • Babapoor, K. J., & Ezheyi, M. J. (2002). The study of relationship between problem solving styles, conflict solving styles, and mental health among Tarbiat modarres university students. Journal of Psychology, 7, 3–16. (in Persian).

    Google Scholar 

  • Badali, M., Mirzaei, P., Shafieifar, N., & Assadi, H. (2022). Problem-based E-learning on students’ learning and academic efficacy during the COVID-19 pandemic. Iranian Journal of Learning & Memory, 4(16), 63–70. https://journal.iepa.ir/article_148575.html

    Google Scholar 

  • Banihashem, S. K., Rezaei, J., Badali, M., & Dana, A. (2014). The impact of using blended learning on students’ creativity. Journal of Innovation and Creativity in Human Science, 4(1), 113–128.

    Google Scholar 

  • Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. ASCD.

  • Cassidy, T., & Long, C. (1996). Problem-solving style, stress and psychological illness: Development of a multifactorial measure. British Journal of Clinical Psychology, 35(2), 265–277.

    Article  CAS  PubMed  Google Scholar 

  • Chairil Hikayat, S., Hairun, Y., & Suharna, H. (2020). Design of realistic mathematics education approach to improve critical thinking skills. Universal Journal of Educational Research, 8(6), 2232–2244.

    Article  Google Scholar 

  • Dehghani, M., Mirdoraghi, F., & Pakmehr, H. (2011). The role of graduate students’ achievement goals in their critical thinking disposition. Procedia-Social and Behavioral Sciences, 15, 2426–2430.

    Article  Google Scholar 

  • Dike, J., Mumuni, A., & Worokwo, C. (2017). Metacognitive Teaching Strategies on Secondary School Students Academic performance. International Journal of Computational Engineering Research, 7(1), 14–20. www.ijceronline.com.

    Google Scholar 

  • Divjak, B., Svetec, B., Horvat, D., & Kadoić, N. (2023). Assessment validity and learning analytics as prerequisites for ensuring student-centred learning design. British Journal of Educational Technology, 54(1), 313–334.

    Article  Google Scholar 

  • Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer Assessment on Academic performance: A Meta-analysis of Control Group studies. Educational Psychology Review, 32(2), 481–509. https://doi.org/10.1007/s10648-019-09510-3

    Article  Google Scholar 

  • Fajari, L. E. W. (2021). Open-ended questions to assess critical-thinking skills in Indonesian Elementary School. International Journal of Instruction, 14(1), 615–630.

    Article  Google Scholar 

  • Halili, S. H. (2019). Technological advancements in education 4.0. The Online Journal of Distance Education and E-Learning, 7(1), 63–69.

    Google Scholar 

  • Hatami, Rezaei, E., & maleki (2018). Measurement and evaluation in e-learning. Publications of Tarbiat Modares University. (in Persian).

  • Houshmandi, S., Rezaei, E., Hatami, J., & Molaei, B. (2019). E-learning readiness among faculty members of medical sciences universities and provide strategies to improve it. Research and Development in Medical Education, 8(2), 105–112. https://doi.org/10.15171/rdme.2019.020

    Article  Google Scholar 

  • Hoveida, R., & Davarpanah, S. H. A. (2019). Role of teacher Leadership in improving students’ problem solving skills with the Mediating Role of Creative thinking. New Educational Approaches, 14(1), 83–100.

    Google Scholar 

  • Hyytinen, H., Ursin, J., Silvennoinen, K., Kleemola, K., & Toom, A. (2021). The dynamic relationship between response processes and self-regulation in critical thinking assessments. Studies in Educational Evaluation, 71, 101090.

    Article  Google Scholar 

  • Jooyani, M., Hosseininasab, S. D., & Panahali, A. (2021). A comparative study on the effectiveness of Metacognitive and Problem solving training on the awareness of teaching – learning strategies of High School Gifted Male students in Ardabil. Iranian Journal of Learning & Memory, 4(13), 20–28. https://journal.iepa.ir/article_131165.html

    Google Scholar 

  • Kosasih, A., Supriyadi, T., Firmansyah, M. I., & Rahminawati, N. (2022). Higher-order thinking skills in Primary School: Teachers’ perceptions of Islamic Education. Journal of Ethnic and Cultural Studies, 9(1), 56–76. https://doi.org/10.29333/ejecs/994

    Article  Google Scholar 

  • Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., & Yan, W. (2016). The effectiveness of blended learning in Health professions: Systematic review and Meta-analysis. Journal of Medical Internet Research, 18(1), e2. https://doi.org/10.2196/jmir.4807

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Mohamadi (2001). & sahebi, ali. Investigating the problem solving style of depressed people and comparing it with normal people. Journal of Cognitive Sciences, 1(1), 24–42. (in Persian).

  • Mousazadeh, N., Momennasab, M., Nia, S., Nazari, H., R., & Hajihosseini, F. (2021). Effective factors in critical thinking disposition in nursing students. Education Research International, 2021, 1–6.

  • Muhayimana, T., Kwizera, L., & Nyirahabimana, M. R. (2022). Using Bloom’s taxonomy to evaluate the cognitive levels of primary leaving English exam questions in Rwandan schools. Curriculum Perspectives, 42(1), 51–63. https://doi.org/10.1007/s41297-021-00156-2

    Article  PubMed Central  Google Scholar 

  • Nachiappan, S., Ahmad Damahuri, A., Ganaprakasam, C., & Suffian, S. (2018). Application of higher Order thinking skills (HOTS) in teaching and learning through communication component and spiritual, attitudes and values component in preschool. Southeast Asia Early Childhood Journal, 7, 24–32. https://doi.org/10.37134/saecj.vol7.3.2018

    Article  Google Scholar 

  • Noh, T., Jeon, K., & Huffman, D. (2005). The effects of thinking aloud pair problem solving on high school students’ chemistry problem-solving performance and verbal interactions. Journal of Chemical Education, 82(10), 1558.

    Article  ADS  Google Scholar 

  • Pakmehr, mirdoraghi, fateme, ghanaie, ali, & karami (2013). Validation, validation and factor analysis of Ricketts critical thinking tendency scale in high school. Educational measurement, 4(11), 33–53. https://www.sid.ir/fa/journal/ViewPaper.aspx?ID=202538 (in Persian).

  • Pate, M. L., & Miller, G. (2011). Effects of think-aloud pair problem solving on secondary-level students’ performance in Career and Technical Education courses. Journal of Agricultural Education, 52(1), 120–131.

    Article  Google Scholar 

  • Rababa, N. (2021). The effect of e-learning in developing high thinking skills. International Journal of Data and Network Science, 5(1), 43–46. https://doi.org/10.5267/j.ijdns.2020.11.004

    Article  Google Scholar 

  • Ricketts, J. C. (2003). Critical thinking skills of selected youth leaders: The efficacy of leadership development, critical thinking dispositions, and student academic performance (Doctoral dissertation, University of Florida)

  • Setyowati, R., Sarwanto, & Muzzazinah (2021). How Students’s Higher Order Thinking Skills through E-Learning during the Covid-19 Pandemic? What does it have to do with University? IOP Conference Series: Earth and Environmental Science, 1808(1). https://doi.org/10.1088/1742-6596/1808/1/012032

  • Shida, N., Sharifah, Hanifah, N., & Halim, A. (2019). The influence of e-learning towards metacognitive enhancement in mathematical problem solving. International Journal of Emerging Technologies in Learning, 14(20), 165–173. https://doi.org/10.3991/ijet.v14i20.11466

    Article  Google Scholar 

  • Siti Rohmi Yuliati, & Ika Lestari. (2018). Higher-order thinking skills (HOTS) analysis of students in solving HOTS question in higher education. PERSPEKTIF Ilmu Pendidikan, 32(2), 181–188.

    Article  Google Scholar 

  • Sun, S., Li, T., & Zhou, X. (2021). Effects of thinking aloud on cognitive effort in translation. Linguistica Antverpiensia New Series – Themes in Translation Studies, 19, 132–151. https://doi.org/10.52034/lanstts.v19i0.556

    Article  Google Scholar 

  • Tanudjaya, C. P., & Doorman, M. (2020). Examining higher order thinking in Indonesian lower secondary mathematics classrooms. In Journal on Mathematics Education (Vol. 11, Issue 2, pp. 277–300). Sriwijaya University. https://doi.org/10.22342/jme.11.2.11000.277-300

  • Van Zundert, M., Sluijsmans, D., & Van Merriënboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20(4), 270–279.

    Article  Google Scholar 

  • Wage, K. E., Buck, J. R., Nelson, J. K., & Hjalmarson, M. A. (2021). What were they thinking? Refining conceptual assessments using think-aloud problem solving. IEEE Signal Processing Magazine, 38(3), 85–93.

    Article  Google Scholar 

  • Zaraii Zavaraki, E., & Rezaei, E. (2011). The impact of using e-portfolio on attitude, motivation and academic achievement of students in Khajeh Nasirddin Toosi university’s e-Learning Center. Quarterly of Educational Measurement, 2(5), 1–36.

    Google Scholar 

  • Zargush, S., Gheisarbeigi, Z., & Zargush, F. (2019). Investigating critical thinking in students. New Achievements in Humanities Studies, 33(3), 38–48. https://www.noormags.ir/view/en/articlepage/1742636 (in Persian).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Educational Technology in Medical Sciences, Smart University of Medical Sciences, Tehran, Iran

    Eisa Rezaei

  2. Department of Education, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Sepide Sadat Beheshti Shirazi

Authors
  1. Eisa Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sepide Sadat Beheshti Shirazi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation and data collection was collected Sepide sadat beheshti shirazi. Data analysis was performed by Eisa Rezaei and Sepide sadat beheshti shirazi. The manuscript was prepared by Sepide sadat beheshti shirazi and edited by Eisa Rezaei. The technical editing was done by Eisa Rezaei.

Corresponding author

Correspondence to Eisa Rezaei.

Ethics declarations

Statements and declarations

This study was a part of an MA thesis titled “Exploring the Impact of Utilizing a Cognitive Evaluation Approach (Think Aloud Peer Assessment) in E-learning on the Enhancement of High-level Thinking Skills in Primary Students.” The research was conducted in compliance with all relevant laws and regulations and adhered to the ethical principles and guidelines for research involving human subjects. The study received approval from the research council of the Science and Research branch at Islamic Azad University, and all participants provided informed consent prior to participation. Any potential ethical concerns related to this research, including issues of privacy and confidentiality, were carefully considered and addressed.

To enhance the transparency and reproducibility of our research, we have included the DOI for data availability beheshti, sepide (2023), “academic honesty”, Mendeley Data, V1, doi: https://doi.org/10.17632/wgjt7ryxw9.1https://data.mendeley.com/datasets/wgjt7ryxw9/1.

Competing interests

On behalf of all the authors, the corresponding author declare that there are no competing interests to be declared.

Ethical considerations

The study was conducted with careful attention to ethical issues. All student participants provided informed consent for their assessment work to be used as part of the research. To protect student privacy and confidentiality, all data was de-identified before analysis. Students were informed that their involvement was voluntary and they could withdraw from the study without penalty. As the TAPA method involved students reviewing and providing feedback on each other’s work, issues of privacy, confidentiality and proper conduct were emphasized to students. No identifying information about students was revealed in the feedback they provided. The University’s research ethics committee approved the study before commencement.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezaei, E., Beheshti Shirazi, S.S. The impact of thinking-aloud peer assessment (TAPA) on the development of high order thinking skills (HOTS) in math e-learning. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12607-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10639-024-12607-0

Keywords

Search

Navigation