Abstract
Digital literacy is one of the essential teacher competencies in the twenty-first century. It is crucial for performing high-quality learning activities supporting science subjects, especially chemistry whereby there are many abstract concepts and limited visualization without digital tools. This study aims to validate the adapted digital literacy questionnaire and investigate pre-service chemistry teachers’ ability to apply digital literacy in the learning context. The digital literacy questionnaire contains 14 items measuring digital technical, digital cognitive, and digital social-emotional aspects. Participants were 885 pre-service chemistry teachers from Kalimantan province in Indonesia selected randomly after finishing six months of the independent program for online learning. The analysis results showed that the partial credit model with the three-dimensional model has better fit parameters than the rating scale model with a three-dimensional model and unidimensional model. The differential item functioning analysis revealed no bias issues based on gender, confirming the measurement invariance statistics achieved. No significant gender differences were found based on pre-service chemistry teachers’ abilities in subdimensions. Pre-service chemistry teachers performed better in the digital technical aspect than in the other aspects of digital literacy. Moreover, the pre-service chemistry teachers’ competencies were categorized and discussed to offer a comprehensive evaluation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of Data and Materials
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Adams, D., Sumintono, B., Mohamed, A., & Mohamad Noor, N. S. (2018). E-Learning readiness among students of diverse backgrounds in a leading Malaysian higher education institution. Malaysian Journal of Learning and Instruction, 15(Number 2), 227–256. https://doi.org/10.32890/mjli2018.15.2.9
Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23. https://doi.org/10.1177/0146621697211001
Adams, R. J., Wu, M. L., Cloney, D., & Wilson, M. R. (2020). ACER ConQuest: Generalised item response modelling software (version 5). Camberwell, Victoria: Australian Council for Educational Research.
Alakpodia, O. N. (2014). Gender differences in computer use skill among students of school of health technology, Ufuoma, Delta State. International Journal of Digital Library Services, 4(4), 1–11.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573. https://doi.org/10.1007/BF02293814
Andrich, D. (2018). Advances in social measurement: A Rasch measurement theory. In Perceived Health and Adaptation in Chronic Disease (pp. 66–91). Routledge.
Ata, R., & Yıldırım, K. (2019). Exploring Turkish pre-service teachers’ perceptions and views of digital literacy. Education Sciences, 9(1), 40. https://doi.org/10.3390/educsci9010040
Azzahra, N. F., & Amanta, F. (2021). Promoting digital literacy skill for students through improved school curriculum. Policy Brief.
Barak, M. (2007). Transition from traditional to ICT-enhanced learning environments in undergraduate chemistry courses. Computers & Education, 48(1), 30–43. https://doi.org/10.1016/j.compedu.2004.11.004
Barak, M., & Dori, Y. J. (2005). Enhancing undergraduate students’ chemistry understanding through project-based learning in an IT environment. Science Education, 89(1), 117–139. https://doi.org/10.1002/sce.20027
Bond, T., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences, 3rd edition. Routledge. https://doi.org/10.4324/9781315814698
Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148
Boone, W. J., & Staver, J. R. (2020). Advances in Rasch analyses in the human sciences. Springer International Publishing. https://doi.org/10.1007/978-3-030-43420-5
Burron, G., & Pegg, J. (2021). Elementary pre-service teachers’ search, evaluation, and selection of online science education resources. Journal of Science Education and Technology, 30(4), 471–483. https://doi.org/10.1007/s10956-020-09891-z
Chan, S.-W., Looi, C.-K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: A Rasch model measurement analysis. Journal of Computers in Education, 8(2), 213–236. https://doi.org/10.1007/s40692-020-00177-2
Cirus, L., & Simonova, I. (2021). Pupils’ digital literacy reflected in teachers’ attitudes towards ICT: Case study of the Czech Republic. SN Computer Science, 2(3), 231. https://doi.org/10.1007/s42979-021-00567-y
Csapó, B., Ainley, J., Bennett, R. E., Latour, T., & Law, N. (2012). Technological issues for computer-based assessment. Assessment and Teaching of 21st Century Skills, 143–230. https://doi.org/10.1007/978-94-007-2324-5_4
Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399–412. https://doi.org/10.1111/bjop.12046
Eckes, T. (2011). Introduction to many-facet Rasch measurement. Peter Lang D. https://doi.org/10.3726/978-3-653-04844-5
Eitemüller, C., Trauten, F., Striewe, M., & Walpuski, M. (2023). Digitalization of multistep chemistry exercises with automated formative feedback. Journal of Science Education and Technology, 32(3), 453–467. https://doi.org/10.1007/s10956-023-10043-2
Ferrari, A. (2013). A framework for developing and understanding digital competence in Europe. IPTS Reports. Luxembourg: European Commission. https://doi.org/10.2788/52966,JRC83167
Fisher, W. P. J. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095.
Geraets, A. A., Nottolini, I. L., Doty, C. M., Wan, T., Chini, J. J., & Saitta, E. K. H. (2021). Preparing GTAs for active learning in the general chemistry lab: Development of an evidence-based rehearsal module for a mixed-reality teaching simulator. Journal of Science Education and Technology, 30(6), 829–840. https://doi.org/10.1007/s10956-021-09923-2
Griffin, P., & Care, E. (2014). Assessment and teaching of 21st century skills: Methods and approach. Springer.
Häkkinen, P., Järvelä, S., Mäkitalo-Siegl, K., Ahonen, A., Näykki, P., & Valtonen, T. (2017). Preparing teacher students for 21st century learning practices (PREP 21): A teachers and teaching: Theory and practice, 23 (1), 25 41. https://doi.org/10.1080/13540602.2016.1203772
IBM Corp. (2017). IBM SPSS statistics for Windows (version 25) [Computer software]. Armonk, NY: IBM SPSS Corp.
Jin, K.-Y., Reichert, F., Cagasan Jr, L. P., de La Torre, J., & Law, N. (2020). Measuring digital literacy across three age cohorts: Exploring test dimensionality and performance differences. Computers & Education, 157, 103968. https://doi.org/10.1016/j.compedu.2020.103968
Kartika, E. F. R., & VH, E. S., & Indriyanti, N. Y. (2021). Development and validation of web-based STEAM online platform to improve learning quality in pre-service chemistry teacher. The Journal of Technology and Science Education, 11(2), 513–525. https://doi.org/10.3926/jotse.1316
Knobel, M., & Lankshear, C. (2006). Digital literacy and digital literacies: Policy, pedagogy and research considerations for education. Nordic Journal of Digital Literacy, 1(1), 12–24. https://doi.org/10.18261/ISSN1891-943X-2006-01-03
Lazonder, A. W., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 86(3), 681–718. https://doi.org/10.3102/0034654315627366
Le, B., Lawrie, G. A., & Wang, J. T. H. (2022). Student self-perception on digital literacy in STEM blended learning environments. Journal of Science Education and Technology, 31(3), 303–321. https://doi.org/10.1007/s10956-022-09956-1
Linacre, J. M. (1989). Many facet Rasch measurement. MESA Press.
Martin, A., & Grudziecki, J. (2006). DigEuLit: Concepts and tools for digital literacy development. Innovation in Teaching and Learning in Information and Computer Sciences, 5(4), 249–267. https://doi.org/10.11120/ital.2006.05040249
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272
Masters, G. N., & Keeves, J. P. (1999). Advances in measurement in educational research and assessment. Pergamon.
Ng, W. (2012a). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065–1078. https://doi.org/10.1016/j.compedu.2012.04.016
Ng, W. (2012b). Empowering scientific literacy through digital literacy and multiliteracies. Nova Science Publishers.
Nguyen, L. A. T., & Habók, A. (2022). Digital literacy of EFL students: An empirical study in Vietnamese universities. Libri, 72(1), 53–66. https://doi.org/10.1515/libri-2020-0165
Nja, C. O., Orim, R. E., Neji, H. A., Ukwetang, J. O., Uwe, U. E., & Ideba, M. A. (2022). Students’ attitude and academic achievement in a flipped classroom. Heliyon, 8(1), e08792. https://doi.org/10.1016/j.heliyon.2022.e08792
Nouri, J., Zhang, L., Mannila, L., & Norén, E. (2020). Development of computational thinking, digital competence and 21st century skills when learning programming in K-9. Education Inquiry, 11(1), 1–17. https://doi.org/10.1080/20004508.2019.1627844
Ojokuku, G. O. (2019). Understanding chemistry for schools and colleges – 3rd edition 2019. CHEM Resource.
Potkonjak, V., Gardner, M., Callaghan, V., Mattila, P., Guetl, C., Petrović, V. M., & Jovanović, K. (2016). Virtual laboratories for education in science, technology, and engineering: A review. Computers & Education, 95, 309–327. https://doi.org/10.1016/j.compedu.2016.02.002
Rahmah, A. (2015). Digital literacy learning system for Indonesian citizen. Procedia Computer Science, 72, 94–101. https://doi.org/10.1016/j.procs.2015.12.109
Ribble, M. S., Bailey, G. D., & Ross, T. W. (2004). Digital citizenship: Addressing appropriate technology behavior. Learning & Leading with Technology, 32(1), 6.
Shidiq, A. S., & Yamtinah, S. (2019). Pre-service chemistry teachers’ attitudes and attributes toward the twenty-first century skills. Journal of Physics: Conference Series, 1157, 042014. https://doi.org/10.1088/1742-6596/1157/4/042014
Soeharto, S., & Csapó, B. (2022). Assessing Indonesian student inductive reasoning: Rasch analysis. Thinking Skills and Creativity, 46, 101132. https://doi.org/10.1016/j.tsc.2022.101132
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
Tondeur, J., Aesaert, K., Pynoo, B., Van Braak, J., Fraeyman, N., & Erstad, O. (2017). Developing a validated instrument to measure preservice teachers’ ICT competencies: Meeting the demands of the 21st century. British Journal of Educational Technology, 48(2), 462–472. https://doi.org/10.1111/bjet.12380
Utama, C., & Nurkamto, J. (2019). Investigating biology pre-service teacher perception of general biology and digital literacy. IOP Conference Series: Earth and Environmental Science, 243(1), 012046. https://doi.org/10.1088/1755-1315/243/1/012046
Vuorikari, R., Kluzer, S., & Punie, Y. (2022). DigComp 2.2, The digital competence framework for citizens: With new examples of knowledge, skills and attitudes. Publications Office of the European Union.
Wilson, M., De Boeck, P., & Carstensen, C. H. (2008). Explanatory item response models: A brief introduction. Assessment of Competencies in Educational Contexts, 91–120.
Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127
Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2–3), 114–128. https://doi.org/10.1016/j.stueduc.2005.05.005
Yanuarto, W. N., & Jaelani, A. (2021). Higher thinking and digital literacy: Empowering technology for pre-service mathematics teacher. Al-Jabar: Jurnal Pendidikan Matematika, 12(2), 329–342. https://doi.org/10.24042/ajpm.v12i2.9637
Zhou, Q., Zhao, Y., Hu, J., Liu, Y., & Xing, L. (2010). Pre-service chemistry teachers’ attitude toward ICT in Xi’an. Procedia - Social and Behavioral Sciences, 9, 1407–1414. https://doi.org/10.1016/j.sbspro.2010.12.342
Zhu, L., Sun, D., Luo, M., Liu, W., & Xue, S. (2023). Investigating pre-service science teachers’ design performance in laboratory class: The inquiry-based design thinking approach. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-023-10050-3
Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1–28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x
Acknowledgements
This paper is dedicated to the memory of Professor Benő Csapó who sadly passed away on 26/06/2023. We would also like to extend our gratitude to Dan Cloney, Ph.D., a Senior Research Fellow in Education Policy and Practice at ACER, for granting us full access to ConQuest version 5 software, which greatly facilitated this publication. Additionally, we profoundly appreciate the time and effort invested by the editors and reviewers in providing insightful feedback that significantly enhanced the quality of our manuscript.
Funding
The study has been made possible by collaboration from the Directorate General of Higher Education (DIKTI), the Chemistry education department at Tanjungpura University, the University of Szeged, and Lambung Mangkurat University. This work was supported by Program Kompetisi Kampus Merdeka (PKKM) Grant for research collaboration [0544/E.E3/PM.00.03/2022], 5th July 2022.
Author information
Authors and Affiliations
Contributions
Hairida Hairida: conceptualization, data curation, formal analysis, investigation, methodology, data visualization, and wrote the paper. Soeharto Soeharto: analysis tools or data and wrote the paper. Charalambous Charalambos: analyzed and interpreted the data and revising final draft. Fitria Arifiyanti: analyzed and interpreted the data and research instrument. Benő Csapó: revising the original draft and analyzed and interpreted the data. Atiek Winarti: performed the experiments and data curation. Eny Enawaty: performed the experiments and data curation. Rahmat Rasmawan: conceived and designed the experiments.
Corresponding author
Ethics declarations
Declarations
Ethical Approval
Ethical research permission was obtained by researchers from the Institutional Review Board at Universitas Tanjungpura, No. 0544002.
Informed Consent
Informed consent was sought for all participants for participation in the project and for their de-identified data to be used for publication. The research was performed in accordance with the ethical standards as laid out in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Statement Regarding Research Involving Human Participants and/or Animals
Not applicable.
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Prof. Csapó Benő has passed away 26/06/2023
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hairida, H., Benő, C., Soeharto, S. et al. Evaluating Digital Literacy of Pre-service Chemistry Teachers: Multidimensional Rasch Analysis. J Sci Educ Technol 32, 643–654 (2023). https://doi.org/10.1007/s10956-023-10070-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10956-023-10070-z