Abstract
While the importance of feedback in education is well established, the effects of structured knowledge in collaborative academic writing remains uncertain. This study introduces an academic writing feedback tool that combines structured knowledge mining, analysis, and visualization. An empirical experiment was conducted in a second-year university class with fifty-five students to examine the impact of the tool on different writing phases. Multiple data sources (i.e., scores, peer comments, discussions, surveys, and interviews) are collected and analyzed using a mixed-method approach. The findings demonstrate that structured knowledge feedback significantly improves specific metrics used to assess academic writing, leading to an overall enhancement in writing quality. The intervention also influences students’ engagement, both behaviorally and cognitively, during online discussions and peer comment phases. Moreover, all students exhibited a positive perception of the writing feedback tool and considered peer comments as the most beneficial collaborative phase when structured knowledge intervention was employed. However, their preferences regarding the presentation form of feedback varied. Finally, the study provides implications for the development and research of NLP-powered (Natural Language Processing) feedback tools. These insights aim to inspire future studies on collaborative academic writing, emphasizing the potential of structured knowledge feedback in fostering effective writing practices.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets used and/or analysed during the current study are available from the first author on reasonable request.
References
Berndt, M., Strijbos, J.-W., & Fischer, F. (2022). Impact of sender and peer-feedback characteristics on performance, cognitive load, and mindful cognitive processing. Studies in Educational Evaluation, 75, 101197. https://doi.org/10.1016/j.stueduc.2022.101197
Chang, C.-C., Tseng, K.-H., & Lou, S.-J. (2012). A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students. Computers & Education, 58(1), 303–320. https://doi.org/10.1016/j.compedu.2011.08.005
Chen, C.-M., Li, M.-C., & Liao, C.-K. (2022). Developing a collaborative writing system with visualization interaction network analysis to facilitate online learning performance. Interactive Learning Environments, 0(0), 1–20. https://doi.org/10.1080/10494820.2022.2028851
Chen, C.-M., & Tsao, H.-W. (2021). An instant perspective comparison system to facilitate learners’ discussion effectiveness in an online discussion process. Computers & Education,164, 104037. https://doi.org/10.1016/j.compedu.2020.104037
Draper, N. R. (2007). Rotatable designs and rotatability. In F. Ruggeri, R. S. Kenett, & F. W. Faltin (Eds.), Encyclopedia of Statistics in Quality and Reliability (1st ed.). Wiley. https://doi.org/10.1002/9780470061572.eqr034
Ellis, R. (2021). The effects of pre-task planning on second language writing: A systematic review of experimental studies. Chinese Journal of Applied Linguistics, 44(2), 131–165. https://doi.org/10.1515/CJAL-2021-0009
Hoang, D. T. N., & Hoang, T. (2022). Enhancing EFL students’ academic writing skills in online learning via Google Docs-based collaboration: A mixed-methods study. Computer Assisted Language Learning, 0(0), 1–23. https://doi.org/10.1080/09588221.2022.2083176
Horstmanshof, L., & Brownie, S. (2013). A scaffolded approach to discussion board use for formative assessment of academic writing skills. Assessment & Evaluation in Higher Education, 38(1), 61–73. https://doi.org/10.1080/02602938.2011.604121
Kerman, N. T., Noroozi, O., Banihashem, S. K., Karami, M., & Biemans, H. J. A. (2022). Online peer feedback patterns of success and failure in argumentative essay writing. Interactive Learning Environments, 0(0), 1–13. https://doi.org/10.1080/10494820.2022.2093914
Knight, S., Shibani, A., Abel, S., Gibson, A., & Ryan, P. (2020). Acawriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research, 12(1), 141–186. https://doi.org/10.17239/jowr-2020.12.01.06
Li, J., & Mak, L. (2022). The effects of using an online collaboration tool on college students’ learning of academic writing skills. System, 105, 102712. https://doi.org/10.1016/j.system.2021.102712
Li, X., Ouyang, F., & Chen, W. (2022). Examining the effect of a genetic algorithm-enabled grouping method on collaborative performances, processes, and perceptions. Journal of Computing in Higher Education, 34(3), 790–819. https://doi.org/10.1007/s12528-022-09321-6
Li, X., Ouyang, F., Liu, J., Wei, C., & Chen, W. (2023a). Examining the effects of a real-time, knowledge-aware tool for academic writing assessment. Journal of Educational Computing Research, 61(6), 1143–1174. https://doi.org/10.1177/07356331221136889.
Li, X., Wei, C., Jiang, Z., Meng, W., Ouyang, F., Zhang, Z., & Chen, W. (2023b). EduNER: A Chinese named entity recognition dataset for education research. Neural Computing and Applications, 35(24), 17717–17731. https://doi.org/10.1007/s00521-023-08635-5.
Lin, C.-C., & Tsai, C.-C. (2012). Participatory learning through behavioral and cognitive engagements in an online collective information searching activity. International Journal of Computer-Supported Collaborative Learning,7(4), 543–566. https://doi.org/10.1007/s11412-012-9160-1
Lin, P.-C., Hou, H.-T., & Chang, K.-E. (2020). The development of a collaborative problem solving environment that integrates a scaffolding mind tool and simulation-based learning: An analysis of learners’ performance and their cognitive process in discussion. Interactive Learning Environments, 1–18. https://doi.org/10.1080/10494820.2020.1719163
Liu, M., Liu, L., & Liu, L. (2018). Group awareness increases student engagement in online collaborative writing. The Internet and Higher Education, 38, 1–8. https://doi.org/10.1016/j.iheduc.2018.04.001
Liu, N.-F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279–290. https://doi.org/10.1080/13562510600680582
López-Pellisa, T., Rotger, N., & Rodríguez-Gallego, F. (2021). Collaborative writing at work: Peer feedback in a blended learning environment. Education and Information Technologies, 26(1), 1293–1310. https://doi.org/10.1007/s10639-020-10312-2
Luaces, O., Díez, J., & Bahamonde, A. (2018). A peer assessment method to provide feedback, consistent grading and reduce students’ burden in massive teaching settings. Computers & Education, 126, 283–295. https://doi.org/10.1016/j.compedu.2018.07.016
Nunes, A., Cordeiro, C., Limpo, T., & Castro, S. L. (2022). Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020. Journal of Computer Assisted Learning, 38(2), 599–620. https://doi.org/10.1111/jcal.12635
Ouyang, F., Chen, S., & Li, X. (2021). Effect of three network visualizations on students’ social-cognitive engagement in online discussions. British Journal of Educational Technology, 52(6), 2242–2262. https://doi.org/10.1111/bjet.13126
Ouyang, F., & Scharber, C. (2017). The influences of an experienced instructor’s discussion design and facilitation on an online learning community development: A social network analysis study. The Internet and Higher Education, 35, 34–47. https://doi.org/10.1016/j.iheduc.2017.07.002
Ouyang, F., Wu, M., Zhang, L., Xu, W., Zheng, L., & Cukurova, M. (2023). Making strides towards AI-supported regulation of learning in collaborative knowledge construction. Computers in Human Behavior, 142, 107650. https://doi.org/10.1016/j.chb.2023.107650
Peng, Y., Li, Y., Su, Y., Chen, K., & Jiang, S. (2022). Effects of group awareness tools on students’ engagement, performance, and perceptions in online collaborative writing: Intergroup information matters. The Internet and Higher Education, 53, 100845. https://doi.org/10.1016/j.iheduc.2022.100845
Shang, H.-F. (2022). Exploring online peer feedback and automated corrective feedback on EFL writing performance. Interactive Learning Environments, 30(1), 4–16. https://doi.org/10.1080/10494820.2019.1629601
Storch, N., & Wigglesworth, G. (2010). Learners’ processing, uptake, and retention of corrective feedback on writing: Case studies. Studies in Second Language Acquisition, 32(2), 303–334. https://doi.org/10.1017/S0272263109990532
Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019). Digital support for academic writing: A review of technologies and pedagogies. Computers & Education, 131, 33–48. https://doi.org/10.1016/j.compedu.2018.12.005
Tan, J. S. H., & Chen, W. (2022). Peer feedback to support collaborative knowledge improvement: What kind of feedback feed-forward? Computers & Education, 187, 104467. https://doi.org/10.1016/j.compedu.2022.104467
Villalon, J., & Calvo, R. A. (2011). Concept maps as cognitive visualizations of writing assignments. Journal of Educational Technology & Society, 14(3), 16–27. https://www.jstor.org/stable/jeductechsoci.14.3.16.
Wambsganss, T., Janson, A., & Leimeister, J. M. (2022). Enhancing argumentative writing with automated feedback and social comparison nudging. Computers & Education, 191, 104644. https://doi.org/10.1016/j.compedu.2022.104644
Wilson, J., & Czik, A. (2016). Automated essay evaluation software in english language arts classrooms: Effects on teacher feedback, student motivation, and writing quality. Computers & Education, 100, 94–109. https://doi.org/10.1016/j.compedu.2016.05.004
Xiang, X., Yuan, R., & Yu, B. (2022). Implementing assessment as learning in the L2 writing classroom: A Chinese case. Assessment & Evaluation in Higher Education, 47(5), 727–741. https://doi.org/10.1080/02602938.2021.1965539
Yu, S. (2021). Giving genre-based peer feedback in academic writing: Sources of knowledge and skills, difficulties and challenges. Assessment and Evaluation in Higher Education, 46(1), 36–53. https://doi.org/10.1080/02602938.2020.1742872
Yu, S., Jiang, L., & Zhou, N. (2020). Investigating what feedback practices contribute to students’ writing motivation and engagement in Chinese EFL context: A large scale study. Assessing Writing, 44, 100451. https://doi.org/10.1016/j.asw.2020.100451
Zheng, L., Fan, Y., Chen, B., Huang, Z., LeiGao, & Long, M. (2023). An AI-enabled feedback-feedforward approach to promoting online collaborative learning. Education and Information Technologies, 0(0), 1–22. https://doi.org/10.1007/s10639-023-12292-5
Zhu, C. (2012). Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology & Society, 15(1), 127–136. Retrieved March 8, 2024, from https://www.jstor.org/stable/jeductechsoci.15.1.127.
Acknowledgements
The authors would like to thank all students and educational experts who participated in this study.
Funding
The authors acknowledge the financial support from the National Natural Science Foundation of China (62177041), the 2021 Key Research and Development Plan of Zhejiang Province (2021C03140), and the Education Science Planning Project of Zhejiang Province titled “A multimodal online collaborative learning evaluation and intervention study based on learning analytics” (2023SCG021).
Author information
Authors and Affiliations
Contributions
Xu Li: Conceptualization, Methodology, Software, Writing—Original Draft. Shiyan Jiang: Methodology, Writing—Review & Editing. Yue Hu: Conceptualization, Data Curation. Xiaoxiao Feng: Data Curation, Investigation. WenZhi Chen: Project administration, Supervision. Fan Ouyang: Conceptualization, Writing—Review & Editing.
Corresponding authors
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, X., Jiang, S., Hu, Y. et al. Investigating the impact of structured knowledge feedback on collaborative academic writing. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12560-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10639-024-12560-y