ABSTRACT

The present study used process data from a computer-based problem-solving task as indications of behavioural level of test-taking effort, and explored how behavioural item-level effort related to overall test performance and self-reported effort. Variables were extracted from raw process data and clustered. Four distinct clusters were obtained and characterised as high effort, medium effort, low effort, and planner. Regression modelling indicated that among students that failed to solve the task, level of effort invested before giving up positively predicted overall test performance. Among students that solved the task, level of effort was instead weakly negatively related to test performance. A low level of behavioural effort before giving up the task was also related to lower self-reported effort. Results suggest that effort invested before giving up provides information about test-takers’ motivation to spend effort on the test. We conclude that process data could augment existing methods of assessing test-taking effort.

摘要

本研究使用来自基于计算机的问题解决任务的过程数据作为应试努力的行为水平的指示，并探讨了行为项目水平的努力与整体测试表现和自我报告的努力之间的关系。从原始过程数据中提取变量并进行聚类。获得了四个不同的集群，并将其表征为高努力、中等努力、低努力和计划者。回归模型表明，在未能解决任务的学生中，在放弃之前投入的努力水平积极预测了整体测试表现。在完成任务的学生中，努力程度与考试成绩呈弱负相关。放弃任务前的低水平行为努力也与较低的自我报告努力有关。结果表明，放弃前投入的努力提供了有关应试者在考试上花费精力的动机的信息。我们得出的结论是，过程数据可以增强评估应试工作的现有方法。