Abstract

The linguistic complexity of many text-based tests can be a source of construct-irrelevant variance, as test-takers’ performance may be affected by factors that are beyond the focus of the assessment itself, such as reading comprehension skills. This experimental study examined the extent to which the use of animated videos, as opposed to written text, could (i) reduce construct-irrelevant variance attributed to language and reading skills and (ii) impact test-takers’ reactions to a situational judgment test. The results indicated that the variance attributed to construct-irrelevant factors was lower by 9.5% in the animated version of the test. In addition, those who took the animated test perceived it to be more valid, fair, and enjoyable, than those who took the text-based test. They also rated the language used as less difficult to understand. The implications of these findings are discussed.

摘要

许多基于文本的测试的语言复杂性可能是结构无关差异的来源，因为考生的表现可能会受到超出评估本身重点的因素的影响，例如阅读理解技能。本实验研究检查了使用动画视频而不是书面文本的程度可以 (i) 减少归因于语言和阅读技能的与结构无关的差异，以及 (ii) 影响考生对情境判断测试的反应. 结果表明，在测试的动画版本中，归因于结构无关因素的方差降低了 9.5%。此外，参加动画测试的人认为它比参加文本测试的人更有效、更公平、更有趣。他们还认为所使用的语言不太难理解。