Computerized adaptive testing (CAT) based on item response theory allows more accurate assessments with fewer questions than the classic paper and pencil (P&P) test. Nonetheless, the CAT construction involves some key questions that, when done properly, can further improve the accuracy and efficiency in estimating the examinees’ abilities. One of the main questions is in regard to choosing the item selection rule (ISR). The classic CAT makes exclusive use of one ISR. However, these rules have differences depending on the examinees’ ability level and on the CAT stage. Thus, the objective of this work is to reduce the dichotomous test size which is inserted in a classic CAT with no significant loss of accuracy in the estimation of the examinee’s ability level. For this purpose, we analyze the ISR performance and then build a personalized item selection process in CAT considering the use of more than one rule. The case study in Mathematics and its Technologies test of the ENEM 2012 shows that the Kullback-Leibler information with a posterior distribution ( KLP ) has better performance in the examinees’ ability estimation when compared with Fisher information ( F ), Kullback-Leibler information ( KL ), maximum likelihood weighted information ( MLWI ), and maximum posterior weighted information ( MPWI ) rules. Previous results in the literature show that CAT using KLP was able to reduce this test size by 46.6 % from the full size of 45 items with no significant loss of accuracy in estimating the examinees’ ability level. In this work, we observe that the F and the MLWI rules performed better on early CAT stages to estimate examinees’ proficiency level with extreme negative and positive values, respectively. With this information, we were able to reduce the same test by 53.3 % using the personalized item selection process, called ALICAT, which includes the best rules working together.

中文翻译：

---

ALICAT：计算机自适应测试中项目选择过程的定制方法

基于项目反应理论的计算机自适应测试 (CAT) 与经典的纸笔 (P&P) 测试相比，可以用更少的问题进行更准确的评估。尽管如此，CAT 构建涉及一些关键问题，如果做得好，可以进一步提高评估考生能力的准确性和效率。主要问题之一是关于选择项目选择规则 (ISR)。经典的 CAT 只使用一个 ISR。但是，这些规则根据考生的能力水平和CAT阶段而有所不同。因此，这项工作的目标是减少插入经典 CAT 的二分测试规模，而不会显着降低对考生能力水平的估计准确性。以此目的，我们分析了 ISR 性能，然后在考虑使用多个规则的情况下在 CAT 中构建个性化的项目选择过程。ENEM 2012数学及其技术测试中的案例研究表明，与Fisher信息（ F ）、Kullback-Leibler信息（ F ）相比，具有后验分布（ KLP ）的Kullback-Leibler信息（ KLP ）在考生的能力估计中具有更好的表现（ KL )、最大似然加权信息 (MLWI) 和最大后验加权信息 (MPWI) 规则。文献中先前的结果表明，使用 KLP 的 CAT 能够将这个测试规模从 45 个项目的完整规模减少 46.6%，而在估计考生能力水平方面的准确性没有显着损失。在这项工作中，我们观察到 F 和 MLWI 规则在早期 CAT 阶段表现更好，分别以极端负值和正值来估计考生的熟练程度。有了这些信息，我们能够使用称为 ALICAT 的个性化项目选择过程将相同的测试减少 53.3%，其中包括协同工作的最佳规则。