Abstract
Traditional measurement models assume that all item responses correlate with each other only through their underlying latent variables. This conditional independence assumption has been extended in joint models of responses and response times (RTs), implying that an item has the same item characteristics fors all respondents regardless of levels of latent ability/trait and speed. However, previous studies have shown that this assumption is violated in various types of tests and questionnaires and there are substantial interactions between respondents and items that cannot be captured by person- and item-effect parameters in psychometric models with the conditional independence assumption. To study the existence and potential cognitive sources of conditional dependence and utilize it to extract diagnostic information for respondents and items, we propose a diffusion item response theory model integrated with the latent space of variations in information processing rate of within-individual measurement processes. Respondents and items are mapped onto the latent space, and their distances represent conditional dependence and unexplained interactions. We provide three empirical applications to illustrate (1) how to use an estimated latent space to inform conditional dependence and its relation to person and item measures, (2) how to derive diagnostic feedback personalized for respondents, and (3) how to validate estimated results with an external measure. We also provide a simulation study to support that the proposed approach can accurately recover its parameters and detect conditional dependence underlying data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In fact, models for aberrant behavior do not assume conditional independence and they capture local dependence by a mixture of solution and aberrant processes. However, a solution process component of these models is motivated by van der Linden ’s model and the conditional independence assumption.
Variability in drift rate for the DIRT model was first introduced by Tuerlinckx and De Boeck (2005), and then, its effect on conditional dependence was further studied in detail by Kang, De Boeck, and Ratcliff (2022), along with a new introduction of variability in starting point. This kind of variability extension was motivated by a cognitive process model called the Ratcliff diffusion model (Ratcliff, 1978; Ratcliff & McKoon, 2008)
Note that these correlation values can vary a lot across data and they do not necessarily show general relationships between the person-wise cognitive components and the person-wise data-based measures. For example, in the ACT data (used in Example 3), \(\hat{\theta }_p\) has correlations of 0.843 and \(-0.589\) and \(\log (\hat{\gamma }_p)\) has correlations of 0.518 and 0.897, with the person-wise response proportions and median RTs, respectively.
The verbal analogies test used to collect the current data is in corporate use and we do not have full permission to provide detailed information on the items.
References
Batchelder, W. H., & Bershad, N. J. (1979). The statistical analysis of a thurstonian model for rating chess players. Journal of Mathematical Psychology, 19(1), 39–60. https://doi.org/10.1016/0022-2496(79)90004-X
Bolsinova, M., De Boeck, P., & Tijmstra, J. (2017). Modelling conditional dependence between response and accuracy. Psychometrika, 82(4), 1126–1148. https://doi.org/10.1007/s11336-016-9537-6
Bolsinova, M., & Maris, G. (2016). A test for conditional independence between response time and accuracy. British Journal of Mathematical and Statistical Psychology, 69(1), 62–79. https://doi.org/10.1111/bmsp.12059
Bolsinova, M., & Molenaar, D. (2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9(1525), 1–12. https://doi.org/10.3389/fpsyg.2018.01525
Bolsinova, M., & Tijmstra, J. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71(1), 13–38. https://doi.org/10.1111/bmsp.12104
Bolsinova, M., Tijmstra, J., & Molenaar, D. (2017). Response moderation models for conditional dependence between response time and response accuracy. British Journal of Mathematical and Statistical Psychology, 70, 257–279. https://doi.org/10.1111/bmsp.12076
Bolsinova, M., Tijmstra, J., Molenaar, D., & De Boeck, P. (2017). Conditional dependence between response time and accuracy: An overview of its possible sources and directions for distinguishing between them. Frontiers in Psychology, 8, 202. https://doi.org/10.3389/fpsyg.2017.00202
Borg, I., & Gorenen, P. (2005). Modern multidimensional scaling: Theory and applications (2 (nd). New York: Springer.
Chen, H., De Boeck, P., Grady, M., Yang, C.-L., & Waldschmidt, D. (2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16–23. https://doi.org/10.1016/j.intell.2018.04.001
Coombs, C. H. (1964). A theory of data. Wiley.
De Boeck, P., Chen, H., & Davison, M. (2017). Spontaneous and imposed speed of cognitive test responses. British Journal of Mathematical and Statistical Psychology, 70(2), 225–237. https://doi.org/10.1111/bmsp.12094
DiTrapani, J., Jeon, M., De Boeck, P., & Partchev, I. (2016). Attempting to differentiate fast and slow intelligence: Using generalized item response trees to examine the role of speed on intelligence tests. Intelligence, 56, 82–92. https://doi.org/10.1016/j.intell.2016.02.012
Elo, A. (1978). The rating of chess players, past and present. London: Batsford.
Friel, N., Rastelli, R., Wyse, J., & Raftery, A. E. (2016). Interlocking directorates in irish companies using a latent space model for bipartite networks. Proceedings of the National Academy of Sciences, 113(24), 6629–6634. https://doi.org/10.1073/pnas.1606295113
Gelman, A. (1996). Inference and monitoring convergence. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov chain monte carlo in practice (pp. 131–143). CRC Press.
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013).
Goldhammer, F., Naumann, J., & Greiff, S. (2015). More is not always better: The relation between item response and item response time in raven’s matrices. Journal of Intelligence, 3(1), 21–40. https://doi.org/10.3390/jintelligence3010021
Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rólke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106(3), 608–626. https://doi.org/10.1037/a0034716
Handcock, M. S., Raftery, A. E., & Tantrum, J. M. (2007). Model-based clustering for social networks. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(2), 301–354. https://doi.org/10.1111/j.1467-985X.2007.00471.x
Hoff, P. D., Raftery, A. E., & Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97(460), 1090–1098. https://doi.org/10.1198/016214502388618906
Ishwaran, H., & Rao, J. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33(2), 730–773. https://doi.org/10.1214/009053604000001147
Jeon, M., Jin, I., Schweinberger, M., & Baugh, S. (2021). Mapping unobserved item-respondent interactions: A latent space item response model with interaction map. Psychometrika, 86, 378–403. https://doi.org/10.1007/s11336-021-09762-5
Jin, I., & Jeon, M. (2019). A doubly latent space joint model for local item and person dependence in the analysis of item response data. Psychometrika, 84, 236–260. https://doi.org/10.1007/s11336-018-9630-0
Kang, I., De Boeck, P., & Partchev, I. (2022). A randomness perspective on intelligence processes. Intelligence, 91, 101632. https://doi.org/10.1016/j.intell.2022.101632
Kang, I., De Boeck, P., & Ratcliff, R. (2022). Modeling conditional dependence of response accuracy and response time with the diffusion item response theory model. Psychometrika, Advance Online Publication.https://doi.org/10.1007/s11336-021-09819-5
Lewandowski, D., Kurowicka, D., & Joe, H. (2009). Generating random correlation matrices based on vines and extended onion method. Journal of Multivariate Analysis, 100(9), 1989–2001. https://doi.org/10.1016/j.jmva.2009.04.008
Lu, J., & Wang, C. (2020). A response time process model for not-reached and omitted items. Journal of Educational Measurement, 57(4), 584–620. https://doi.org/10.1111/jedm.12270
Lu, J., Wang, C., & Shi, N. (2021). A mixture response time process model for aberrant behaviors and item nonresponses. Multivariate Behavioral Research, Ahead-of-Print.https://doi.org/10.1080/00273171.2021.1948815
Man, K., Harring, J. R., Jiao, H., & Zhan, P. (2019). Joint modeling of compensatory multidimensional item responses and response times. Applied Psychological Measurement, 43(8), 639–654. https://doi.org/10.1177/0146621618824853
Meng, X.-B., Tao, J., & Chang, H.-H. (2015). A conditional joint modeling approach for locally dependent item responses and response times. Journal of Educational Measurement, 52(1), 1–27. https://doi.org/10.1111/jedm.12060
Mitchell, T. J., & Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404), 1023–1032.
Molenaar, D. , Tuerlinckx, F. , & van der Maas, H. L. J. (2015). Fitting diffusion item response theory models for responses and response times using the r package diffirt. Journal of Statistical Software, 66(4), 1–34. https://doi.org/10.18637/jss.v066.i04
Partchev, I., & De Boeck, P. (2012). Can fast and slow intelligence be differentiated? Intelligence, 40(1), 23–32. https://doi.org/10.1016/j.intell.2011.11.002
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108.
Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922.
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24(1), 3–32. https://doi.org/10.1177/01466216000241001
Roberts, J. S., & Laughlin, J. E. (1996). A unidimensional item response model for unfolding responses from a graded disagree-agree response scale. Applied Psychological Measurement, 20(3), 231–255. https://doi.org/10.1177/014662169602000305
Smith, A. L., Asta, D. M., & Calder, C. A. (2019). The Geometry of Continuous Latent Space Models for Network Data. Statistical Science, 34(3), 428–453. https://doi.org/10.1214/19-STS702
Stan Development Team. (2021). Stan Modeling Language User’s Guide and Reference Manual. URL:http://mc-stan.org/
Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273–286. https://doi.org/10.1037/h0070288
Tuerlinckx, F. (2004). The efficient computation of the cumulative distribution and probability density functions in the diffusion model. Behavior Research Methods, Instruments, and Computers, 36(4), 702–716. https://doi.org/10.3758/BF03206552
Tuerlinckx, F., & De Boeck, P. (2005). Two interpretations of the discrimination parameter. Psychometrika, 70(4), 629–650. https://doi.org/10.1007/s11336-000-0810-3
van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287–308. https://doi.org/10.1007/s11336-006-1478-z
van der Linden, W. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46(3), 247–272. https://doi.org/10.1111/j.1745-3984.2009.00080.x
van der Linden, W. J., & Glas, C. A. W. (2010). Statistical tests of conditional independence between responses and/or response times on test items. Psychometrika, 75, 120–139. https://doi.org/10.1007/s11336-009-9129-9
van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2), 339–356. https://doi.org/10.1080/20445911.2011.454498
van der Maas, H. L. J., & Wagenmakers, E.-J. (2005). A psychometric analysis of chess expertise. The American journal of psychology, 118, 29–60.
van Rijn, P. W., & Ali, U. S. (2017). A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing. British Journal of Mathematical and Statistical Psychology, 70(2), 317–345. https://doi.org/10.1111/bmsp.12101
Wang, C., & Xu, G. (2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68(3), 456–477. https://doi.org/10.1111/bmsp.12054
Wang, C., Xu, G., & Shang, Z. (2018). A two-stage approach to differentiating normal and aberrant behavior in computer based testing. Psychometrika, 83(1), 223–254. https://doi.org/10.1007/s11336-016-9525-x
Zhan, P., Jiao, H., & Liao, D. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2), 262–286. https://doi.org/10.1111/bmsp.12114
Zhan, P. , Jiao, H. , Wang, W. C. , & Man, K. (2018). A multidimensional hierarchical framework for modeling speed and ability in computer-based multidimensional tests. Available online at: arXiv:1807.04003
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kang, I., Jeon, M. & Partchev, I. A Latent Space Diffusion Item Response Theory Model to Explore Conditional Dependence between Responses and Response Times. Psychometrika 88, 830–864 (2023). https://doi.org/10.1007/s11336-023-09920-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-023-09920-x