当前位置: X-MOL 学术Organ. Res. Methods › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A Test-Retest Reliability Generalization Meta-Analysis of Judgments Via the Policy-Capturing Technique
Organizational Research Methods ( IF 8.247 ) Pub Date : 2021-05-12 , DOI: 10.1177/10944281211011529
Ze Zhu 1 , Alan J. Tomassetti 2 , Reeshad S. Dalal 1 , Shannon W. Schrader 1 , Kevin Loo 1 , Isaac E. Sabat 3 , Balca Alaybek 1 , You Zhou 1 , Chelsea Jones 1 , Shea Fyffe 1
Affiliation  

Policy capturing is a widely used technique, but the temporal stability of policy-capturing judgments has long been a cause for concern. This article emphasizes the importance of reporting reliability, and in particular test-retest reliability, estimates in policy-capturing studies. We found that only 164 of 955 policy-capturing studies (i.e., 17.17%) reported a test-retest reliability estimate. We then conducted a reliability generalization meta-analysis on policy-capturing studies that did report test-retest reliability estimates—and we obtained an average reliability estimate of .78. We additionally examined 16 potential methodological and substantive antecedents to test-retest reliability (equivalent to moderators in validity generalization studies). We found that test-retest reliability was robust to variation in 14 of the 16 factors examined but that reliability was higher in paper-and-pencil studies than in web-based studies and was higher for behavioral intention judgments than for other (e.g., attitudinal and perceptual) judgments. We provide an agenda for future research. Finally, we provide several best-practice recommendations for researchers (and journal reviewers) with regard to (a) reporting test-retest reliability, (b) designing policy-capturing studies for appropriate reportage, and (c) properly interpreting test-retest reliability in policy-capturing studies.



中文翻译:

通过策略捕获技术对判决进行重测的可靠性泛化元分析

策略捕获是一种广泛使用的技术,但是策略捕获判断的时间稳定性长期以来一直引起人们的关注。本文强调了在策略捕获研究中报告可靠性,尤其是重测可靠性的重要性。我们发现955个策略捕获研究中只有164个(即17.17%)报告了重测信度估计值。然后,我们对捕获政策的研究进行了可靠性概括性荟萃分析,该研究报告了重测可靠性估计值,而我们得出的平均可靠性估计值为0.78。我们还检查了16种潜在的方法学和实质性先例,以进行重测信度(等同于有效性概括研究中的主持人)。我们发现,重测信度对16个因素中的14个具有较强的变异性,但纸笔研究的信度比网络研究的信度高,行为意向判断的信度比其他信度(例如态度)要高。和感性)的判断。我们提供了未来研究的议程。最后,我们为研究人员(和期刊审稿人)提供以下几项最佳实践建议:(a)报告重测信度,(b)为适当的报道设计政策捕捉研究,以及(c)正确解释重测信度在政策捕捉研究中。我们提供了未来研究的议程。最后,我们为研究人员(和期刊审稿人)提供以下几项最佳实践建议:(a)报告重测信度,(b)为适当的报道设计政策捕捉研究,以及(c)正确解释重测信度在政策捕捉研究中。我们提供了未来研究的议程。最后,我们为研究人员(和期刊审稿人)提供以下几项最佳实践建议:(a)报告重测信度,(b)为适当的报道设计政策捕捉研究,以及(c)正确解释重测信度在政策捕捉研究中。

更新日期:2021-05-12
down
wechat
bug