Objective structured clinical examination (OSCE) is widely used to assess medical students' clinical skills. Virtual OSCEs were used in place of in-person OSCEs during the COVID-19 pandemic; however, their reliability is yet to be robustly analyzed. By applying generalizability (G) theory, this study aimed to evaluate the reliability of a hybrid OSCE, which admixed in-person and online methods, and gain insights into improving OSCEs' reliability. During the 2020-2021 hybrid OSCEs, one examinee, one rater, and a vinyl mannequin for physical examination participated onsite, and a standardized simulated patient (SP) for medical interviewing and another rater joined online in one virtual breakout room on an audiovisual conferencing system. G-coefficients and 95% confidence intervals of the borderline score, namely border zone (BZ), under the standard 6-station, 2-rater, and 6-item setting were calculated. G-coefficients of in-person (2017-2019) and hybrid OSCEs (2020-2021) under the standard setting were estimated to be 0.624, 0.770, 0.782, 0.759, and 0.823, respectively. The BZ scores were estimated to be 2.43-3.57, 2.55-3.45, 2.59-3.41, 2.59-3.41, and 2.51-3.49, respectively, in the score range from 1 to 6. Although hybrid OSCEs showed reliability comparable to in-person OSCEs, they need further improvement as a very high-stakes examination. In addition to increasing clinical vignettes, having more proficient online/on-demand raters and/or online SPs for medical interviews could improve the reliability of OSCEs. Reliability can also be ensured through supplementary examination and by increasing the number of online raters for a small number of students within the BZs.

中文翻译：

---

日本适应大流行的在线现场混合毕业 OSCE 的可行性和可靠性。

客观结构化临床考试（OSCE）广泛用于评估医学生的临床技能。在 COVID-19 大流行期间，使用虚拟 OSCE 代替了面对面的 OSCE；然而，它们的可靠性尚未得到强有力的分析。通过应用概括性（G）理论，本研究旨在评估混合面对面和在线方法的混合 OSCE 的可靠性，并深入了解提高 OSCE 的可靠性。在 2020-2021 年混合 OSCE 期间，一名考生、一名评估员和一个用于体检的乙烯基人体模型现场参加，一名用于医学访谈的标准化模拟患者 (SP) 和另一名评估员在视听会议系统的一个虚拟分组会议室中在线加入。计算了标准 6 站、2 个评估者和 6 项设置下的边界分数（即边界区 (BZ)）的 G 系数和 95% 置信区间。标准设置下的面对面（2017-2019）和混合OSCE（2020-2021）的G系数估计分别为0.624、0.770、0.782、0.759和0.823。BZ 分数估计分别为 2.43-3.57、2.55-3.45、2.59-3.41、2.59-3.41 和 2.51-3.49，分数范围为 1 至 6。尽管混合 OSCE 表现出与面对面 OSCE 相当的可靠性，作为一项风险非常高的考试，他们需要进一步改进。除了增加临床小插曲之外，拥有更熟练的在线/按需评估员和/或在线 SP 进行医学访谈可以提高 OSCE 的可靠性。还可以通过补充考试和增加 BZ 内少数学生的在线评分员数量来确保可靠性。