Education in Doctor of Medicine programs has moved towards an emphasis on clinical competency, with entrustable professional activities providing a framework of learning objectives and outcomes to be assessed within the clinical environment. While the identification and structured definition of objectives and outcomes have evolved, many methods employed to assess clerkship students' clinical skills remain relatively unchanged. There is a paucity of medical education research applying advanced statistical design and analytic techniques to investigate the validity of clinical skills assessment. One robust statistical method, multitrait-multimethod matrix analysis, can be applied to investigate construct validity across multiple assessment instruments and settings. Four traits were operationalized to represent the construct of critical clinical skills (professionalism, data gathering, data synthesis, and data delivery). The traits were assessed using three methods (direct observations by faculty coaches, clinical workplace-based evaluations, and objective structured clinical examination type clinical practice examinations). The four traits and three methods were intercorrelated for the multitrait-multimethod matrix analysis. The results indicated reliability values in the adequate to good range across the three methods with the majority of the validity coefficients demonstrating statistical significance. The clearest evidence for convergent and divergent validity was with the professionalism trait. The correlations on the same method/different traits analyses indicated substantial method effect; particularly on clinical workplace-based assessments. The multitrait-multimethod matrix approach, currently underutilized in medical education, could be employed to explore validity evidence of complex constructs such as clinical skills. These results can inform faculty development programs to improve the reliability and validity of assessments within the clinical environment.

中文翻译：

---

评估临床能力：多特征多方法矩阵构建有效性研究。

医学博士课程的教育已转向强调临床能力，可信赖的专业活动提供了在临床环境中评估的学习目标和成果的框架。虽然目标和结果的识别和结构化定义已经发展，但用于评估见习学生临床技能的许多方法仍然相对不变。很少有医学教育研究应用先进的统计设计和分析技术来调查临床技能评估的有效性。一种强大的统计方法，即多特征多方法矩阵分析，可用于研究跨多种评估工具和设置的构建有效性。四个特征被操作化来代表关键临床技能的构建（专业性、数据收集、数据合成和数据交付）。使用三种方法评估这些特征（教师教练的直接观察、基于临床工作场所的评估和客观结构化临床检查类型的临床实践检查）。对于多性状-多方法矩阵分析，四个性状和三种方法是相互关联的。结果表明，三种方法的可靠性值处于适当到良好的范围内，大多数有效性系数显示出统计显着性。趋同效度和发散效度的最明显证据是专业特质。相同方法/不同性状分析的相关性表明方法效果显着；特别是基于临床工作场所的评估。目前在医学教育中尚未充分利用的多特征-多方法矩阵方法可用于探索临床技能等复杂结构的有效性证据。这些结果可以为教师发展计划提供信息，以提高临床环境中评估的可靠性和有效性。