Standard-setting is an essential component of test development, supporting the meaningfulness and appropriate interpretation of test scores. However, in the high-stakes testing environment of aviation, standard-setting studies are underexplored. To address this gap, we document two stages in the standard-setting procedures for the Overseas Flight Training Preparation Test (OFTPT), a diagnostic English test for ab initio pilots, aligned to the International Civil Aviation Organization (ICAO)’s Language Proficiency Rating Scale (LPRS). Performance-level descriptors (PLDs) were empirically generated in Stage 1 in collaboration with six subject matter experts (SMEs). These PLDs made explicit the correspondence between linguistic performance levels within the target language use domain and the ICAO scale. Findings suggest that the ICAO scale is not fine-grained enough to distinguish levels of linguistic readiness among ab initio pilots, nor does it adequately reflect the knowledge, skills, and abilities valued by SMEs within this domain. In Stage 2, 12 SMEs were recruited to set standards and were divided into two groups to investigate the replicability of Ebel method standard-setting procedures. Cut scores were determined for the OFTPT reading and listening tests, which were inferentially linked to the LPRS. There were no significant differences in the cut scores arrived at by both groups and reliability was excellent, suggesting that test users can have confidence in the standards set.

中文翻译：

---

制定飞行学员航空英语诊断测试标准

标准制定是测试开发的重要组成部分，支持测试分数的意义和适当解释。然而，在航空的高风险测试环境中，标准制定研究尚未得到充分探索。为了弥补这一差距，我们记录了海外飞行训练准备测试 (OFTPT) 标准制定程序的两个阶段，这是一项针对从头开始的飞行员的诊断性英语测试，符合国际民航组织 (ICAO) 的语言能力评级规模（LPRS）。绩效级别描述符 (PLD) 是在第一阶段与六位主题专家 (SME) 合作凭经验生成的。这些 PLD 明确了目标语言使用领域内的语言表现水平与 ICAO 等级之间的对应关系。调查结果表明，国际民航组织的量表不够细粒度，无法区分从头算起飞行员的语言准备程度，也没有充分反映中小企业在该领域所重视的知识、技能和能力。在第二阶段，招募了12家中小企业来制定标准，并将其分为两组，以调查玉宝方法标准制定程序的可复制性。OFTPT 阅读和听力测试的切分分数是根据 LPRS 推断得出的。两组得出的分数没有显着差异，并且可靠性非常好，这表明测试用户可以对设定的标准充满信心。