An open question in research on multimodal figuration is how to mitigate the analyst’s bias in identifying and interpreting metaphor and metonymy; an issue that determines the generalizability of the findings. Little is known about the causes that motivate different annotations. Inter-rater reliability tests are useful to investigate the sources of variation in annotations by independent researchers that can help inform and refine protocols.

Inspired by existing procedures for verbal, visual, and filmic metaphor identification, we formulated instructions to identify multimodal metaphor and metonymy and tested it against a corpus of 21 generic advertisements and 21 genre-specific advertisements (mobile phones). Two independent researchers annotated the advertisements in six rounds. A joint discussion followed each round to consider conflicting annotations and refine the protocol for the ensuing round.

By examining the evolution of inter-rater reliability results, we found that (1) we reached similar levels of agreement for the identification of metaphor and metonymy, although converging on the interpretation of metonymy was more difficult; (2) some genre specificities made it easier to agree on the annotations for mobile advertisements than for the general advertisements; and (3) there was a consistent increase in the kappa scores reaching substantial agreement by the sixth round.

多模态比喻研究中的一个悬而未决的问题是如何减轻分析师在识别和解释隐喻和转喻时的偏见；决定研究结果的普遍性的问题。对于引发不同注释的原因知之甚少。评估者间的可靠性测试对于调查独立研究人员注释中的变异来源非常有用，这有助于告知和完善协议。

受现有言语、视觉和电影隐喻识别程序的启发，我们制定了识别多模态隐喻和转喻的指令，并针对 21 个通用广告和 21 个特定类型广告（手机）的语料库进行了测试。两名独立研究人员对广告进行了六轮注释。每轮之后都会进行联合讨论，以考虑相互冲突的注释并完善下一轮的协议。

通过检查评估者间可靠性结果的演变，我们发现（1）我们在隐喻和转喻的识别上达到了相似的一致性水平，尽管在转喻的解释上达成一致更加困难；(2) 某些类型的特殊性使得移动广告比一般广告更容易在注释上达成一致；(3) kappa 分数持续增加，到第六轮基本一致。