In everyday life, visual information often precedes the auditory one, hence influencing its evaluation (e.g., seeing somebody’s angry face makes us expect them to speak to us angrily). By using the cross-modal affective paradigm, we investigated the influence of facial gestures when the subsequent acoustic signal is emotionally unclear (neutral or produced with a limited repertoire of cues to anger). Auditory stimuli spoken with angry or neutral prosody were presented in isolation or preceded by pictures showing emotionally related or unrelated facial gestures (angry or neutral faces). In two experiments, participants rated the valence and emotional intensity of the auditory stimuli only. These stimuli were created from acted speech from movies and delexicalized via speech synthesis, then manipulated by partially preserving or degrading their global spectral characteristics. All participants relied on facial cues when the auditory stimuli were acoustically impoverished; however, only a subgroup of participants used angry faces to interpret subsequent neutral prosody. Thus, listeners are sensitive to facial cues for evaluating what they are about to hear, especially when the auditory input is less reliable. These results extend findings on face perception to the auditory domain and confirm inter-individual variability in considering different sources of emotional information.

中文翻译：

---

愤怒的面部暗示会影响随后口语韵律的意义解释

在日常生活中，视觉信息通常先于听觉信息，从而影响其评价（例如，看到某人愤怒的脸让我们期望他们愤怒地对我们说话）。通过使用跨模态情感范式，我们研究了当随后的声学信号在情感上不清楚（中性或产生有限的愤怒线索）时面部表情的影响。以愤怒或中性韵律说出的听觉刺激是单独呈现的，或者先于显示与情感相关或无关的面部表情（愤怒或中性面孔）的图片。在两项实验中，参与者仅评估听觉刺激的效价和情绪强度。这些刺激是根据电影中的表演语音创建的，并通过语音合成进行去词汇化，然后通过部分保留或降低其全局频谱特征来进行操纵。当听觉刺激缺乏声音时，所有参与者都依赖于面部暗示。然而，只有一小部分参与者使用愤怒的面孔来解释随后的中性韵律。因此，听众对评估他们将要听到的内容的面部线索很敏感，尤其是当听觉输入不太可靠时。这些结果将面部感知的发现扩展到听觉领域，并证实了考虑不同来源的情感信息时的个体间差异。