3.1. Participants

A total of 107 participants took part in Experiment 1, including Mandarin Chinese and English native speakers. All participants were over 18 and at the time of testing had normal or corrected-to-normal vision. A total of 55 Mandarin Chinese native speakers (21 females; mean age = 21, SD = 1) were recruited from three different universities in Zhengzhou, China. Although English is compulsory for students in China, these participants were functionally monolingual in Mandarin Chinese (self-assessed). A total of 52 English native speakers (28 females, mean age = 20, SD = 2.02) were recruited in York, UK. The inclusion criterion was to be functionally monolingual in English (self-assessed).

3.2. Materials

The stimuli consisted of 48 animations in total. Each was 4.5 seconds long. The animations show 12 event scenes (example in Figure 1). The animations are co-presented with a photo sequence in four configurations or conditions, manipulated in terms of manner, path or ground. We labeled the four conditions full match, manner mismatch, path mismatch, and full mismatch, as illustrated in Figure 1. To minimize noise, each event scene within a quadruplet shares the same agent (a man) and background, the stimulus differences are constrained to path/manner/ground manipulations. Each participants viewed 36 trials (36 picture primes, 12 critical videos, and 24 filler videos). Four trials were used for practice. The total number of trials per participant is 36, rather than the full 48, to avoid repetition effects (the four conditions share one identical scene, we let each participant see one condition in each scene only). From the 36 trials, half were matches and the other half were mismatches.

Figure 1. An example combination of a picture prime with a video quadruplet used in the b-CFS experiment. The sequence photos show (left to right) a full match between the video and the prime (a sequence picture of a man carrying a suitcase into the room), a manner mismatch (a video of a man pulling a suitcase into the room), a path mismatch (a video of a man carrying a suitcase out of the room), and a full mismatch with a visual oddity (a man carrying a dinosaur).

The manners of motion included six types, namely pulling, pushing, rolling, dragging, carrying in front, and carrying on the back. The paths of motion also included six types, into, out of, across, around, up, and down. Apart from manner salience increased through manner mismatch, and path salience increased through path mismatch, we also included a full match and a full mismatch condition. The full match condition was the reference for comparisons. The full mismatch condition was an added control designed to check if a visual oddity, such as a dinosaur, breaks through suppression relatively faster than the stimuli in the other three conditions. The time it took to detect a stimulus was the measure of the extent to which high-level properties, including habitual linguistic encoding and familiarity, influenced perceptual suppression (Stein et al., Reference Stein, Hebart and Sterzer2011). The video stimuli in this study identical with those used in Vanek and Fu (Reference Vanek and Fu2023), which is also a b-CFS study but without sequence pictures as primes. Adding primes in the form of sequence pictures improved the design in this study by sharpening the contrast for both manner-based and path-based mismatches. The full list of stimuli is available on the project page https://osf.io/54gse/.

The main equipment for the b-CFS experiment was a pair of mirror stereoscope goggles with four mirrors inside. The angle of the mirrors was adjustable so that the participants’ vision for their left and right eye could be separated. The experiment was run in Matlab (MathWorks, Natick, MA) using the Psychophysics toolbox (Brainard, Reference Brainard1997).

3.3. Procedure

Before the participants arrived in the laboratory, they were asked to watch a video on YouTube (AllAboutVisionVideo, 2018), which demonstrated how to find their dominant eye. This procedure was re-checked by the researcher in the laboratory, after the participant signed the consent forms.

In the experimental process (Figure 2), each participant was seated in front of a mirror stereoscope and placed their head on a chin rest. The mirror stereoscope stood on a fixed tripod in front of a computer screen. The distance between the screen and the mirror stereoscope was 45 cm, and the distance between the participants’ eyes and the mirror stereoscope was 2 cm (with slight variation across participants). The dominant eye (e.g., the left eye in Figure 2) was shown a dynamic Mondrian-like mask, which flickered at the frequency of 10 Hz. The non-dominant eye (e.g., the right eye in Figure 2) was presented with a video clip (4.5 seconds) placed randomly in one of the four corners of the screen. The contrast of the target stimulus, which was initially invisible, increased from 0% to 100% within 2.5 seconds. Initial invisibility served as an eraser of visual awareness to enable gradual perceptual enhancement as the contrast increased (Jiang et al., Reference Jiang, Costello and He2007). One participant saw 36 trials, 4 for practice and 32 as part of the main experiment. In each trial, there were two steps, and the whole b-CFS procedure took up to 20 mins. As the first step, each trial started with two white frames shown on the screen, with the aim of helping the participant adjust the mirror stereoscope goggle provided. Once they only saw one white frame on the screen, they proceeded to the b-CFS mode by pressing a key on the keyboard. Then, in the second step, a flickering screen was shown, while an animation gradually appeared in one of the four corners of the screen. The participant’s task was to identify the location of the animation by pressing one of four buttons (top-left, top-right, bottom-left, or and bottom-right) corresponding to the location of the stimulus video. Stimulus locations were counterbalanced, and their order of presentation was randomized. The times taken to detect the stimuli were analyzed as indicators of suppression strength.

Figure 2. Illustration of the b-CFS procedure. The sequence of screens on the left demonstrates what was shown on the screen, and the one on the right shows the image that participants actually saw through the mirror stereoscope.

3.4. Results

The first point is on accuracy rates. Overall, participants identified the correct locations of the motion events with high accuracy, M = 92%, SD = 0.09 in the Chinese group, and M = 93%, SD = 0.06 in the English group. Data from three Chinese speakers and one English speaker were excluded due to low response accuracy (≤75%, set based on Francken et al., Reference Francken, Kok, Hagoort and De Lange2011, and Slivac et al., Reference Slivac, Hervais-Adelman, Hagoort and Flecken2021). Further four English native speakers’ data were removed list-wise because in the debrief it transpired that they also had communicative knowledge of another language. Only correct responses were included in the data analysis. Outliers, namely responses greater than ±2.5 standard deviations away from the group mean RT, were also excluded from analyses. Due to large variation, any data longer than 35 seconds were regarded as extreme values and were discounted from analyses. The 35-second cut-off was based on the scatterplot of the correct responses. In total, 119 data points were removed as outliers in the b-CFS experiment.

Reaction times were calculated from stimulus onset until button press. The average reaction times needed to correctly detect the stimulus in each condition are shown in Table 1 separately per group. The critical comparison goes in the expected direction, with the mean RT differences between path and manner mismatches smaller in the Chinese group (589 ms) than in the English group (1655 ms).

Table 1. Means and SDs of the reaction times (ms) taken to detect videos by Chinese and English monolinguals across four conditions

Figure 3 shows in greater detail that Chinese speakers needed a similar amount of time for the stimuli to break through suppression in manner and path mismatch conditions. In contrast, English speakers’ data exhibit a greater difference in these two conditions. To statistically test the effects of Condition and Language groups, as well as their interaction, we used linear mixed-effects regression through the lme4 package (Bates et al., Reference Bates, Mächler, Bolker and Walker2015) in R (R Core Team, 2020) using the full match condition as reference. The fixed effects were Language group (2 levels) and Condition group (4 levels), both dummy-coded, the outcome variable was the log-transformed reaction times, and the random effect factors were Subject and Item. The maximal random effect structure that converged included by-subject random intercepts and by-item random intercepts. We proceeded with a forward variable selection and compared a full model with Language group (lmer(logRT ~ 1 + condition * language_group + (1|subject) + (1|item)) with a reduced model excluding Language group (lmer(logRT ~ 1 + condition + (1|subject) + (1|item)). This comparison returned a significant effect of Language group (χ2 (4) = 46.67, p < 0.001), showing that the Mandarin group was overall slower. Also, a significant interaction emerged between Condition (manner mismatch x full match) and Language groups (English x Chinese) in the full model (β = −0.18, SE = 0.06, t = −2.854, p = 0.004; raw RT β = −835 ms).

Figure 3. Log-transformed stimulus detection times in Experiment 1 shown per group (CH = Chinese, EN = English) and condition (FM = full match, FMI = full mismatch, MMI = manner mismatch, PMI = path mismatch). Box-plots show the medians and 50% of the log RTs within the boxes. Violin and raincloud plots complement visualization by showing the data distribution pattern in each condition and group.

To further explore the differences between conditions within each language group, we ran post-hoc tests using the emmeans package in R (Lenth, Reference Lenth2021). All possible comparisons were run for both groups. In the English group, RTs in the manner mismatch (MMI) condition were significantly shorter (β = −0.19, SE = 0.07, t = −2.82, p = 0.006; raw RT β = −827 ms) than in the path mismatch (PMI) condition. Also, RTs in the manner mismatch condition were also significantly shorter compared to those in the full match (FM) (β = −0.16, SE = 0.07, t = 2.38, p = 0.020; raw RT β = −852 ms) and full mismatch (FMI) (β = −0.16, SE = 0.07, t = 2.30, p = 0.024; raw RT β = −825 ms) conditions. However, no significant difference was found between the manner and path mismatch conditions (β = 0.03, SE = 0.07, t = 0.48, p = 0.63; raw RT β = 1030 ms) in the Mandarin group. No significant difference also characterized the other conditions in the Mandarin group, including full match versus full mismatch (β = −0.004 SE = 0.07, t = −0.06, p = 0.97; raw RT β = −996 ms), manner mismatch versus full match (β = 0.02, SE = 0.07, t = −0.33, p = 0.75; raw RT β = 980 ms) and also manner mismatch versus full mismatch (β = 0.02, SE = 0.07, t = −0.27, p = 0.79; raw RT β = 980 ms).

3.5. Discussion

Reaction time data show that manner salience, compared to path salience, provided a detection speed advantage for English speakers but not for Mandarin speakers. This finding aligns with the predicted direction of a processing speed difference reported previously for manner in English and Mandarin native speakers (Vanek & Fu, Reference Vanek and Fu2023). This study extends recent work on the effects of language on visual perception (Lupyan et al., Reference Lupyan, Rahman, Boroditsky and Clark2020) to motion event processing. Information about manner, made typically prominent in English motion descriptions, was found to exert influence on a low-level process of visual feature detection during binocular rivalry (Pasley et al., Reference Pasley, Mayes and Schultz2004). This effect can be explained as the automatic recruitment of manner labels that increase the salience of manner information in visual input and thus facilitate its detection (Perry & Lupyan, Reference Perry and Lupyan2013). In the b-CFS experiment, the time it takes to detect the change in the kind of motion functions as a low-level psychophysiological correlate of the high-level representation of manner. In English, where manner encoding is comparatively more prominent, the assessment of continuously flashing sensory signals with a gradually appearing motion event was more manner-based. However, in Chinese, less weight is given to manner information in verbal encoding, and this difference in linguistic modulation was likely to be mirrored when Mandarin speakers were detecting the emergence of motion events from visual noise. We were curious to see whether similarly predictable patterns hold on a higher level of visual processing, so we designed a separate experiment with an upregulated involvement of language.

4.1. Participants

One hundred and two participants were recruited in Experiment 2 in two new cohorts of Mandarin Chinese and English native speakers. The same inclusion criteria applied, to be over 18 and at the time of testing have normal or corrected-to-normal vision. A total of 51 Mandarin Chinese native speakers (28 females; mean age = 21.30, SD = 2.90) were recruited in Zhengzhou, China. All were functionally monolingual in Mandarin Chinese (self-assessed). A total of 51 English native speakers (34 females, mean age = 19.49, SD = 1.23) were recruited in York, UK. In this group, all were functionally monolingual in English (self-assessed).

4.2. Materials

The video materials were identical with those used in the b-CFS experiment. However, instead of having to match sequential pictures with the target videos, the SV experiment used sentences as primes. An example sentence quadruplet in English/Mandarin is, ‘A man is rolling a log towards a cabin’/‘Yi1 ge2 nan2 ren2 zheng4 zai4 ba3 yi2 ge4 mu4 zhuang1 gun3 xiang4 yi1ge4 xiao3 mu4 wu1’ used for the full match condition, ‘A man is carrying a log towards a cabin’/‘Yi1 ge2 nan2 ren2 zheng4 zai4 ba3 yi2 ge4 mu4 zhuang1 ban1 xiang4 yi1ge4 xiao3 mu4 wu1’ for manner mismatch, ‘A man is rolling a log away from a cabin’/‘Yi1 ge2 nan2 ren2 zheng4 zai4 ba3 yi2 ge4 mu4 zhuang1 gun3 li2 yi1ge4 xiao3 mu4 wu1’ for path mismatch, and ‘A man rolled a dinosaur towards a cabin’/Yi1 ge2 nan2 ren2 zheng4 zai4 ba3 yi2 ge4 kong3 long2 gun3 xiang4 yi1ge4 xiao3 mu4 wu1’ for full mismatch. The role of the prime sentences was to enable the participants to create mental situation models (Zwaan & Radvansky, Reference Zwaan and Radvansky1998) against which to compare the motion event presented in the subsequent video. In Experiment 2, each participant was presented 36 trials (36 sentence primes, 12 critical videos and 24 filler videos) out of which four trials were used for practice. For the ease of comparability across the study, we kept the condition labels identical with those in Experiment 1. As the critical dimension in Experiment 2 is sentence-video correspondence, it is important to note that in the condition labeled manner mismatch the sentence and the video corresponded in path, and the condition labeled path mismatch equals sentence-video correspondence in manner. In all videos, manner and path information was available simultaneously from the very beginning of the action. This is a methodological strength ruling out the possibility of response times having been influenced by motion component sequencing in the visual input.

The self-paced sentence-video (SV) verification experiment was run in Matlab (MathWorks, Natick, MA) using the Psychophysics toolbox (Brainard, Reference Brainard1997) to collect data from the Mandarin Chinese speakers. We used the Gorilla Experiment Builder (www.gorilla.sc) to create and host our experiment (Anwyl-Irvine et al., Reference Anwyl-Irvine, Massonnié, Flitton, Kirkham and Evershed2018) to collect data from the English speakers. The collected RTs were log-transformed to reduce skewness in a non-normal distribution. We followed this step as a standard statistical procedure used in related studies measuring RTs to track low-level motion perception (e.g., Slivac et al., Reference Slivac, Hervais-Adelman, Hagoort and Flecken2021) as well as high-level visual processing of motion events (e.g., Sakarias, Reference Sakarias2019).

4.3. Procedure

Two main steps characterize the SV experiment (Figure 4). After a fixation cross, participants saw one sentence presented in their native language on the computer screen, describing a motion event. After reading the sentence, they pressed the space bar to continue. Upon button press, their task was to carefully watch a video which either matched or did not match the sentence they had read before. Their task was to press the left arrow (labeled ‘YES’) to indicate a match, or right arrow (labeled ‘NO’) to indicate a mismatch. The video played in a loop until they made their decision. Stimulus presentation followed a randomized order.

Figure 4. Illustration of the sentence-video verification procedure. A fixation cross is followed by a sentence prime (self-paced) and subsequently by a video.

4.4. Results

We first examined accuracy rates. In general, participants verified sentence-video matches and mismatches with high accuracy, M = 88.13%, SD = 0.05 in the Chinese group, and M = 90.36%, SD = 0.08 in the English group. Data from three Chinese speakers and one English speaker were excluded due to low response accuracy (≤ 75%). Other English native speakers’ data were removed because it came to light during the debrief that they were also fluent in another language. Only correct responses fitting within ±2.5 standard deviations from the group mean RT were included in the data analysis. Due to large variation, any data longer than 15 seconds were regarded as extreme values and were discounted from analyses. The 15-second cut-off was based on the scatterplot of the correct responses. In total, 78 data points were removed as outliers in the SV experiment.

Reaction times were calculated from video onset until button press. The average RTs needed to correctly verify sentence-video (mis)matches in each condition are shown in Table 2. The full mismatches were recognized fastest in each group as predicted. The key between-group difference concerns the mean RT differences between path and manner mismatches, smaller in the Chinese group (150 ms) than in the English group (731 ms), but in the opposite direction than predicted. Unexpectedly, manner mismatches took longer to recognize than path mismatches in the English group.

Table 2. Means and SDs of the reaction times (ms) taken to recognize (mis)matches by Chinese and English monolinguals across the four conditions

Figure 5 presents the overlaps and differences between the two groups’ sentence-video verification times in finer detail. It shows that Chinese speakers took roughly the same to correctly verify mismatching sentence-video pairs in manner and path mismatch conditions. Greater differences emerged for the English speakers between these two conditions, albeit the directionality of this difference was unexpected. To statistically test the effects of Condition and Language groups, and their interaction, we built linear mixed-effects regression models following the process in Experiment 1. Here too, we report the maximal random effect structure that converged, that is with random slopes over condition by subject and random slopes over group by item. We compared a full model with Language group (lmer(log(RT) ~ 1 + condition * language_group + (1 + condition|subject) + (1 + language_group|item)) with a reduced model excluding group lmer(log(RT) ~ 1 + condition + (1 + condition|subject) + (1 + language_group|item)). This comparison returned a significant effect of Language group (χ2 (4) = 44.89, p < 0.001), which shows that, on the whole, Mandarin speakers took longer to respond.

Figure 5. Log-transformed sentence-video verification times for the correct responses in Experiment 2 shown per group (CH = Chinese, EN = English) and condition (FM = full match, FMI = full mismatch, MMI = manner mismatch, PMI = path mismatch). Box-plots show the medians and 50% of the log RTs within the boxes. Violin and raincloud plots complement visualization by showing the data distribution pattern in each condition and group.

To further investigate the differences within each language group, post-hoc analyses were conducted using the emmeans package in R (Lenth, Reference Lenth2021). The English group took significantly longer to verify manner mismatches than path mismatches (β = −0.26, SE = 0.13, t = 2.08, p = 0.043; raw RT β = −771 ms).

However, no significant difference was found between manner (MMI) and path mismatch (PMI) conditions in the Chinese group. Also, RTs of both language groups in the full mismatch (FMI) condition were significantly shorter compared to those in the full match (FM) condition (β = −0.18, SE = 0.08, t = −2.27, p = 0.028; raw RT β = −835 ms) When checked separately, English speakers were significantly faster in the full mismatch condition than in the full match (β = −0.32, SE = 0.11, t = −2.91, p = 0.005; raw RT β = −762 ms), manner mismatch (β = −0.49, SE = 0.12, t = −4.00, p < 0.001; raw RT β = −613 ms), as well as the path mismatch condition (β = −0.23, SE = 0.11, t = −2.13, p = 0.039; raw RT β = −795 ms). The same holds for the Chinese group, who were significantly faster in the full mismatch condition than in the full match (β = −0.18, SE = 0.08, t = −2.19, p = 0.033; raw RT β = −835 ms), manner mismatch (β = −0.28, SE = 0.10, t = −2.81, p = 0.007; raw RT β = −756 ms), and path mismatch condition (β = −0.21, SE = 0.08, t = −2.66, p = 0.011; raw RT β = 811 ms). However, in the Chinese group there was no significant difference between manner mismatch and path mismatch (β = 0.064, SE = 0.1, t = 0.64, p = 0.524; raw RT β = 1066 ms).

4.5. Discussion

Sentence-video verification times showed that manner prominence typical of English motion verbalization differentially affected the processing of manner-based versus path-based mismatches. However, manner-path equipollence, characteristic of motion encodings in Mandarin Chinese, did not lead to pronounced processing differences between manner-based versus path-based mismatches. The finding that English speakers took longer to decide about manner-based than path-based mismatches may seem puzzling, but only at the first glimpse. Lower-level processes like stimulus detection rest on a contrast spotting ability, where mismatches may matter more. High-level processes like sentence-video verification involve verbally primed semantic analysis, for which correspondence between the key motion component facilitates high-level decisions (Ji, Reference Ji2017). In other words, in English, where manner encoding is comparatively more prominent, the upregulation of manner through a linguistic prime led to a faster assessment of path mismatches, which through a lens of a sameness check were essentially manner-matches. Cross-linguistic difference in the speed of manner-matches faster in English than in Mandarin speakers is in line with earlier findings testing high-level processing of caused motion (Ji, Reference Ji2017). This effect can be explained as rapid, linguistically induced upregulation of manner labels (Lupyan, Reference Lupyan2012) that facilitate the high-level cognitive process of a sentence-video match verification. However, this account can just partly explain the results since being able to quickly establish a manner match is insufficient information when the task is to press ‘NO’ for mismatches. Path must have been processed before button presses too. In the present design, the temporal distance between path information presented first linguistically and then visually was shorter than that between the two sources of manner information, so it is possible that path mismatch verifications were (co-)supported by a recency effect. This temporal distance account may be intuitively appealing, but it cannot fully embrace the observed cross-linguistic differences. Instead, what we find more powerful in this respect is the predictive processing account, which involves language-specific expectation optimization to quickly recognize sentence-video mismatches. We elaborate on this point in the General Discussion.

5.1. On how detecting motion and judging sentence-video matches become more manner-sensitive

Two main innovative contributions emerged from this study to inform motion event cognition research. First, cross-linguistic effects of differential encoding of manner were observed in a prelinguistic time-window of visual feature detection. Differences in detection speed when manner-salient motion break through suppression into awareness aligned with language-driven hypotheses based on encoding manner and path in Mandarin Chinese and English. This finding confirms that cross-linguistic effects on motion event processing are nontrivial, resilient, and exerting influence already when perceptual signals are detected. In the words of Casasanto et al. (Reference Casasanto, Boroditsky, Phillips, Greene, Goswami, Bocanegra-Thiel, Santiago-Diaz, Fotokopoulu, Pita and Gil2004), ‘language can shape even primitive, low-level mental processes’ (p. 575). We interpret this influence within the framework of predictive processing (Lupyan & Clark, Reference Lupyan and Clark2015). The underlying mechanism of the observed cross-linguistic effect on the speed of manner detection within this account is explained as a downward flow of language-modulated predictions about visual signals that emerge through suppression. Greater weight of manner in the linguistic encoding of motion events in English contributes to the formation of English speakers’ predictions to be relatively more manner-based than is the Chinese speakers’ predictions. Without recruiting language consciously, when visual percepts mismatch in manner, English speakers may detect it faster because of increased relevance of manner information involved in their expectations about the incoming sensory data. These downward predictions are likely to differ in Chinese speakers, whose anticipatory mechanism is more likely to combine emerging visual stimuli with equally weighed manner-based and path-based expectations. In sum, variation in the ways of talking about motion in Chinese and English are built into predictions which can influence how motion events are perceived.

Second, findings from the sentence-video verification task were also in line with the related prediction, and confirmed that English and Chinese speakers differed in their processing of manner and path information at the high level, when verifying sentence-video correspondences. Although the Mandarin speakers were overall slower than the English speakers (and that holds for both experiments), the critical comparison to monitor processing patterns of manner and path was within language group, and only the differential scores were looked at cross-linguistically. Greater difference in processing manner-based versus path-based stimulus pairs in the English group than in the Mandarin group indicates that language cues, in this case the relatively greater prominence of manner in English motion event expressions, acted as important building blocks of situation models (Zwaan & Radvansky, Reference Zwaan and Radvansky1998) against which the subsequent motion videos were compared. Considering the Talmy’s typology, an E-language like Mandarin and an S-language like English might not be the ideal examples of a contrastive encoding of manner, but they could still predict the observed between group differences. In extension of the present study, one could expect more pronounced between-group differences with a design that compares speakers of a V-language like Spanish or Japanese with speakers of a S-language like English.

5.2. On the asymmetry in reactions when manner is suppressed or verbally highlighted

The asymmetry in processing speed in response to stimuli manipulated for manner and path information, which emerged in the English group across the two experiments, is perhaps the most intriguing result. More specifically, while manner mismatches took English speakers less time than path mismatches to detect in the b-CFS experiment, they took longer to judge in the self-paced sentence-video verification experiment. This asymmetric pattern points to the involvement of very different mechanisms guiding English speakers’ low-level visual feature detection process and a high-level semantic verification process. Within the framework of predictive processing, the manipulation of manner information was detected more quickly than the manipulation of path information because of a sharper contrast with default manner-based predictions when incoming sensory signals were breaking through suppression. More contrastive signals are less predictive input that travels up the visual hierarchy and assists learning as it triggers an update of subsequent predictions. This way, less expected input helps to refine further precision estimates. However, in sentence-video verification, the situation differs in that the sentences mentally activated the matching upcoming videos, and information that was linguistically primed got recognized as a corresponding visual match faster.

Another surprising result in the present study was that in the b-CFS experiment, the time English speakers needed to detect stimuli in the manner mismatch condition was the shortest compared to all the other conditions. A possible reason can be found in salience theory (Slobin, Reference Slobin, Hickmann and Roberts2006), which advocates that on top of differentiating languages based on the main verb that expresses a motion event, languages can be further divided into how they capture the levels of salience in manner information. Manner salience in English expressions of motion is high, making manner stand out. Manner-based top-down predictions about what kind of motion to expect to show through suppression quickly alerted the participants and as soon as this expectation broke, allowing English participants to react to manner-based mismatches faster. The relevance of manner could then have carried over to other trials of a similar type. In Feist and Férez’s (Reference Feist and Férez2013) words, ‘higher codability of manner of motion correlates with improved memory for manner’ (p. 398). However, no such manner mismatch detection advantage was found in the Chinese group, as they processed all types of mismatches with similar speed. In sum, these b-CFS results suggest that low-level visual processing of caused motion is also connected to the levels of salience in their corresponding linguistic expressions.

On another level of comparison between conditions and experiments, both the English and the Mandarin group took similar amounts of time to detect the full matches (prime picture identical with the video breaking through suppression) and the full mismatches (picture and video with an distinctively different object) in the b-CFS experiment. This was not the case in the sentence-video verification experiment, where both the English and the Mandarin speakers needed less time to verify the full mismatches where a conspicuously different object (a dinosaur) was moved. This variation in results of the two tasks further demonstrates that it was not the conspicuous object that led to between-group differences in breakthrough times in the b-CFS experiment. Instead, it was the mismatches in the manner of motion driving the detection time differences, while the object caused to move was relatively more suppressed. To confirm the functionality of the flickering mask for stimulus suppression, a small-scale validity check was run in the preparation stage. For the validity check prior to Expt. 1 and Expt. 2, ten participants (mean age = 19.1, SD = 0.74; 6 females) other than those taking part in the main experiments were asked to complete a detection task. The purpose of the validity check, a routine exercise for b-CFS experiments to test mask functionality, was to ensure that through the changes in contrast shown to the participants we were able to regulate the time delay when motion events became visible. The same stimuli as in the b-CFS experiment were used, and each participant viewed half of the stimuli with a flickering mask and the other half without a mask, with random order and stimulus counterbalancing in place. The average accuracy rate in the validity check was 0.88 (SD = 0.10). The average suppression time with a mask was 8113 ms (SD = 4390 ms), and without a mask, it was 5730 ms (SD = 1351 ms). A significant main effect of mask (χ2 (1) = 9.31, p = 0.02) was found when we compared a mixed-effects model with a mask included and a reduced model with a mask excluded. These results showed that the mask worked, in other words that it was sufficiently powerful to suppress the stimuli during the b-CFS experiment.

To contextualize our findings, the roughly equal processing pattern and the manner-biased processing pattern found in Mandarin and English speakers, respectively, were shared across the two levels tested in this study. The observed patterns align with previous work examining how manner and path processing differs between Mandarin/Cantonese and English speakers (Ji, Reference Ji2017; Wang & Wei, Reference Wang and Wei2021). These two studies converged in showing that manner and path information in similarity judgement contexts were processed in group-specific ways. English speakers took longer to make judgements when manner differed compared to when path differed. The explanation provided was that English speakers processed the manner and path information in a sequence, whereas Mandarin speakers processed both motion components in parallel (Ji, Reference Ji2017). While the validity of this account would need to stand the test of sequential and simultaneous presentation of manner and path information in a fully crossed design, it holds that the present study contributes to the extant body of literature confirming that caused motion event processing in English speakers is more manner-based than in Mandarin speakers. The specific contribution comes from two levels of processing. First, applying the b-CFS paradigm to motion event research proved advantageous to document that language-specific encodings of motion are automatically (on the low level) utilized to make predictions about the incoming sensory input. Second, applying the sentence-picture verification paradigm helped to clarify that language-specific encodings of motion operate differently when they are consciously (on the high level) employed during sematic analyses that are primed by verbal cues.

At the high level of linguistically primed motion event processing, English speakers found it harder to make verifications when manner differed compared to when path differed. If the relativistic view holds, faster verifications of path mismatches may appear counter-intuitive at first because English typically encodes manner (Talmy, Reference Talmy2000) rather than path in the main verb. In the wider context, the critical reader may find the path mismatch advantage for English speakers evocative of what could be dubbed ‘a reversed Whorfian effect’ observed in Papafragou et al. (Reference Papafragou, Hulbert and Trueswell2008). Let us recall that Papafragou et al. (Reference Papafragou, Hulbert and Trueswell2008) found English speakers’ late eye-fixations focusing on the path, which reflects the component of motion not typically expressed in the main verb. English speakers’ increased attention allocated to path is a signal of greater cognitive effort not assisted by (overt or covert use of) the native language when visually inspecting scenes in preparation for a memory task. The relative increase in attention to path, when compared to a group of Greek speakers, resulted in English speakers’ more accurate recognition of the path-endpoint object. Not quite convinced by the universalist interpretation advocated in the original article, one could see this ‘reversed Whorfian effect’ as a signal of language-specific differences in predictive processing (Lupyan & Clark, Reference Lupyan and Clark2015), which aligns with the relativistic view. More specifically, English speakers’ late fixations launched to an event component that the main verbs do not readily encode could work as optimization of dissimilar weights when visually presented manner and path details are integrated to tune predictions for maximized success in an upcoming memory task. What the motion event recall in Papafragou et al.’s (Reference Papafragou, Hulbert and Trueswell2008) shares with this study’s sentence-video mismatch verification is the relatively generous time-window during which the motion component not made salient via verbal encoding gained a processing advantage. English speakers’ shorter reaction times in path mismatches can also be explained as an optimizer of weights for the visually salient but linguistically underfocused path information that needs to be considered for successful task completion. Such optimization rests on the idea that the predictive mind estimates the uncertainty of the incoming visual data and tunes predictions based on both the upregulated and the underfocused information in the linguistic prime.

5.3. On event integration theory, limits, and future directions

The findings of both experiments are of immediate relevance to Talmy’s typology of event integration, distinguishing S-languages and V-languages depending on which sentence constituents encode semantic notions such as manner and path (2000:117), and also to the expanded typology by Slobin (Reference Slobin, Verhoeven and Stromqvist2004) adding E-languages such as Mandarin, Jaminjung (Schultze-Berndt, Reference Schultze-Berndt2000) and Thai (Zlatev & Peerapat, Reference Zlatev, Peerapat, Strömqvist and Verhoeven2004) to the taxonomy. The present study brings new evidence that the inequivalence of encoding manner in English and Mandarin predictably influences how top-down information affects not only high-level semantic but also low-level perceptual processing of motion, with speaker variation groupable by the type of manner encoding in their native language. Manner plays a more prominent role than path in motion event processing of English speakers, whether language during the task had a chance to be recruited consciously or not. For Chinese speakers, however, manner and path are of relatively more equal weight, both in forming predictions about sensory input and in building situation models about caused motion. These explanations combine well with the predictive processing account (Lupyan & Clark, Reference Lupyan and Clark2015), and they also complement the broader context of extant event cognition research with Mandarin versus English speakers (e.g., Tang et al., Reference Tang, Vanek and Roberts2021; Zhang & Vanek, Reference Zhang and Vanek2021).

Regarding motion types, this study could choose between voluntary or caused motion (Talmy, Reference Talmy and Shopen1985). One option was to focus on motion events involving an agent that moves along a path in a specific manner, known as spontaneous or voluntary motion (as in the cat is crawling under the sofa). The other option was to choose motion events in which an agent displaces an object in a specific manner so that the object moves along a path, known as cause motion. We chose caused motion for this study to increase its relevance for, and connectivity, with a growing body of experimental work using this motion event type (Ji et al., Reference Ji, Hendriks and Hickmann2011b; Ji & Hohenstein, Reference Ji and Hohenstein2014a, Reference Ji and Hohenstein2014b; Montero-Melis & Bylund, Reference Montero-Melis and Bylund2017; Tusun, Reference Tusun2023; Tusun & Hendriks, Reference Tusun and Hendriks2022; Wang & Wei, Reference Wang and Wei2021). Another motivation was to partially replicate and enhance recent experimental work on the early stages of caused motion processing across Mandarin versus English speakers (Vanek & Fu, Reference Vanek and Fu2023), verifying the suitability of b-CFS as a method that can probe into the automatic processing of dynamic stimuli. Much of previous research using the b-CFS paradigm used static pictures as stimuli (e.g., photos of simple objects like a pumpkin in Lupyan & Ward, or photos of faces in different positions in Jiang et al., Reference Jiang, Costello and He2007). However, to examine motion event processing with static images would tap into inference rather than perception, and for this study the latter was key. Also, filtering out and manipulating the target components of motion events, more specifically, trying to increase and decrease the salience of manner and path information across static stimuli, would have been problematic. Was the effectiveness of b-CFS compromised by fine distinctions in dynamic stimuli (Pournaghdali & Schwartz, Reference Pournaghdali and Schwartz2020)? Between-group differences at a low level of visual detection aligning with the inequivalence of manner in its linguistic encoding in English and Mandarin showed that b-CFS is applicable as a method to test preverbal processing of motion events. In the future, studies might benefit from an even tighter control over non-motion related aspects, such as colour (as, e.g., in Slivac et al., Reference Slivac, Hervais-Adelman, Hagoort and Flecken2021).

Another improvement in future work could be through an increase in the number of the items, perhaps even with an added measure of graded contrast in manner and path mismatches. Among other potential limitations is the comparability of response times between groups. Given that the Mandarin speakers took longer than English speakers to respond across all conditions in both experiments (as also found in Vanek & Fu, Reference Vanek and Fu2023; Zhang & Vanek, Reference Zhang and Vanek2021), it is possible that variations in cultural background or other non-linguistic factors could have influenced the results. For example, it is possible that Mandarin speakers, influenced by cultural factors, may have prioritized confidence in their responses over speed. Emphasis on precision could potentially result in null effects across some conditions. For this reason, it was reassuring to see that full mismatches generated the fastest responses in both experiments as expected, a result which aids the validity of the theoretically motivated argument about language specificity in visual processing. Furthermore, to fully appreciate the contribution of b-CFS to motion event processing research, methodological triangulation via EEG, an approach that does not rely on button presses, could serve as an independent validity check of automaticity in motion event processing.