Abstract
Although previous studies have investigated the relationship between inhibition of return (IOR) and multisensory integration, the influence of non-spatial has not been explored. The present study aimed to investigate the influence of non-spatial IOR on audiovisual integration by using a “prime-neutral cue–target” paradigm. In Experiment 1, which manipulated prime validity and target modality, the targets were positioned centrally, revealing significant non-spatial IOR effects in the visual, auditory, and audiovisual modalities. Analysis of relative multisensory response enhancement (rMRE) indicated substantial audiovisual integration enhancement in both valid and invalid target conditions. Furthermore, the enhancement was weaker for valid targets than for invalid targets. In Experiment 2, the targets were positioned above and below to rule out repetition blindness (RB); this experiment successfully replicated the results observed in Experiment 1. Notably, Experiments 1 and 2 consistently found that the correlation between modality differences and rMRE for valid targets indicated that differences in signal strength between visual and auditory modalities contributed to a reduction in audiovisual integration. However, the absence of correlation with the invalid target suggests that attention, as a key factor, may play a significant role in this process. The present study highlights how non-spatial IOR reduces audiovisual integration and sheds light on the complex interaction between attention and multisensory integration.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data will be made available on request.
References
Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15(9), 839–843. https://doi.org/10.1016/j.cub.2005.03.046
Bertelson, P., Vroomen, J., de Gelder, B., & Driver, J. (2000). The ventriloquist effect does not depend on the direction of deliberate visual attention. Perception & Psychophysics, 62(2), 321–332. https://doi.org/10.3758/BF03205552
Baart, M., Stekelenburg, J. J., & Vroomen, J. (2014). Electrophysiological evidence for speech-specific audiovisual integration. Neuropsychologia, 53, 115–121. https://doi.org/10.1016/j.neuropsychologia.2013.11.011
Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51(13), 1484–1525. https://doi.org/10.1016/j.visres.2011.04.012
Carrasco, I. (2014). Gender gap in innovation: an institutionalist explanation. Management Decision, 52(2), 410–424. https://doi.org/10.1108/MD-07-2012-0533
Chen, Q., Fuentes, L. J., & Zhou, X. (2010). Biasing the organism for novelty: A pervasive property of the attention system. Human Brain Mapping, 31(8), 1146–1156. https://doi.org/10.1002/hbm.20924
Chi, Y., Yue, Z., Liu, Y., Mo, L., & Chen, Q. (2014). Dissociable identity-and modality-specific neural representations as revealed by cross-modal non-spatial inhibition of return. Human Brain Mapping, 35(8), 4002–4015. https://doi.org/10.1002/hbm.22454
Chica, A. B., & Lupiáñez, J. (2009). Effects of endogenous and exogenous attention on visual processing: An inhibition of return study. Brain Research, 1278, 75–85. https://doi.org/10.1016/j.brainres.2009.04.011
Chou, W. L., & Yeh, S. (2005). Object-based inhibition of return: Evidence from overlapping objects. Chinese Journal of Psychology, 47(1), 1–13.
Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic Press. https://doi.org/10.4324/9780203771587
Egner, T., & Hirsch, J. (2005). Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information. Nature Neuroscience, 8(12), 1784–1790. https://doi.org/10.1038/nn1594
Eldridge, L. L., Knowlton, B. J., Furmanski, C. S., Bookheimer, S. Y., & Engel, S. A. (2000). Remembering episodes: A selective role for the hippocampus during retrieval. Nature Neuroscience, 3(11), 1149–1152. https://doi.org/10.1038/80671
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429–433. https://doi.org/10.1038/415429a
Fairhall, S. L., & Macaluso, E. (2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European Journal of Neuroscience, 29(6), 1247–1257. https://doi.org/10.1111/j.1460-9568.2009.06688.x
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior research methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
Fiebelkorn, I. C., Saalmann, Y. B., & Kastner, S. (2013). Rhythmic sampling within and between objects despite sustained attention at a cued location. Current Biology, 23(24), 2553–2558. https://doi.org/10.1016/j.cub.2013.10.063
Fox, E., & de Fockert, J. W. (2001). Inhibitory effects of repeating color and shape: Inhibition of return or repetition blindness? Journal of Experimental Psychology: Human Perception and Performance, 27(4), 798. https://doi.org/10.1037/0096-1523.27.4.798
Friedman, N. P., & Miyake, A. (2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133, 101–135. https://doi.org/10.1037/0096-3445.133.1.101
Fuentes, L. J., & Santiago, E. (1999). Spatial and semantic inhibitory processing in schizophrenia. Neuropsychology, 13(2), 259. https://doi.org/10.1037/0894-4105.13.2.259
Hillyard, S. A., Kutas, M. (1983). Electrophysiology of cognitive processing. Annual Review of Psychology, 34, 33–61. https://doi.org/10.1146/annurev.ps.34.020183.000341
Holcomb, P. J. (1993). Semantic priming and stimulus degradation: Implications for the role of the N400 in language processing. Psychophysiology, 30(1), 47–61. https://doi.org/10.1111/j.1469-8986.1993.tb03204.x
Hollingworth, A., & Rasmussen, I. P. (2010). Binding objects to locations: the relationship between object files and visual working memory. Journal of Experimental Psychology: Human Perception and Performance, 36(3), 543. https://doi.org/10.1037/a0017836
Jia, J., Liu, L., Fang, F., & Luo, H. (2017). Sequential sampling of visual objects during sustained attention. PLOS Biology, 15(6), e2001903. https://doi.org/10.1371/journal.pbio.2001903
Johnson, J. A., & Zatorre, R. J. (2006). Neural substrates for dividing and focusing attention between simultaneous auditory and visual events. NeuroImage, 31(4), 1673–1681. https://doi.org/10.1016/j.neuroimage.2006.02.026
Jones, K. A., Porjesz, B., Almasy, L., Bierut, L., Dick, D., Goate, A., ..., Begleiter, H. (2006). A cholinergic receptor gene (CHRM2) affects event-related oscillations. Behavior Genetics, 36, 627–639. https://doi.org/10.1007/s10519-006-9075-6
Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing of object files: Object-specific integration of information. Cognitive Psychology, 24, 175–219. https://doi.org/10.1016/0010-0285(92)90007-O
Kanwisher, N. G. (1987). Repetition blindness: Type recognition without token individuation. Cognition, 27, 117–143. https://doi.org/10.1016/0010-0277(87)90016-3
Kanwisher, N. G. (1991). Repetition blindness and illusory conjunctions: Errors in binding visual types with visual tokens. Journal of Experimental Psychology: Human Perception and Performance, 17, 401–421. https://doi.org/10.1037/0096-1523.17.2.404
Keitel, C., & Müller, M. M. (2016). Audio-visual synchrony and feature-selective attention co-amplify early visual processing. Experimental Brain Research, 234(5), 1221–1231. https://doi.org/10.1007/s00221-015-4392-8
Klein, R. M. (2000). Inhibition of return. Trends in cognitive sciences, 4(4), 138–147. https://doi.org/10.1016/S1364-6613(00)01452-2
Klein, R. M., & Taylor, T. L. (1994). Categories of cognitive inhibition with reference to attention. In D. Dagenbach & T. H. Carr (Eds.), Inhibitory processes in attention, memory, and language (pp. 113–150). Academic Press.
Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2010). Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134(3), 372–384. https://doi.org/10.1016/j.actpsy.2010.03.010
Law, M. B., Pratt, J., & Abrams, R. A. (1995). Color-based inhibition of return. Perception & Psychophysics, 57(3), 402–408. https://doi.org/10.3758/BF03213064
Laurienti, P. J., Kraft, R. A., Maldjian, J. A., Burdette, J. H., & Wallace, M. T. (2004). Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research, 158, 405–414. https://doi.org/10.1007/s00221-004-1913-2
Leone, L. M., & McCourt, M. E. (2013). The roles of physical and physiological simultaneity in audiovisual multisensory facilitation. i-Perception, 4(4), 213–228. https://doi.org/10.1068/i0532
Loose, R., Kaufmann, C., Auer, D. P., & Lange, K. W. (2003). Human prefrontal and sensory cortical activity during divided attention tasks. Human Brain Mapping, 18(4), 249–259. https://doi.org/10.1002/hbm.10082
Lu, Z., Li, Q., Gao, N., Yang, J., & Bai, O. (2019). A novel audiovisual P300-speller paradigm based on cross-modal spatial and semantic congruence. Frontiers in Neuroscience, 13, 1040. https://doi.org/10.3389/fnins.2019.01040
Lupiáñez, J., Milliken, B., Solano, C., Weaver, B., & Tipper, S. P. (2001). On the strategic modulation of the time course of facilitation and inhibition of return. The Quarterly Journal of Experimental Psychology: Section A, 54(3), 753–773. https://doi.org/10.1080/713755990
Macaluso, E., Noppeney, U., Talsma, D., Vercillo, T., Hartcher-O’Brien, J., & Adam, R. (2016). The curious incident of attention in multisensory integration: Bottom-up vs. top-down. Multisensory Research, 29(6), 557–583. https://doi.org/10.1163/22134808-00002528
Mengotti, P., Boers, F., Dombert, P. L., Fink, G. R., & Vossel, S. (2018). Integrating modality-specific expectancies for the deployment of spatial attention. Scientific Reports, 8(1), 1210.
Miller, J. (1986). Timecourse of co-activation in bimodal divided attention. Perception & Psychophysics, 40(5), 331–343. https://doi.org/10.3758/bf03203025
Milliken, B., Tipper, S. P., Houghton, G., & Lupianez, J. (2000). Attending, ignoring, and repetition: On the relation between negative priming and inhibition of return. Perception & Psychophysics, 62, 1280–1296. https://doi.org/10.3758/BF03212130
Mishler, A., & Neider, M. (2016). Evidence for the redundant signals effect in detection of categorical targets. Journal of Vision, 16(12), 1024. https://doi.org/10.1167/16.12.1024
Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., & Laurienti, P. J. (2008). Modality-specific selective attention attenuates multisensory integration. Experimental Brain Research, 184(1), 39–52. https://doi.org/10.1007/s00221-007-1080-3
Otto, T. U., Dassy, B., & Mamassian, P. (2013). Principles of multisensory behavior. Journal of Neuroscience, 33(17), 7463–7474. https://doi.org/10.1523/JNEUROSCI.4678-12.2013
Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. Attention and Performance X: Control of Language Processes, 32, 531–556.
Pratt, J., Hillis, J., & Gold, J. M. (2001). The effect of the physical characteristics of cues and targets on facilitation and inhibition. Psychonomic Bulletin & Review, 8(3), 489–495. https://doi.org/10.3758/BF03196183
Prime, D. J., & Jolicoeur, P. (2009). Response-selection conflict contributes to inhibition of return. Journal of Cognitive Neuroscience, 21(5), 991–999. https://doi.org/10.1162/jocn.2009.21105
Raab, D. H. (1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24, 574–590. https://doi.org/10.1111/j.2164-0947.1962.tb01433.x
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108. https://doi.org/10.1037/0033-295X.85.2.59
Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in cognitive sciences, 20(4), 260–281. https://doi.org/10.1016/j.tics.2016.01.007
Redden, R. S., MacInnes, W. J., & Klein, R. M. (2021). Inhibition of return: An information processing theory of its natures and significance. Cortex, 135, 30–48. https://doi.org/10.1016/j.cortex.2020.11.009
Rothbart, M. K., Posner, M. I., & Rosicky, J. (1994). Orienting in normal and pathological development. Development and Psychopathology, 6(4), 635–652. https://doi.org/10.1017/S0954579400004715
Satel, J., Wilson, N. R., & Klein, R. M. (2019). What neuroscientific studies tell us about inhibition of return. Vision, 3(4), 58. https://doi.org/10.3390/vision3040058
Senkowski, D., Saint-Amour, D., Höfle, M., & Foxe, J. J. (2011). Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness. NeuroImage, 56(4), 2200–2208. https://doi.org/10.1016/j.neuroimage.2011.03.075
Spence, C. (2010). Crossmodal spatial attention. Annals of the New York Academy of Sciences, 1191(1), 182–200. https://doi.org/10.1111/j.1749-6632.2010.05440.x
Spence, C., Nicholls, M. E., & Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63(2), 330–336. https://doi.org/10.3758/BF03194473
Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130(4), 799. https://doi.org/10.1037/0096-3445.130.4.799
Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255–266. https://doi.org/10.1038/nrn2331
Stein, B. E., Stanford, T. R., & Rowland, B. A. (2014). Development of multisensory integration from the perspective of the individual neuron. Nature Reviews Neuroscience, 15(8), 520–535. https://doi.org/10.1038/nrn3742
Talsma, D. (2015). Predictive coding and multisensory integration: an attentional account of the multisensory mind. Frontiers in Integrative Neuroscience, 9, 19. https://doi.org/10.3389/fnint.2015.00019
Talsma, D., & Woldorff, M. G. (2005). Selective attention and multisensory integration: Multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17(7), 1098–1114. https://doi.org/10.1162/0898929054475172
Talsma, D., Doty, T. J., & Woldorff, M. G. (2007). Selective attention and audiovisual integration: Is attending to both modalities a prerequisite for early integration? Cerebral cortex, 17(3), 679–690. https://doi.org/10.1093/cercor/bhk016
Tanaka, Y., & Shimojo, S. (1996). Location vs feature: Reaction time reveals dissociation between two visual functions. Vision Research, 36(14), 2125–2140. https://doi.org/10.1016/0042-6989(95)00272-3
Tang, X., Wu, J., & Shen, Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208–224. https://doi.org/10.1016/j.neubiorev.2015.11.002
Tipper, S. P., Weaver, B., Jerreat, L. M., & Burak, A. L. (1994). Object-based and environment-based inhibition of return of visual attention. Journal of Experimental Psychology: Human Perception and Performance, 20(3), 478. https://doi.org/10.1037/0096-1523.20.3.478
Ulrich, R., Miller, J., & Schröter, H. (2007). Testing the race model inequality: An algorithm and computer programs. Behavior Research Methods, 39(2), 291–302. https://doi.org/10.3758/BF03193160
Van der Stoep, N., Spence, C., Nijboer, T. C. W., & Van der Stigchel, S. (2015). On the relative contributions of multisensory integration and crossmodal exogenous spatial attention to multisensory response enhancement. Acta Psychologica, 162, 20–28. https://doi.org/10.1016/j.actpsy.2015.09.010
Van der Stoep, N., Van der Stigchel, S., & Nijboer, T. C. W. (2015). Erratum to: Exogenous spatial attention decreases audiovisual integration. Attention Perception & Psychophysics, 77(2), 464–482. https://doi.org/10.3758/s13414-014-0785-1
Van der Stoep, N., Van der Stigchel, S., Nijboer, T. C. W., & Spence, C. (2017). Visually induced inhibition of return affects the integration of auditory and visual information. Perception, 46(1), 6–17. https://doi.org/10.1177/0301006616661934
Vroomen, J., Bertelson, P., & De Gelder, B. (2001). The ventriloquist effect does not depend on the direction of automatic visual attention. Perception & Psychophysics, 63(4), 651–659. https://doi.org/10.3758/BF03194427
Wang, A., Wu, X., Tang, X., & Zhang, M. (2020). How modality processing differences affect cross-modal non-spatial repetition inhibition. PsyCh journal, 9(3), 306–315. https://doi.org/10.1002/pchj.332
Wang, L., Yue, Z., & Chen, Q. (2012). Cross-modal non-spatial repetition inhibition. Attention, Perception, & Psychophysics, 74(5), 867–878. https://doi.org/10.3758/s13414-012-0289-9
Weissman, D. H., Giesbrecht, B., Song, A. W., Mangun, G. R., & Woldorff, M. G. (2003). Conflict monitoring in the human anterior cingulate cortex during selective attention to global and local object features. NeuroImage, 19, 1361–1368. https://doi.org/10.1016/S1053-8119(03)00167-8
White, H. A. (2007). Inhibitory control of proactive interference in adults with ADHD. Journal of Attention Disorders, 11(2), 141–149. https://doi.org/10.1177/1087054706295604
Woodworth, R. S. (1938). Experimental psychology. Holt, New York.
Wu, X., Wang, A., Tang, X., & Zhang, M. (2019). Different visual and auditory latencies affect cross-modal non-spatial repetition inhibition. Acta Psychologica, 200, 102940. https://doi.org/10.1016/j.actpsy.2019.102940
Wu, X., Wang, A., & Zhang, M. (2020). Cross-modal nonspatial repetition inhibition: An ERP study. Neuroscience Letters, 734, 135096. https://doi.org/10.1016/j.neulet.2020.135096
Yang, W. P., Chu, B. Q., Yang, J. J., Yu, Y. H., Wu, J. L., & Yu, S. Y. (2014). Elevated audiovisual temporal interaction in patients with migraine without aura. The Journal of Headache and Pain, 15(1), 44. https://doi.org/10.1186/1129-2377-15-44
Zhang, M., Wu, X., & Wang, A. (2021). Crossmodal non-spatial repetition inhibition due to modality shift. Perception, 50(2), 116–128. https://doi.org/10.1177/0301006620988209
Funding
This research was supported by the Suzhou Science and Technology Development Plan (People’s Livelihood Science and Technology: SKY2022113), the 14th five-year plan of Jiangsu Province Education Science (B/2021/01/87), the Humanities and Social Sciences Research Project of Soochow University (22XM0017) and Interdiscipline Research Team of Humanities and Social Sciences of Soochow University (2022). X.T. was supported by the Youth Project of Humanities and Social Sciences Financed by Ministry of Education of China (22YJC190020). M.Z. was supported by the National Natural Science Foundation of China (31871092) and the Japan Society for the Promotion of Science KAKENHI (20K04381).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Competing interest
The authors declare that they have no known competing for financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, X., Tang, X., Wang, A. et al. Non-spatial inhibition of return attenuates audiovisual integration owing to modality disparities. Atten Percept Psychophys (2023). https://doi.org/10.3758/s13414-023-02825-y
Accepted:
Published:
DOI: https://doi.org/10.3758/s13414-023-02825-y