No CrossRef data available.
Article contents
A Phenomenological Approach to Wearable Technologies and Viscerality: From embodied interaction to biophysical music performance
Published online by Cambridge University Press: 16 November 2023
Abstract
During the past decade, embodied knowledge has provided novel important insights to rethink mediation technology, thereby paving the way for a transdisciplinary approach to wearable technologies. Stemming from a phenomenological-based approach and considering current trends in sonic interaction design, this article proposes an extensive account on embodied approaches to mediation technology and underlines the increasing importance of somatic knowledge within the field. It also presents an autoethnographic analysis of my own performance, which provides an original contribution to the artistic application of wearable technologies. Stemming from an ongoing research-creation on musical improvisation with biophysical technologies, the case study emphasises how an embodied and visceral approach to interaction can transform wearable devices into an active sensory-perceptual mode of experiencing, which is capable of stimulating the performer’s sensorimotor metaplasticity. The reconfiguration of a body’s automations through the use of sound feedback is a process that unfolds with a high degree of sensitivity in which the body can be poetically understood as an emergent territoriality, inhabited and transfigured by the sound.
- Type
- Article
- Information
- Copyright
- © The Author(s), 2023. Published by Cambridge University Press
References
REFERENCES
Addessi, A. R., Maffioli, M. and Anelli, F. 2015. The Miror Platform for young Children’S Music and Dance Creativity: Reflexive Interaction Meets Body-Gesture, Embodied Cognition, and Laban Educational Dance. Perspectives: Journal of the Early Childhood Music & Movement Association 10(1): 9–17.Google Scholar
Agamben, G. 2009. What is an Apparatus? And Other Essays. Stanford, CA: Stanford University Press.Google Scholar
Arfib, D., Couturier, J. M., Kessous, L. and Verfaille, V. 2002. Strategies of Mapping between Gesture Data and Synthesis Model Parameters Using Perceptual Spaces. Organised Sound 7(2): 127–44.CrossRefGoogle Scholar
Barrett, G. D. 2022. ‘How We Were Never Posthuman’: Technologies of the Embodied Voice in Pamela Z’s Voci. Twentieth-Century Music 19(1): 3–27.CrossRefGoogle Scholar
Bedau, M. A. 2008. Downward Causation and Autonomy in Weak Emergence. In Bedau, M. A. and Humphreys, P. (eds.) Emergence. Contemporary Readings in Philosophy and Science. Cambridge, MA: MIT Press, 155–88.CrossRefGoogle Scholar
Bernard, M. 2001. De la creation choregraphique. Pantin, France: Centre national de la danse.Google Scholar
Bevilacqua, F., Boyer, E. O., Françoise, J., Houix, O., Susini, P., Roby-Brami, A. and Hanneton, S. 2016. Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies. Frontiers in Neuroscience 10. doi.org/10.3389/fnins.2016.00385
CrossRefGoogle ScholarPubMed
Bevilacqua, F., Schnell, N., Françoise, J., Boyer, É. O., Schwarz, D. and Caramiaux, B. 2017. Designing Action–Sound Metaphors Using Motion Sensing and Descriptor-Based Synthesis of Recorded Sound Materials. In Lesaffre, M., Maes, P.-J. and Leman, M (eds.) The Routledge Companion to Embodied Music Interaction. New York: Routledge, 391–401.CrossRefGoogle Scholar
Bresin, R., Hermann, T. and Hunt, A. 2012. Interactive Sonification. Journal on Multimodal User Interfaces 5(3-4): 85–6.CrossRefGoogle Scholar
Broadhurst, S. and Machon, J. (eds.) 2006. Performance and Technology: Practices of Virtual Embodiment and Interactivity. London: Palgrave Macmillan.CrossRefGoogle Scholar
Burger, B., Saarikallio, S., Luck, G., Thompson, M. R. and Toiviainen, P. 2013. Relationships between Perceived Emotions in Music and Music-Induced Movement. Music Perception 30(5): 517–33.CrossRefGoogle Scholar
Camurri, A., Hashimoto, S., Ricchetti, M., Ricci, A., Suzuki, K., Trocca, R. and Volpe, G. 2000. Eyesweb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems. Computer Music Journal 24(1): 57–69.CrossRefGoogle Scholar
Choinière, I. 2015. Intercorporeality and Technology: Toward a New Cognitive, Aesthetic and Communicative Paradigm in the Performing Arts. Doctoral thesis, Plymouth University.Google Scholar
Choinière, I. 2018. Somatic/Embodiment/Technology as an Evolutive Strategy: The Ontological Shift of the Performative Body in Contact with Technologies. Journal of Dance & Somatic Practices 10(2): 189–206.CrossRefGoogle Scholar
Choinière, I. 2020. Médiation phénoménale du corps vécu. Embodiment et corporéalité en émergence sous l’effet des technologies. Chiasmi International 22: 333–49.CrossRefGoogle Scholar
Clark, A. 2008. Supersizing the Mind: Embodiment, Action and Cognitive Extension. Oxford: Oxford University Press.CrossRefGoogle Scholar
Cox, A. 2011. Embodying Music: Principles of the Mimetic Hypothesis. Music Theory Online, 17(2). https://mtosmt.org/issues/mto.11.17.2/mto.11.17.2.cox.php (accessed 22 February 2023).CrossRefGoogle Scholar
Csikszentmihalyi, M. 2014. Flow and the Foundations of Positive Psychology. Dordrecht: Springer Netherlands.CrossRefGoogle Scholar
Davidson, A. 2016. Ontological Shifts: Multi-Sensoriality and Embodiment in a Third Wave of Digital Interfaces. Journal of Dance & Somatic Practices 8(1): 21–42.CrossRefGoogle Scholar
Davies, C. 2003 Landscape, Earth, Body, Being, Space and Time in the Immersive. In Malloy, J. (ed.) Women, Art and Technology. Cambridge, MA: MIT Press, 322–37.Google Scholar
Davies, C. 2004. Virtual Space. In Penz, F., Radick, G. and Howell, R. (eds.) Space: In Science, Art and Society. Cambridge: Cambridge University Press, 69–104.Google Scholar
Degara, N., Hunt, A. and Hermann, T. 2015. Interactive Sonification. IEEE MultiMedia 22(1): 20–3.CrossRefGoogle Scholar
Di Scipio, A. 2003. ‘Sound is the Interface’: From Interactive to Ecosystemic Signal Processing. Organised Sound 8(3): 269–77.CrossRefGoogle Scholar
Donnarumma, M. 2011. XTH SENSE: A Study of Muscle Sounds for an Experimental Paradigm of Musical Performance. Proceedings of the 2011 International Computer Music Conference. University of Huddersfield, UK: ICMC, 243–8.Google Scholar
Donnarumma, M. 2016. Configuring Corporeality: Performing Bodies, Vibrations and New Musical Instruments. Doctoral thesis, Goldsmiths University of London.Google Scholar
Donnarumma, M. 2017. Beyond the Cyborg: Performance, Attunement and autonomous Computation. International Journal of Performance Arts and Digital Media 13(2): 1–15.CrossRefGoogle Scholar
Dourish, P. 2001. Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Françoise, J., Candau, Y., Fdili Alaoui, S. and Schiphorst, T. 2017. Designing for Kinesthetic Awareness: Revealing User Experiences through Second-Person Inquiry. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver Colorado USA. ACM, 5171–83.Google Scholar
Françoise, J., Fdili Alaoui, S., Schiphorst, T. and Bevilacqua, F. 2014. Vocalizing Dance Movement for Interactive Sonification of Laban Effort Factors. Proceedings of the 2014 Conference on Designing Interactive Systems, Vancouver BC Canada. ACM, 1079–82.Google Scholar
Giomi, A. 2020a. Somatic Sonification in Dance Performances. From the Artistic to the Perceptual and Back. Proceedings of the 7th International Conference on Movement and Computing, Jersey City/Virtual NJ USA. ACM, 1–8.Google Scholar
Giomi, A. 2020b. Virtual Embodiment: An Understanding of the Influences of Merleau-Ponty’s Philosophy of Technology on Performance and Digital Media. Chiasmi International 22: 297–315.CrossRefGoogle Scholar
Giomi, A. and Fratagnoli, F. 2018. Listening Touch: A Case Study about Multimodal Awareness in Movement Analysis with Interactive Sound Feedback. Proceedings of the 5th International Conference on Movement and Computing, Genoa, Italy. ACM, 1–8.Google Scholar
Giomi, A. and Leonard, J. 2020. Towards an Interactive Model-Based Sonification of Hand Gesture for Dance Performance. Proceedings of the International Conference on New Interfaces for Musical Expression, Birmingham, UK, 369–74.Google Scholar
Godøy, R. I. 2014. Understanding Coarticulation in Musical Experience. Sound, Music and Motion: 10th International Symposium CMMR 2013, Marseille, France, 535–47.Google Scholar
Godøy, R. I. 2018. Sonic Object Cognition. In Beder, R. (ed.) Springer Handbook of Systematic Musicology. New York: Springer Berlin Heidelberg, 761–77.CrossRefGoogle Scholar
Gromala, D. J. 2007. Towards a Phenomenological Theory of the Visceral in the Interactive Arts. Doctoral thesis, Plymouth University.Google Scholar
Grosshauser, T., Bläsing, B., Spieth, C. and Hermann, T. 2012. Wearable Sensor-Based Real-Time Sonification of Motion and Foot Pressure in Dance Teaching and Training. Journal of the Audio Engineering Society 60(7/8): 580–9.Google Scholar
Hansen, M. B. N. 2006. Bodies in Code: Interfaces with Digital Media. New York: Routledge.Google Scholar
Haraway, D. J. 1991. Simians, Cyborgs and Women: The Reinvention of Nature. New York: Routledge.Google Scholar
Hayles, N. K. 1999. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature and Informatics. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Hunt, A. and Hermann, T. 2004. The Importance of Interaction in Sonification. Proceedings of the 04-Tenth Meeting of the International Conference on Auditory Display. Sydney: ICAD, 1–7.Google Scholar
Ihde, D. and Malafouris, L. 2019. Homo faber Revisited: Postphenomenology and Material Engagement Theory. Philosophy & Technology 32(2): 195–214.CrossRefGoogle ScholarPubMed
Jensenius, A. R. and Bjerkestrand, K. A. V. 2012. Exploring Micromovements with Motion Capture and Sonification. In Brooks, A. L. (ed.) Arts and Technology, Vol. 101. Berlin: Springer Berlin, 100–7.CrossRefGoogle Scholar
Kim, Y. and Gilman, S. L. 2019. The Oxford Handbook of Music and the Body. Oxford: Oxford University Press.CrossRefGoogle Scholar
Knapp, R. B. and Lusted, H. S. 1990. A Bioelectric Controller for Computer Music Applications. Computer Music Journal 14(1): 42–7.CrossRefGoogle Scholar
Kozel, S. 2007. Closer. Performance, Technologies, Phenomenology. Cambridge, MA: MIT Press.Google Scholar
Kuppers, P. 2006. Addenda, Phenomenology, Embodiment: Cyborgs and Disability Performance. In Broadhurst, S. and Machon, J. (eds.) Performance and Technology. London: Palgrave Macmillan, 169–80.CrossRefGoogle Scholar
Kuypers, P. 2006. Des Trous Noirs. Un entretien avec Hubert Godard. Nouvelles de Danse: Scientifiquement Danse, 53: 56–75.Google Scholar
Leman, M. 2008. Embodied Music Cognition and Mediation Technology. Cambridge, MA: MIT Press.Google Scholar
Leman, M. and Maes, P.-J. 2015. The Role of Embodiment in the Perception of Music. Empirical Musicology Review, 9(3–4): 236–46.CrossRefGoogle Scholar
Leman, M., Lesaffre, M. and Maes, P. J. 2017. Introduction: What Is Embodied Music Interaction? In Lesaffre, M., Maes, P.-J. and Leman, M. (eds.) The Routledge Companion to Embodied Music Interaction. New York: Routledge: 1–10.Google Scholar
Lesaffre, M. 2018. Investigating Embodied Music Cognition for Health and Well-Being. In Bader, R. (ed.) Springer Handbook of Systematic Musicology. Berlin: Springer, 779–91.CrossRefGoogle Scholar
Lesaffre, M., Maes, P.-J. and Leman, M. (eds.) 2017. The Routledge Companion to Embodied Music Interaction. New York: Routledge.CrossRefGoogle Scholar
Lucier, A. 1976. Statement on: Music for Solo Performer. In Rosenboom, D. (ed.) Biofeedback and the Arts, Results of Early Experiments. Vancouver: Aesthetic Research Center of Canada Publications, 60–1.Google Scholar
Machon, J. 2009. (Syn)Aesthetics. Redefining Visceral Performance. London: Palgrave Macmillan.CrossRefGoogle Scholar
Maes, P.-J., Dyck, E. V., Lesaffre, M., Leman, M. and Kroonenberg, P. M. 2014. The Coupling of Action and Perception in Musical Meaning Formation. Music Perception 32(1): 67–84.CrossRefGoogle Scholar
Malafouris, L. 2013. How Things Shape the Mind: A Theory of Material Engagement. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Marshall, P. and Hornecker, E. H. 2013. Theories of Embodiment in HCI. In Price, S., Jewitt, C. and Brown, B. (eds.) The SAGE Handbook of Digital Technology Research. London: Sage, 144–58.CrossRefGoogle Scholar
Massumi, B. 2002. Parables for the Virtual: Movement, Affect, Sensation. Durham, NC: Duke University Press.Google Scholar
Matyja, J. R. and Schiavio, A. 2013. Enactive Music Cognition: Background and Research Themes. Constructivist Foundations 8(3): 351–7.Google Scholar
McLuhan, M. [1964] 1994. Understanding Media: The Extensions of Man. Cambridge, MA: MIT Press.Google Scholar
Menary, R. 2010. Introduction to the Special Issue on 4E Cognition. Phenomenology and the Cognitive Sciences 9(4): 459–63.CrossRefGoogle Scholar
Miranda, E. R. 2006. Brain-Computer Music Interface for Composition and Performance. International Journal on Disability and Human Development 5(2): 119–26.CrossRefGoogle Scholar
Murray-Browne, T., Mainstone, D., Bryan-Kinns, N. and Plumbley, M. D. 2011. The Medium is the Message: Composing Instruments and Performing Mappings. Proceedings of the International Conference on New Interfaces for Musical Expression, Oslo, Norway, 56–9.Google Scholar
Niewiadomski, R., Mancini, M., Cera, A., Piana, S., Canepa, C. and Camurri, A. 2019. Does Embodied Training Improve the Recognition of Mid-Level Expressive Movement Qualities Sonification? Journal on Multimodal User Interfaces 13(3): 191–203.CrossRefGoogle Scholar
Nijs, L. 2017. The Merging of Musician and Musical Instrument: Incorporation, Presence and Levels of Embodiment. In Lesaffre, M., Maes, P.-J. and Leman, M (eds.) The Routledge Companion to Embodied Music Interaction. New York: Routledge, 49–57.CrossRefGoogle Scholar
Nijs, L., Lesaffre, M. and Leman, M. 2013. The Musical Instrument as a Natural Extension of the Musician. In Castellengo, M., Genevois, H. and Bardez, J.-M. (eds.) Music and Its Instruments. Sampzon: Editions Delatour, 467–84.Google Scholar
Nymoen, K., Godøy, R. I., Jensenius, A. R. and Torresen, J. 2013. Analyzing Correspondence between Sound Objects and Body Motion. ACM Transactions on Applied Perception 10(2): 1–22.CrossRefGoogle Scholar
Pamela, Z, 2003. A Tool is a Tool. In Malloy, J. (ed.) Women, Art and Technology. Cambridge, MA: MIT Press, 348–61.Google Scholar
Parisi, F. 2019. Temporality and metaplasticity. Facing Extension and Incorporation through Material Engagement Theory. Phenomenology and the Cognitive Sciences 18(1): 205–21.Google Scholar
Pitozzi, E. 2016. Body Soundscape: Perception, Movement and Audiovisual Developments in Contemporary Dance. In Kaduri, Y. (ed.), The Oxford Handbook of Music, Sound and Image in the Fine Arts. New York: Oxford University Press, 256–88.Google Scholar
Quinz, E. and Menicacci, A. 2006. Étendre la perception? Biofeedback et transferts intermodaux en danse. In Nouvelles de Danse: Scientifiquement Danse. Quand la danse puise aux sciences et réciproquement. Brussels: Contredanse, 76–96.Google Scholar
Rizzolatti, G. and Sinigaglia, C. 2008. Mirrors in the Brain: How Our Minds Share Actions and Emotions. New York: Oxford University Press.Google Scholar
Rodgers, T. 2010. Pink Noises: Women on Electronic Music and Sound. Durham, NC: Duke University Press.Google Scholar
Schiller, G. 2006. Kinaesthetic Traces across Material Forms: Stretching the Screen’S Stage. In Broadhurst, S. and Machon, J. (eds.) Performance and Technology: Practices of Virtual Embodiment and Interactivity. London: Palgrave Macmillan, 100–11.CrossRefGoogle Scholar
Smith, H. and Dean, R. T. (eds.) 2009. Practice-Led Research, Research-Led Practice in the Creative Arts. Edinburgh: Edinburgh University Press.CrossRefGoogle Scholar
Stelarc. 1991. Prosthetics, Robotics and Remote Existence: Postevolutionary Strategies. Leonardo 24(5): 591–5.CrossRefGoogle Scholar
Tanaka, A. 1993. Musical Issues in Using Interactive Instrument Technology with Application to the BioMuse. Proceedings of the 1993 International Computer Music Conference, Waseda University, Tokyo, Japan. ICMC, 124–6.Google Scholar
Tanaka, A. 2012. Biomuse to Bondage: Corporeal Interaction in Performance and Exhibition. In Chatzichristodoulou, M. and Zerihan, R. (eds.) Intimacy across Visceral and Digital Performance. London: Palgrave Macmillan, 159–69.CrossRefGoogle Scholar
Tanaka, A. and Donnarumma, M. 2019. The Body as Musical Instrument. In Kim, Y. and Gilman, S. L. (eds.) The Oxford Handbook of Music and the Body. Oxford: Oxford University Press, 78–96.Google Scholar
Timmers, R. and Granot, R. Y. 2016. Music as a Multimodal Experience. Psychomusicology: Music, Mind and Brain 26(2): 101–2.CrossRefGoogle Scholar
Van Nort, D. 2015. [Radical] Signals from Life: From Muscle Sensing to Embodied Machine Listening/Learning within a Large-Scale Performance Piece. Proceedings of the 2nd International Workshop on Movement and Computing, Vancouver, BC, Canada. ACM, 124–7.Google Scholar
Varela, F. J., Thompson, E. and Rosch, E. 1991. The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Visi, F., Schramm, R. and Miranda, E. 2014. Gesture in Performance with Traditional Musical Instruments and Electronics: Use of Embodied Music Cognition and Multimodal Motion Capture to Design Gestural Mapping Strategies. Proceedings of the 2014 International Workshop on Movement and Computing – MOCO ’14, Paris, France. ACM, 100–5.Google Scholar
Votava, P. and Berger, E. 2015. The Heart Chamber Orchestra An Audio-Visual Real-Time Performance for Chamber Orchestra Based on Heartbeats. eContact! 14(2). https://econtact.ca/14_2/votava-berger_hco.html (accessed 22 February 2023).Google Scholar
Waisvisz, M. 1985. The Hands: A Set of Remote MIDI-Controllers. Proceedings of the 1993 International Computer Music Conference, Burnaby, BC, Canada, ICMC, 313–18.Google Scholar
Wechsler, R. 2006. Artistic Considerations in the Use of Motion Tracking with Live Performers: A Practical Guide. In Broadhurst, S. and Machon, J (eds.) Performance and Technology: Practices of Virtual Embodiment and Interactivity. London: Palgrave Macmillan, 60–77.CrossRefGoogle Scholar
Yang, J., Hermann, T. and Bresin, R. 2019. Introduction to the Special Issue on Interactive Sonification. Journal on Multimodal User Interfaces 13(3): 151–3.CrossRefGoogle Scholar
Zbyszyński, M., Di Donato, B., Visi, F. G. and Tanaka, A. 2021. Gesture-Timbre Space: Multidimensional Feature Mapping Using Machine Learning and Concatenative Synthesis. Perception, Representations, Image, Sound, Music: 14th International Symposium, CMMR 2019, Marseille, France, 600–22.Google Scholar