Tracking multimodal cohesion in Audio Description: Examples from a Dutch audio description corpus

Nina Reviers

Abstract


One of the main questions addressed by multimodality research—the main conceptual framework for analysing audiovisual texts—is how the different modes of audiovisual texts combined—visual, verbal, aural—create supplementary meaning in texts, over and above the meanings conveyed by the individual constituents. Ensuring that this multimodal interaction or multimodal cohesion remains intact is a key challenge in the practice of audiovisual translation (AVT), and particularly in Audio Description (AD) for the blind and visually impaired. The present article therefore studies the functioning of multimodal cohesion in audio-described texts by analysing the types of interaction between descriptive units and sound effects in a selection of Dutch audio-described films and series. The article begins with a detailed description of the methodology which is based on multimodal transcription and concludes with an overview of the types of multimodal cohesive relations identified.


Keywords


Media Accessibility; Audio Description; Corpus Studies; Multimodality

Full Text:

PDF

References


Baldry, A. , & Thibault, P. J. (2006). Multimodal transcription and text analysis: A multimodal toolkit and coursebook with associated online course. London: Equinox. 

Bordwell, D., & Thompson, K. (2013). Film art: An introduction (9th ed.). New York, NY: McGraw-Hill.

Braun, S. (2011). Creating coherence in audio description. Meta: Translators' Journal, 56(3), 645–662.

Chion, M. (1994). Audio-vision: Sound on screen. New York, NY: Columbia University Press.

Fryer, L. (2010). Audio description as audio drama: A practitioner’s point of view. Perspectives: Studies in Translatology, 8(3), 205–213.

Halliday, M. A. K., & Matthiessen, C. (2014). Halliday’s introduction to functional grammar. London: Routledge.

Hirvonen, M., & Tiittula, L. (2010). A method for analysing multimodal research material: Audio description in focus. Electronic proceedings of the KäTu symposium on translation and interpreting studies. Retrieved from: https://www.sktl.fi/@Bin/40698/Hirvonen%26Tiittula_MikaEL2010.pdf (last accessed 05-10-2017).

Independent Television Commission (ITC). (2000). ITC guidance on standards for audio description. London: ITC.

Orero, P., & Szarkowska, A. (2014). The importance of sound for audio description. In P. Orero, A. Matamala, & A. Maszerowska (Eds.), Audio description: New perspectives illustrated (pp. 121–139). Amsterdam: John Benjamins.

Pérez-González, L. (2014). Audiovisual translation: Theories, methods, issues. London: Routledge.

Remael, A. (2012). For the use of sound. Audio description and sound: A few key issues. Monti: Multidisciplinary in Audiovisual Translation, 4, 255–276.

Remael, A., & Reviers, N. (2018). Multimodal cohesion in accessible films: A first inventory. In L. Pérez-González (Ed.), The Routledge handbook of audiovisual translation (pp. 260–280). London: Routledge.

Reviers, N., Remael, A., & Daelemans, W. (2015). The language of audio description in Dutch: Results of a corpus study. In A. Jankowska &. A. Szarkowska (Eds.), New points of view on audiovisual translation and accessibility (pp. 167–189). Bern: Peter Lang.

Reviers, N. (2016). On context and intersemiotic cohesion in audio description. In M. Kažimír (Ed.), Kontexty: Interdisciplinárny zborník (pp. 272–297). Gorlice: Ośrodek Kultury Prawosławnej.

Royce, T. (2007). Intersemiotic complementarity: A framework for multimodal discourse analysis. In T. B. Royce W. (Ed.), New directions in the analysis of multimodal discourse (pp. 63–109). Mahwah, NJ: Lawrence Erlbaum. 

Taylor, C. (2003). Multimodal transcription in the analysis, translation and subtitling of Italian films. The Translator, 9, 191–205.

Tseng, C.-I. (2013). Cohesion in film: Tracking film elements. Basingstoke: Palgrave Macmillan.

Van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge. 

Van Looy, E. (2008). Loft [film]. Belgium.

Zabalbeascoa, P. (2008). The nature of the audiovisual text and its parameters. In J. Díaz Cintas (Ed.), The didactics of audiovisual translation (pp. 21–37). Amsterdam: John Benjamins.