Why methods matter : Approaching multimodality in translation research

The study of multimodal phenomena calls upon translation scholars to cross disciplinary boundaries and adopt a range of theoretical and methodological approaches. The diversity of the multimodal landscape brings about research challenges that must be carefully addressed to ensure that these research efforts yield useful and credible results. This special issue is dedicated to a discussion on how to engage in multimodal translation research: how traditional research methods can be adapted and what kinds of novel approaches can be adopted or developed in order to deal with a diversity of multimodal data. In this introduction, we first discuss definitions of mode and multimodality and reflect on the nature of multimodality as a topic of research within Translation Studies. We then explain our rationale for dedicating the special issue to research methods and introduce three areas of multimodal translation research that, in our view, merit particular attention from a methodological point of view. Finally, we introduce the articles contained in this special issue.


Introduction
Communication in today's society has become primarily multimodal: we encounter combinations of modes -words, images, sounds and so on -whenever we look at our computers, smartphones or television screens.Multimodality is also highly relevant to translation: a significant number of the texts being translated nowadays are multimodal, ranging from user manuals, websites, textbooks and comics to audiovisual products such as films and videogames.The translation and the translational enquiry of such texts needs to include a careful consideration of both the individual meaning-making resources involved and the way they tie together to form a multimodal entity.Accessible translation, such as audio description (AD), is also essentially multimodal, because it involves translating information from one mode to another, from visually perceived stimuli to verbal information.
Multimodality offers an intriguing special case of methodological debate in Translation Studies, because it requires us to face up to the task of redefining what we are researching.The distinguishing feature of multimodal materials is the presence and interaction of the different modes, and it is therefore often necessary to place added emphasis in research on the non-verbal elements of a multimodal message instead of focusing exclusively on the verbal text.Translation Studies has a long tradition of exploring verbal texts, but when other modes are introduced into the research design, we are faced with this question: How do we find the optimal methods to deal with them all and in particular with their interplay with one another.There is still much to discover about the methods and tools that can help us explain how translation operates in multimodal contexts.
This special issue aims to contribute to the dissemination of information about different ways to approach multimodality in translation research and also to the encouragement of increasingly transparent and rigorous research in the field.The contributors to this special issue accordingly explore how multimodal topics have been and can be approached in Translation Studies, what the benefits and challenges associated with various methods are and how the methods can be applied to different contexts.
In what follows, we first introduce multimodality, discuss the various definitions that exist of the concept of "mode" and explore the kinds of modes that are typically present in multimodal texts.Then we discuss the nature of multimodality and multimodal translation as topics of research.We also explain why we chose methods as the focus of this special issue and introduce a few areas of multimodal translation research that are particularly interesting from a methodological perspective and in the context of this special issue.Finally, we introduce the articles contained in this special issue.

Multimodality and modes
The concept of multimodality has attracted growing research interest since the early 2000s.Multimodally oriented research-or "multimodal studies" (O'Halloran & Smith, 2011, p. 1)-has its roots in Gunther Kress and Theo van Leeuwen's seminal works, Reading Images: The Grammar of Visual Design (1996) and Multimodal Discourse: The Modes and Media of Contemporary Communication (2001), both of which built on M.A.K. Halliday's (1978) notion of language as a social-semiotic system and a systemic functional grammar.As its starting point, this approach takes the three metafunctions Halliday proposed for verbal language and extends them in order to apply them to other forms of communication, such as images and music (Jewitt, 2014, p. 129).The systemic functional perspective on multimodality is still prominent in multimodal studies (see section 3.2.1).
Multimodality first became a focus of interest in areas of research such as semiotics and applied linguistics, but it has since become an increasingly important perspective to research in practically all disciplines that examine communication, including studies in visual arts, design, media and education.As Jewitt (2013) describes it, multimodal studies aims to understand "communication and representation as more than language" (p.250) and, therefore, to challenge the previous predominance of language in research.One could therefore argue that multimodal studies has emerged from the need to understand how verbal meaning-making resources relate to and are affected by resources that are nonverbal-meaning all the other types of resources that can be employed to convey meaning.
Multimodality is typically defined from one of two perspectives: it can be described either as the coexistence of multiple modes within a particular context (e.g., Gibbons, 2012, p. 8) or as the process of decoding the coexisting modes from a viewer's or a reader's standpoint (e.g., Everett, 2015, p. 3).The latter emphasises that coexisting modes do not actually interact unless they are being interpreted by someone; multimodality is understood as being the interaction of modes in the cognitive system of the viewer or the reader.
In order to understand either of these definitions, we must reflect on what we mean by mode.According to Gunther Kress, a mode is a resource for meaning-making, one used for representation and communication (Kress, 2010, p. 82).Multimodal studies typically proposes that verbal language constitutes a mode (for a discussion of whether spoken language and written language can be understood as separate modes, see, for example, Ketola, 2018, p. 31).Yet, the question of what else constitutes a mode is not a straightforward one: this typically depends on the nature of the multimodal material that is being analysed as well as on the research questions that are being asked.Instead of offering fixed classifications with a set number of modes, definitions of mode in the multimodal studies literature therefore tend to offer open-ended lists of examples of modes.Those that are often mentioned in these lists include images, moving images, layout, gestures, music, and sound (see, for example, Adami, 2017, p. 451;Kress, 2010, p. 79;Page, 2010, p. 1).
In part, this lack of definiteness is a function of the diversity of multimodal artefacts and events, but it also stems from the social-semiotic origins and orientations prevalent in multimodal studies: the social-semiotic approach to multimodality defines modes from a socially specific perspective.In other words, what constitutes a mode "depends on the social group who uses it and the range of meanings that the group can express through its resources" (Adami, 2017, p. 464).Kress uses font as an example in his reflection of what constitutes a mode: for others, font might be a mere property or sub-mode of written text (cf.Pauwels, 2012, p. 250), but for a community of graphic designers it is likely to be a "full means for representation" and it is therefore a mode for that community (Kress, 2010, p. 87).Adami's (2017, p. 464) reflection on the topic offers an even more specific example: she proposes that for a community of wine tasters, wine constitutes a fully articulate mode with modal resources such as colour, aroma and taste.
In multimodal studies, the term mode is sometimes also used in connection with physiological or sensory channels such as seeing (the visual mode), hearing (aural/auditory mode), touching (tactile mode), smelling (the olfactory mode) and tasting (gustatory mode).Others refer to these physiological channels as sensory modalities.For the reader, the unfortunate overlap between these terms can be slightly misleading.It is therefore worth emphasising that, for instance, the term visual mode takes on different definitions depending on the material that is being analysed, as we can see in the articles contained in this special issue: it may refer to still images (articles examining illustrated texts), moving images (articles examining audiovisual material) or information that is perceived with sight (articles examining AD).
The non-verbal mode that has so far sparked the most research interest in multimodally oriented translation research is probably the still image.The role of images in translation has been examined, for instance, in print advertisements (e.g., Torresi, 2007;Millán-Varela, 2004), children's picture books (e.g., Oittinen, Ketola, & Garavini, 2018;Van Meerbergen, 2010), comics (e.g., Kaindl, 2004) and illustrated technical texts (e.g., Ketola, 2016Ketola, , 2018)).Naturally, audiovisual translation is also an area of research where it is extremely interesting to consider the functions of modes and the overall modal configuration of a multimodal text.In filmic AD, for instance, visually perceived information is converted into an oral verbal form.In dubbing, the translation needs to fit into strict limitations posed by the other modes of the source text (ST; for instance, visual synchrony).In subtitling, the translation is presented as an additional mode-written text-which competes for the viewer's visual attention.
Audiovisual translation has, generally speaking, a prominent presence in studies on multimodal translation.Because it integrates both aural and visual modes, it provides a good example of many of the characteristics and challenges typical of multimodality.As such, it could even be called an "archetypal" (Perego, 2012, p. 7) form of multimodal translation.Of course, research should not represent only one area of multimodality and audiovisual translation alone cannot answer all the questions regarding multimodality.However, as audiovisual translations and their research can address questions of multimodality in an enlightening way, we think it is justifiable to use them as example cases to represent multimodality whenever they illuminate a relevant point.

Multimodal translation as a cluster concept
As the multiplicity and tentativeness of the preceding definitions demonstrates, multimodality cannot be described as a monolithic concept.Instead, it covers a wide variety of genres, forms of communication, and combinations of modes and semiotic resources.This means that multimodality can also be associated with many kinds of translation.It would therefore be almost impossible to give a single, all-encompassing definition of what multimodal translation means.Accordingly, rather than labelling it a simple, easily distinguishable phenomenon, multimodal translation could be called a "cluster concept".Maria Tymoczko (2007, pp. 83-90) has proposed the idea of a cluster concept as a way to make sense of all translation in an inclusive way.In her words, translation "cannot [simply be defined in terms of necessary and sufficient features" (Tymoczko, 2007, p. 85).Translations cannot be distinguished from other products or phenomena by some characteristic that all translations would have in common, but they do have "partial and overlapping similarities" which allow us to use and understand the concept of translation (Tymoczko, 2007, p. 85). Similarly, Luis Pérez-González (2014a, pp. 141-142) points out that even the field of audiovisual translation on its own is so diverse that it requires multiple definitions, and he consequently endorses a flexible theoretical understanding of translation as a cluster concept.
The same arguments apply to multimodal translation: it certainly covers a multiplicity of activities that do not fit under a single definition with clear boundaries.It would therefore not be a practical goal to try to formulate a unified definition of multimodal translation.Instead, research could aim to explore as many contexts and genres as possible in order to develop the most comprehensive understanding possible of the variety of ways in which this cluster concept becomes actualized.
Describing multimodal translation as a cluster concept leads us to a fundamental question: Do we understand multimodal translation in the same way as any other kind of translation or is it a different phenomenon?The fact that a variety of modes are involved means that the translational activities do become varied.For example, intersemiotic translation, such as translating visual information into verbal information in AD, is a special case of multimodal translation that expands the definition of translation, because it takes place between verbal and non-verbal modes.In fact, all multimodal translations must take non-verbal modes into account to some extent, and they may introduce new challenges to the translator's task, such as modifying visual information or introducing shifts in the translation of the verbal content so as to remain consistent with the non-verbal modes.The field of multimodal translation therefore inherently encompasses many special or marginal cases of translation.
There is no simple answer to the question whether multimodal translation can be considered a distinct phenomenon.But it is undeniable that the definition of translation must be particularly flexible, and the borders around this cluster concept must be especially hazy, in order to accommodate all the activities that can be termed "multimodal translation".
Another relevant question is whether this flexible perspective on translation leads to a view of research on multimodal translation as a distinct subfield of Translation Studies, something we might call Multimodal Translation Studies.Although we do not attempt to answer this question conclusively, the breadth of research conducted in the area in recent years does suggest that research into multimodal translation is expanding and diversifying.Furthermore, while many current approaches are indebted to Translation Studies and stem from a long research tradition in the discipline, some developments towards defining the object of research may suggest departures unique to multimodal research contexts.Perhaps it would be most realistic to see a possible subdiscipline of Multimodal Translation Studies as a way of challenging any narrow definitions of translation and of broadening the scope of Translation Studies.In this introduction, we use the term "multimodal translation studies", a translation-specific derivation from the term "multimodal studies", to foreground multimodally oriented translation research as a research orientation with a specific objective of addressing the presence and interplay of different modes in translational contexts.It remains to be seen whether it develops into a genuine, prominent subfield of Translation Studies.In order for that to happen, multimodal translation studies should demonstrate its distinct identity by finding ways of addressing questions that have been less frequently covered in Translation Studies, questions such as the interplay of verbal and non-verbal modes in the construction of meaning, and thus introduce new research questions, new methods and new interdisciplinary connections into Translation Studies.
The nature of multimodal translation as a cluster concept will become apparent only if, as Tymoczko (2007, pp. 86-87) recommends, we observe and describe many examples of it and observe their similarities and differences.Therefore, while much interesting research has already been produced in recent years, the cluster concept nature of multimodal translation certainly warrants yet further continuing research.Furthermore, it confirms the need to test new methods that can uncover different aspects of multimodal translation and form a deeper understanding of its nature.Therefore, in the following section, we continue to explore the nature of multimodal translation studies by discussing some of the methodological challenges related to the field.

Interdisciplinarity in multimodal translation studies
Even though we tend to share an understanding of the nature of scholarly investigation, we do not always understand the terms method, methodology or even research approach in the same way.The terms used to distinguish between the different aspects and application phases of a method are sometimes ambiguous.What some refer to as a method or a methodology is termed a methodological design or a scientific approach by others.
Academic knowledge offers theories to explain the phenomena that surround us.We require information that is systematically acquired to describe existing patterns and discover new ones.Therefore, in the first place, it is of vital importance to adopt a methodology or a research approach that allows the researcher to gather representative data.In the second place, it is essential to employ a method that describes and explains the data and the relationships that exist between parts of the data.These explanations give rise to models and theories that describe the phenomenon under study.In this sense, a method is a collection of operations and processes which are carried out so that the required data can be systematically analysed and so that reliable and verifiable arguments can then be drawn from it.Saldanha and O'Brien (2013) describe methodology as "a general approach to studying a phenomenon" whereas method "is a specific research technique" (p.13).The authors describe tools, in turn, as technological instruments designed for different methods (2013), such as "a voice recording device, screen recording software and a keystroke logging tool" (p.13).In this sense, eye tracking, for instance, is a specific method whereas an eyetracker is a tool: the software used during the application of the method.
It is useful to keep the definitions of these terms in mind when considering the state of research in Translation Studies.The field is heterogeneous in the way research is conducted, and because Translation Studies is inherently interdisciplinary in its methodology, descriptions of the tools and methods used can vary greatly.Translation Studies itself has adopted a number of approaches from other disciplines: from the traditionally prevalent Literary Criticism and Linguistics to more recently established links to disciplines such as Psychology and Sociology (Saldanha & O'Brien, 2013, pp. 1-2).Therefore, as Saldanha and O'Brien (2013) point out, Translation Studies does not come across as a discipline with a "single coherent methodology" (p. 2), even though many of the methods borrowed from other disciplines have been modified to fit the special circumstances of Translation Studies and many tools are used in a specifically tailored way.This interdisciplinarity and the constant modification of tools and methods explain why methodological discussion plays an important role in the discipline.If there is no overarching, established methodology, it is important to explain and critically evaluate the breadth of options for interdisciplinary cross-overs and their implementations specific to Translation Studies.Furthermore, it is always possible to explore new sources of methodological inspiration and potentially useful tools.A deeper understanding of the available research methods and their applications can facilitate a more disciplined approach to research design and encourage a more nuanced debate on the nature of translation as an activity and Translation Studies as a discipline.
As is the case with Translation Studies generally, it is impossible to delineate a single, coherent methodological framework for multimodal translation studies.In other words, both Translation Studies and multimodal translation studies are also cluster concepts in a methodological sense.Different types of multimodal translation may form interdisciplinary connections in different directions-audiovisual translation with film studies, technical translation with technical communication and videogames with humancomputer interaction.
The need to perceive multimodal translation as a methodological cluster concept is reinforced by the criticism directed by some scholars at prominent trends in multimodal translation studies-namely, that the research methods adopted for multimodal contexts have tended to be similar to those used with verbal texts-which has led to attention being focused on language at the expense of other modes (Kaindl, 2013, p. 257).In this regard, several scholars have called for more varied methods that take account of both the nonverbal and the verbal aspects of multimodal messages (Kaindl, 2013, p. 266;Kokkola & Ketola, 2015, pp. 226-227;Pérez-González, 2014a, p. 185).
Following these recommendations and developing a broader methodological toolkit would mean, first, placing emphasis on what is unique and specifically challenging about multimodal translation studies and, secondly, shifting away from the verbal focus of Translation Studies.In order to accomplish this more varied methodological framework, it would be helpful to look for models from other disciplines and multidisciplinary innovations that have not previously been harnessed commonly in translation research.Embracing a combination of new interdisciplinary connections and a solid understanding of the nature of translation as a target of research could help to move research forward in innovative and ambitious ways.
As we see it, the core methodological challenge facing multimodal translation research is integrating the analysis of non-verbal modes with that of verbal information.This challenge can take on different forms depending on the nature of the research being carried out.In sections 3.2.1 to 3.2.3,we introduce three areas of multimodal translation studies that, in our view, merit particular reflective attention from a methodological point of view.The first of these is the application of the metafunctional analysis to translational enquiry.Metafunctional analysis offers a ready-made tool for analysing non-verbal modes.Yet, as we discuss in the following section, the integration of analyses of verbal and non-verbal information poses particular challenges in this type of research.
The second area of research, described in section 3.2.2, is corpus-based methods in multimodal translation studies, particularly in the analysis of audiovisual texts.As advocated by Baños, Bruti and Zanotti (2013, p. 488), in order to deal with the complex semiotic fabric of audiovisual texts, corpus-based audiovisual translation research should integrate multimodal analysis with a corpus linguistics methodology.But the practical application of this idea encounters a number of conceptual and technological challenges.
The third area of research we wish to draw attention to is reception research.The reception of translated multimodal texts is of particular interest, since the reader's or viewer's interpretation reflects the way in which the translation interacts with other modes.However, as we write in section 3.2.3,our view is that multimodality has to date not been considered sufficiently or explicitly in reception research.

Systemic functional analyses in multimodal translation studies
The systemic functional approach to multimodality is linguistically inspired: as its starting point, it adopts the three metafunctions Halliday (1978) proposed for verbal language-the ideational, the interpersonal and the textual-and extends them to apply to all modes.In this view of multimodality, "common semiotic principles operate in and across different modes" (Kress & van Leeuwen, 2001, p. 2).The approach, therefore, treats all modes as identical in their meaning-making strategies.In multimodal studies, the metafunctional analysis is particularly popular in the analysis of illustrated texts, in which such an analysis is carried out on both the verbal and the visual parts of the text, the words and the images.The metafunctional analysis of images is founded on the social-semiotic account of how meaning is realized visually that was introduced by Kress and van Leeuwen (1996).
In the metafunctional analysis of verbal language, the ideational metafunction refers to the way in which external reality is represented in the text: the people, objects and places-referred to as "the participants"-as well as actions, circumstances and so on being discussed (e.g., Halliday & Matthiessen 2004/2013, p. 29).Similarly, image analysis at the ideational level includes identifying both the participants in the image and the circumstances between the participants (Kress & van Leeuwen, 1996, p. 42).The interpersonal metafunction of verbal language refers to the relationship that exists between the author of the text and the reader of the text (Halliday & Matthiessen 2004/2013, pp. 29-30).The author may address the reader in various ways-for instance, by instructing, negotiating, persuading or giving orders.Image analysis at the interpersonal level refers to identifying the presumed relationships between the creator of the image, its viewer and the participants in the image.This metafunction is said to be realised at three levels: eye contact, size of frame (close shot, medium shot, long shot) and perspective (Kress & van Leeuwen, 1996, pp. 42-43, pp. 116-129).The textual metafunction of verbal language refers to the property of a text that links individual words into coherent clauses.It involves aspects such as cohesion, information structure and thematic structure (Halliday & Matthiessen 2004/2013, p. 30).In image analysis, this metafunction is referred to as compositional and it refers to identifying the composition of the image, the colour(s) and the relative sizes of the participants and their positioning (up, down, left, right, centre) in relation to the other participants (Kress & van Leeuwen, 1996, p. 43, pp. 177-179).
Systemic functional analysis has frequently been employed in multimodal translation studies.Examples include analyses of subtitles (e.g., Espindola, 2012;Mubenga, 2009Mubenga, , 2010)), translations of advertisements (e.g., Feng & Espindola, 2013;Millán-Varela, 2004), children's picture books (Van Meerbergen, 2010) and magazines (Chueasuai, 2013).The prominence of the systemic functional approach to multimodality in translation research is also emphasised in Luis Pérez-González's article Multimodality in Translation and Interpreting Studies, written as a general introduction to the topic for A Companion to Translation Studies (2014b).In the article, the author introduces the arrival and the promise of the systemic functional approach, referring to it throughout the article as "multimodal theory", which could even be interpreted as suggesting that multimodality is synonymous with the systemic functional approach.
Systemic functional analysis can be complimented for offering systematic procedures for analysing multimodal material (Martinec & Salway, 2005, p. 341).A possible limitation of the method, however, is that, based on existing studies, the metafunctional analyses of different modes, performed separately, cannot always be combined in a productive manner (for detailed reviews of metafunctional analyses in multimodal translation studies, see, for example, Ketola, 2018).If the analyses cannot be combined, what we are discussing is a procedure for producing separate, monomodal analyses instead of a multimodal analysis of the interaction of the modes.Another concern that has been reported about the analyses of multimodal texts is the problem of "infinite detail": metafunctional analyses often produce painstakingly detailed descriptions of the components of multimodal texts but they rarely formulate generalizations about multimodal meaning construction (Forceville, 2007(Forceville, , p. 1236)).

Corpus-based multimodal translation research
When we refer to the quantity and representativeness of the data being of paramount importance in data collection, we are referring to corpus-based Translation Studies.Generally speaking, corpus analysis is hardly new in Translation Studies, since it has a long and fruitful history as a research method (Saldanha & O'Brien, 2013).This is reflected in the considerable number of papers that present studies on how to solve different translation problems or that confirm different hypotheses regarding translation as either a process or a product.Corpus analysis has been successfully applied to the acquisition of translation competence (Quinci, 2015;Rodríguez Inés, 2013), to translation universals (Halverson, 2007), to the translation of style, and to the description of text genres and even to specialised texts (Jiménez Crespo, 2008;López Rodríguez, 2007.However, all of these studies have targeted monomodal texts or only one mode of a multimodal text.When corpus analysis is applied to multimodal texts, however, the scenario undergoes a complete transformation. Because of the different modes in multimodal texts, it is still not possible to align an ST automatically with a target text (TT), or even to do so with the different modes in the text.Despite the rapid progress being made with technology, computer scientists have still not been able to create a program capable of achieving this feat.Indeed, after a somewhat arduous adaptation process, certain software applications, such as Atlas.ti,AntConc and MQDA, are able only to relate visual and audio units semi-automatically with the TTs that result from the translation of these modes.In the corpus-based analysis of multimodal translation, however, these applications are yet unable to facilitate the segmentation of the text into units of analysis, nor are they able to facilitate the subsequent semantic annotation of these units.These limitations constitute a bottleneck in this innovative type of corpus research.
In contrast, traditional corpus studies have managed to meet these challenges successfully (Rodríguez-Inés, 2017, p. 268).This is because the segmentation of monomodal textual corpora is based either on different levels of linguistic structure (morpheme, lexeme, syntagm, etc.) or on structural aspects of a text (paragraphs, lines, etc.).Unfortunately, this is not the case with corpus-based studies of multimedia or multimodal texts, in which segmentation is considerably more complex.
What has still not been studied sufficiently in corpus-based studies are the grammars of certain semiotic modes and the texts in which they operate-for instance, dynamic or static images, gestures and sound types.Accordingly, when the researcher's objective is to divide a multimodal ST into visual, sound and/or tactile units, this segmentation must be based on a detailed analysis of its components, namely, the semiotic codes it is composed of.For example, in film studies, the European (Casetti & di Chio, 1990), and American schools (Bordwell & Thompson, 1986) have proposed interesting theories that are currently being applied to the segmentation of audiovisual corpora.In theories about the interpretation of art and artistic education there is a long tradition of segmenting and decomposing static two-and three-dimensional images into codes and smaller components in order to be able to interpret their artistic composition and function (Dondis, 1973).However, these advances in different disciplines that focus on the grammatical aspects of their own types of text have still not been recurrently or systematically applied in corpus-based studies of multimodal translated texts, although some attempts have been made to do so (Pérez Payá, 2010).
Another feature of corpus studies is the complex tagging system that must be applied to multimodal texts when studying a certain phenomenon.This way of attaching information to text segments or to the object of analysis is extremely productive in monomodal corpora, because it is mainly automated.But when a corpus is multimodal, text tagging is rarely performed because it is extraordinarily difficult to do.There have been attempts to tag gestures and functions in other languages, however.A case in point is the tagging done by the ELAN POS tagger for British sign language (i.e. the Digging to Signs Project/BSL Corpus Project).Even corpus-based interpreting studies do not use any type of tagging for the supra-segmental elements of discourse.In this sense, most multimodal corpora are of the type described in Abuczki and Ghazaleh (2013): Similarly to corpus-driven approaches that study lexical bundles (multi-word sequences) (Biber 2010: 170-172), some of the multimodal corpus researches are inspired by the notion of semiotic bundles (Arzarello et al. 2005) where modeling language production includes the manipulation of resources as well as gesture and talk.Some functional annotation schemes (Allwood et al. 2007) try to code the meaning relations between gestures and co-occurring speech in a systematic way, and label communicative events according to the alignment of speech and gesture.(p.89) In some multimodal texts, such as audiovisual texts or those in which static images are part of the multimodality, no systematic tagging of the characteristics that define different text genres is being done yet.
Finally, because of the problems and limitations outlined above, multimodal text tagging is generally performed manually or semi-automatically.Needless to say, this process is slow and complicated.It also requires the tagging set to be professionally and conceptually justified, the selection of which presumably depends on the researcher's objective.Besides, these tagging sets should be based on theories that have already explained the issue under analysis (Göpferich, 2010).Furthermore, the tagging process must be subject to a strictly regulated agreement between those who perform the tagging (an intertagger agreement).
All of this evidently requires consensus between different research initiatives so that a coherent tagging system can be adapted specifically for Translation Studies.Only in this way will analyses be more productive.During the tagging process, for instance, taggers may consistently disagree on certain exemplars or types of exemplar.Disagreement may be considered to be an indication that the theoretical definition of an instance lacks clarity and accuracy and therefore requires redefinition (or even further theorising) (Hovy & Lavid, 2010, p. 30).
Still, there are critical voices which claim that corpus studies decontextualise the object of study.Some academics claim that corpus studies do not offer a real bottom-up approach; moreover, the lack of a sufficiently large or representative corpus can also invalidate results.According to Evison (2010, p. 132), combining qualitative and quantitative data is particularly suitable for small domain-specific corpora.Other scholars affirm that a smaller sample permits a more in-depth interpretation of the results (Soler Gallego, 2013, p. 192); it can also permit a bottom-up analysis based on corpus data that complements the top-down analysis in which the starting point is a wider conceptual vision of the research question.
As discussed here, multimodally oriented corpus-based translation research still has some open questions related to the alignment of modes, the segmentation of units of analysis and the need for a tagging system that takes account of the different modes.Nevertheless, as in general Translation Studies, the corpus-based approach is one of the most promising methods in multimodal translation research.

Reception research and experimental research methods
While many areas of Translation Studies are actively concerned with the target audience, audience reception is particularly interesting in the case of multimodal translated texts.In the case of multimodal texts, because the message is being delivered via an interplay of modes, it is not self-evident how audiences will process these messages and which translation solutions will serve them best.To deal with these imponderables, therefore, an active strand of experimental reception research has emerged in recent years.Reception research has been particularly prolific in audiovisual translation and accessibility, but there are also some instances of reception studies to be considered in, for example, game localisation (e.g., Mangiron, 2016Mangiron, , 2018;;O'Hagan, 2009) and the translation of tourist brochures (e.g., Soto Almela, 2013).In what follows, most of the studies referenced relate to audiovisual materials.Naturally, this focus does not cover the full scope of multimodal translations, but audiovisual translation at least provides a useful example of how reception research has been conducted and the way in which the notion of multimodality is present in that research.However, we also want to call for additional research on other areas of multimodal translation in order to gain a more comprehensive understanding of reception in multimodal contexts.Studies in which audiences' responses to the interplay of modes are addressed would be especially welcome as a complement to existing research.
Reception research is not a single research method or tool.Instead, it encompasses a number of approaches to investigating how audiences read, interpret and understand translated multimodal messages.In the context of audiovisual translation, Orero et al. (2018, p. 113) distinguish between online and offline measures: offline measures refer to methods of data gathering where research participants report on their viewing experience afterwards in, for example, questionnaires or interviews; online measures refer to methods which gather data during the viewing, such as eye tracking, EEG, galvanic skin response and heart rate (for a critical review of online and offline measures, see Kruger & Doherty, 2018).Online measures therefore rely largely on technological tools that allow researchers to chart participants' physiological and cognitive responses, whereas offline measures rely on participants' self-reporting.In other words, online measures tend to collect objective data, whereas offline measures use test participants' own statements or assessments, which can be more subjective.In addition to the frequently used questionnaires and interviews, offline measures also include, for example, focus group research (e.g., Caniato, 2014;Tuominen, 2012), whereas online measures also cover approaches less dependent on technology, such as observation (e.g., Fuentes Luque, 2003;O'Hagan, 2009).
Quite often, reception studies combine more than one method in order to triangulate the results and to construct a more comprehensive picture of reception.Nevertheless, many studies are based on an individual reception context and a limited group of participants.Therefore, a more extensive replication of studies employing the same methodology in different contexts is needed if we are genuinely to understand the nature of reception across different audiences (Orero et al. 2018, p. 107;Tuominen, 2018, p. 85).
It would be beyond the scope of this introduction to discuss all the methods used in reception research.For one thing, reception can mean so many things-from the physiological process of reading to the broad sociocultural consequences of translated multimodal messages-that a multiplicity of methods is needed.Secondly, this means that the most prominent research methods alone are not sufficient to explain reception, but individual pilot studies are able to offer a valuable first step towards new ways of understanding aspects of reception.An example is O'Hagan's ( 2009) study, where she observes a single player playing a localized videogame.Her study demonstrates how observation can reveal, for example, a player's emotional relationship with a game and thus open an avenue of enquiry that would be difficult to reveal with methods such as questionnaires or eye tracking.Furthermore, these kinds of innovative methods can lead to new research questions, which can then be investigated further in additional studies employing either the same or different methods.Indeed, many such innovative methods are constantly being introduced in reception research, where the opportunities for interdisciplinary cross-overs are vast.Such cross-overs between disciplines draw as they may from fields such as Cognitive Psychology and Neuroscience, on the one hand, and Media Studies, Pragmatics and Ethnography, on the other.
Of the offline measures, different kinds of questionnaires are the most widely used means of collecting data on reception.In addition, interviews are occasionally used for similar purposes.Questionnaires and interviews can be used to gauge audiences' opinions on and interpretations of different aspects of the translated multimodal message, such as their reactions to humour (e.g., Chiaro, 2007), their understanding of culture-specific references (e.g., Antonini, 2007;Desilla, 2014) or their general attitudes towards audiovisual translations and access to translated programmes (e.g., Alves Veiga, 2006;Widler, 2004).Moreover, questionnaires can be used to collect psychometric data on "audience immersion, enjoyment and cognitive load" (Kruger & Doherty, 2018, p. 93;e.g., Bairstow, 2011).These examples demonstrate both that the use of questionnaires is varied and that they can produce both qualitative and quantitative data, either predominantly subjective views or attitudes or more objective measurements of understanding or cognitive processing (Kruger & Doherty, 2018, p. 93).
The analytical methods associated with questionnaire data are consequently also varied.Typically, questionnaire data contain at least an element of quantitativeness, but open-ended questions also produce qualitative data that provides an opportunity for a more contextualised discussion and allows respondents to choose their own words to represent the breadth and variety of experiences related to reception.Generally speaking, offline research presents an opportunity to collect audiences' experiences and reactions, but both questionnaire or interview design and the selection of analytical methods need to be approached in a systematic and transparent way if they are to produce findings that are ecologically valid and which faithfully represent the respondents' views.
It is not self-evident that all offline measures inherently explore the reception of multimodality, even when the material is multimodal.Many studies focus on the reception of the verbal aspect and in such studies multimodality is more of a background factor than an element in reception.However, some studies do explore the roles that different modes play in reception and in constructing an overall understanding of the message.For example, a study by Bairstow (2011) tested viewers' comprehension of a subtitled programme and explored whether the subtitles might distract viewers' attention away from other modes.In similar fashion, Perego et al. (2016, p. 213) tested viewers' cognitive processing by asking questions about both the visual and the verbal content of the subtitled material, and so the study did address multimodality to some extent.Nevertheless, it would be worthwhile to investigate further how the nature of multimodality influences reception and, furthermore, to design offline approaches that are able to account for multimodality's being a crucial factor in understanding translated messages.
In the last decade or so, experimental online measures have gained prominence over offline methods, primarily as a way of collecting objective and extensive data on reception.Eye tracking, in particular, has become a major method in reception research, predominantly on subtitled materials.In addition, some recent studies have been using EEG; but other possible online methods are still in the process of making their way into the field (Kruger & Doherty, 2018, p. 97;Orero et al., 2018, p. 115).Eye tracking has been used to investigate a great variety of topics, from the fundamental question of whether subtitles are read at all (e.g., d'Ydewalle et al., 1991) to topics such as reading time (Szarkowska & Gerber-Morón, 2018), reactions to errors and poor synchronisation (Lång et al., 2013), the processing of subtitle segmentation (Perego et al., 2010) and the effects of different translation strategies (Ghia, 2012).The findings are often triangulated with other types of data, such as questionnaires (see Kruger & Doherty, 2018, pp. 99-101 for a comprehensive list of eye-tracking studies and their use of triangulation).
The variety of existing studies demonstrates the multiple uses of eye tracking.As Kruger and Doherty (2018, p. 97) point out, eye tracking "has enabled researchers to directly observe viewers' visual attention and make inferences about underlying cognitive processes".Furthermore, Orero et al. (2018, p. 114) suggest that eye tracking can be used to "quantify the attention to and attention distribution between various parts of the screen, as well as to gain an understanding of the nature of the processing".In other words, eye tracking can be used to explore how audiences pay attention to the visual and verbal information present on screen.Eye tracking therefore has significant potential as a tool for studying multimodality.
Nevertheless, to date, multimodality and the interplay of modes have not been a prominent consideration in online reception research.Ramos Caro (2018, p. 99) emphasises this when she indicates that the empirical study of the reception of translated multimodal texts is not consolidated.She suggests that this is mainly due to the difficulty of applying different methodologies (Brems & Ramos-Pinto, 2013;Cabezas-Cáceres, 2013;Chiaro & Antonini, 2005;Chmiel & Mazur, 2012).The issues range from the location of subjects, the selection of stimuli and the manipulation of variables to the difficulty involved in analysis and the fact that the interpretation of results is a highly complex process.This reveals a gap in the research that deserves to be filled.
Existing studies, especially those using eye tracking, have been able to investigate how attention is focused on various visual elements, but much remains to be said about multimodality and the interplay of all the different modes (Kruger & Doherty, 2018, p. 103).Some online methods may be able to chart the occurrence of multimodal interactions in reception, whereas offline methods may aim to account for the significance audience members attach to different modes and their interplay.For these reasons, existing methods and research questions could be adapted and new approaches designed to explore the reception of multimodal translations in ways that include more consideration for multimodality and extend the research to a wider range of genres and types of multimodality.
The future of reception research with multimodal materials appears set to instigate even more ambitious studies that often benefit from the increased availability of technological means to measure physiological responses.This means that the development of new methods and interdisciplinary connections is likely to continue and also to take reception research in new directions.Reception research is also becoming more rigorous and transparent, with calls from many scholars to pay attention to the selection of appropriate methods, to the accurate reporting of the methods used (see, for example, Kruger & Doherty, 2018, p. 103;Orero et al., 2018, p. 116) and to the replicability and scope of the studies.However, it is less clear how much attention will be paid, in addition to the online approaches, to the more qualitative, individual and contextual side of reception, such as the emotional aspects highlighted in the studies by O'Hagan (2009) and Ramos Caro (2018).This element of reception is also crucial, because it can put in context some of the findings of the more quantitative studies and explain the significance of multimodal messages to the audience members.

New perspectives to multimodality in translation
In this section, we introduce the articles included in this special issue and the methodological approaches that they represent.The articles are divided into four thematic groups: the first includes those that explore new ways of applying well-established methodological approaches to dealing with multimodal materials; the second introduces some newer, more tentative approaches; the third contains studies that focus on the process of multimodal translation; and the fourth gives an airing to some methods being developed that enable the researchers to study how translated multimodal texts are understood, interpreted and perceived.

Re-imagining multimodal research traditions
This issue contains three articles which set out to develop further the methodological apparatuses that multimodal translation studies often borrows from interdisciplinary multimodal studies.One article seeks to find a novel application for Baldry and Thibault's (2006) multimodal transcription model and the other two aim to develop further the systemic functional analysis introduced in section 3.2.1:they combine it with another analysis method, the result being novel ways of elaborating on the insights that are produced by means of systemic functional procedures.
Nina Reviers' article discusses AD and multimodal cohesion-the cohesive links between the modes of an ST-with a particular emphasis on the role of sound.The author develops a theoretical and methodological framework for this purpose, combining principles of multimodal transcription with concepts regarding the semiotics of sound.And she offers a sample analysis of a selection of audio-described texts taken from a corpus of Dutch audio-described films.The analysis aims to identify how sound and AD work together towards (re)creating cohesion in AD and which lexico-grammatical features can be used to support this cohesion.
The article by Hanem El Farahaty discusses the translation of webcomics and GIFs, reflecting on the challenges of translating them from Arabic into English in particular.The article proposes a novel theoretical framework for analysing comics and GIFs.In doing so, it builds on Serafini's (2010) research on perception, structure and ideology, Kress and van Leeuwen's (2006) discussion on metafunctions of images and the elements of visual design, as well as Genette and Maclean's (1991) discussion on paratext.The new framework aims to offer a detailed tool for image analysis that could be used by translators to analyse how meaning is created and shaped in webcomics and GIFs.
Tzu-yi Elaine Lee's article shows that the graphic novel and its translation have become objects of enormous interest in the publishing world.Although the graphic novel is a solidly established text genre, it is also extremely dynamic.Perhaps for this reason, it continues to be of great academic interest to scholars.Based on three different translations of Louisa May Alcott's Little Women, Lee explores the difference between translation and adaptation, depending on the function that the translation will perform in the target language culture.The article therefore provides a theoretical framework that is applied to an observational methodological approach.Although the starting point of this research lies in intertextuality and double meaning, the practical part of Lee's analysis is based on postulates of visual grammar by Kress and van Leeuwen.The author uses different examples to interpret the adapted images and/or the relationship between the translated text and the adapted image with a view to relating the adapted image to the transmission of certain ideologies through different translations.
As these three authors demonstrate, applying the Hallidayan dimensions of verbal language to the examination of other sign systems leads to a finely tuned analytical tool that has the potential to produce great amounts of detailed description of the way in which meaning is created in a particular multimodal product.Furthermore, as all three authors illustrate, such analysis may benefit greatly from the inclusion of other theoretical concepts to structure the analysis.

New approaches to multimodality in context
The second part of this special issue presents analytical approaches that have not yet been widely adopted in the research on multimodal translations.The first is Hannah Silvester's article, which proposes an integrated, holistic approach to analysing subtitled films.Her analytical model incorporates the macro-level cultural context of film releases together with the micro-context of the film itself within the overall background of the subtitling situation and the technical requirements imposed on subtitles.The analytical model therefore provides a way of exploring the manner in which the macrocontextual background manifests itself in the film and influences its translation.While it is familiar from other fields of Translation Studies, this approach is less established in research on subtitled materials.It is presented as being particularly useful for films with a political message that stems from the macro-level background.Silvester's proposed method therefore enables the researcher to pull together the various strands of meaning-making present in a subtitled film and to conduct an analysis that takes account of the multimodal, multidimensional and contextually determined nature of translated films.
Another novel approach is proposed by Min-Hsiu Liao, whose article discusses translated texts that appear in a museum context.Using methods such as geosemiotics, she explores the ways in which museum texts interact with museum objects and their general surroundings.What is particularly novel in Liao's methodological proposal is her discussion of the manner in which translations construct meaning within a threedimensional space: multimodality takes place not only on page or screen, but also in authentic surroundings such as museums, where translated texts appear alongside the objects they describe and in spaces they define.Consequently, specific methods are needed to analyse the entire space in which translations are being presented and in which audiences encounter them.
Laura Mejías Climent proposes a new approach to the analysis of videogames.In these complex audiovisual texts, multimodality is enriched by the interactive component of the text.This means that players contribute meaning-making elements to a videogame through their own experience and interaction with the game.Like the other contributors to this volume, Mejias Climent not only redescribes videogames from a multimodal perspective but also addresses the methodological challenge of including a new semiotic mode in multimodality, namely, that of touch or the player's use of the game controlswithout the interactivity of the controller (buttons or joystick) it would be impossible to play the videogame.However, her most innovative contribution is to provide a new typology of dubbing synchronisation, based on game situations, which is quite different from that of conventional dubbing.Since this article provides data and real game examples based on the author's experience, the research methodology can be regarded as both bottom-up and observational.

The multimodal translation process
In addition to approaches that analyse multimodal translated materials, this special issue contains three articles which explore how translators tackle the challenges of multimodality.These articles focus on the translation process and decision-making related to multimodality.The first two focus on corpus research and examine the issues we discussed in section 3.2.2,namely aligning STs and TTs as well as developing tagging systems.The contribution of Catalina Jiménez Hurtado and Silvia Martínez-Martínez focuses on accessible translation, specifically subtitling for the deaf and hard of hearing (SDHH).Their study goes about the multimodal translation process with respect to the translation of both articulated and non-articulated sounds into natural language.The method used is the analysis of corpora applied to multimodality, which the authors enhance with an innovative tagging system.The semantic tagging system that they have designed is composed of two levels which reflect the array of cognitive operations performed throughout the translation process.This tagging system is based on a representative selection of ST translation units and stems both from a bottom-up observational process and from the translation strategies used.Their analysis of the results reveals a set of lexico-semantic and syntactic parameters that are used recurrently and which could initially be regarded as a controlled language of SDHH in In their contribution to this volume, Silvia Soler Gallego and Maria Olalla Luque Colmenero deal with AD as an intersemiotic, that is, multimodal, translation modality that is currently used as an accessibility tool in a growing number of museums around the world.More specifically, the authors offer a detailed exposition of a methodological proposal to carry out product-and context-oriented descriptive studies of subjectivity in audio-descriptive guides for art museums.The methodology proposed combines a corpus-based analysis of audio-descriptive guides with the analysis of data-collected through interviews, questionnaires and various bibliographical resources-related to the context of communication in which the ADs are designed.This methodology is illustrated with examples from a study that investigates this question in a number of museums in Europe and the United States of America.
The authors propose that the theory of construal (Langacker, 1987(Langacker, /2008) and its further development by Croft & Cruse (2004) be applied to the study of visual art AD.Based on this theoretical apparatus, they create a set of semantic categories or codes with which to annotate the corpus of ADs by means of qualitative analysis software (Atlas.ti).One set of codes identifies instances of evaluative language and metaphor in the corpus, while a second set identifies the visual component of the artwork that those instances refer to.
The same software application is used in the proposed methodology to analyse the interviews of museum educators, accessibility coordinators, audio describers and companies involved in the creation of the audio-descriptive guides of the corpus.The joint analysis of the results from the corpus-based and the contextual analyses makes it possible to formulate hypotheses in which one or several contextual factors are advanced as possible causes of the incidence of subjective content in the ADs.
Maija Hirvonen and Liisa Tiittula's article introduces a methodology for examining a collaborative AD process as multimodal social interaction.The proposed methodology is multimodal conversation analysis, which aims to reveal how multimodal communication resources (talk, gaze, gestures and so on) are used in interaction.The case study analysed in the article deals with a real-life collaborative translation task where an AD team writes a script for an AD of a film.The authors discuss the methodological choices that researchers face when examining human interaction in translation and conclude that the methodology opens new avenues for research into the translation process and collaborative translation.
We are sure that the methods employed in these three articles can and will open up new horizons for research on and the analysis of multimodal translation processes, such as the study of the recurrent patterns used in different communicative contexts.As the article by Soler Gallego and Luque Colmenero in this section demonstrates, it is often useful to combine quantitative and qualitative methods to facilitate a more in-depth discussion of the research topic.We will see further examples of this multi-method approach in the articles contained in the next section.There, some of the articles use representative and quantitative methods to obtain data that are subsequently analysed from a qualitative perspective, such as a case study or an evaluation, based on questionnaires or structured interviews.

Interpreting and understanding translated multimodal messages
As we discussed in section 3.2.3,multimodality presents an added dimension of meaningmaking for those who are trying to understand the translated multimodal message.Therefore, the perspective of reception offers important topics for research, as we see in the five articles in this section.The methods used to engage with the topic range from those employed in Cognitive Neuroscience to focus groups; they demonstrate the variety of relevant research questions and methods that relate to multimodality.
Multimodal communication is also central to knowledge transfer in different areas and domains of specialised knowledge, as reflected in the text genres typical of science and technology.In fact, traditional instruments of knowledge access, such as metaphors, are also being studied from a multimodal perspective.The first two articles in this section deal with the accessibility of visual information in the specialised field of medical communication.
The article by Maribel Tercedor and Alicia Casado-Valenzuela examines the role of multimodal communication as a way of accessing medical knowledge.More specifically, they analyse visual images that activate a clearly metaphorical structure.The authors examine how this facilitates the general public's access to medical information.For this purpose, their assumption is that in multimodal text production the selection of suitable images is crucial to knowledge transfer in the sphere of specialised translation.
This study seeks to find answers to two research questions.The first seeks to identify the type of images with a metaphorical structure that are most frequently used in medical texts for the general public and illustrate complex medical concepts.The second research question aims to ascertain whether metaphorical images are preferred to nonmetaphorical ones and also whether the metaphors in these images facilitate users' access to knowledge.
To address the first question, the authors base their study on cognitive theories regarding metaphorical images and demonstrate their usefulness to the translation process by providing a typology of these images often also present in other cultures.Furthermore, the study uses a data-driven approach.This was possible thanks to VariMed, a database of multimodal texts.The results show that metaphor is a key element in the transmission of expert knowledge.
The second research question is addressed experimentally.The authors therefore analyse the real-time knowledge access of the targeted receivers by means of visual metaphorical elements.This step is essential before deciding which images are most suitable in multimodal communication (including multilingual contexts).For this purpose, they administer a double questionnaire to a set of respondents, paying particular attention to its ecological validity.
The topic of the article by Juan Antonio Prieto-Velasco and Vicent Montalt-Resurrecció is the comprehensibility of multimodal patient information guides, which are often the result of the intralingual and intermodal translation of specialised medical texts into a non-specialist informational text.Illustrations are typically used to support the verbal message and make it more comprehensible in these kinds of contexts, but we know little about how visual information may improve comprehensibility.This article addresses that gap in the research.Prieto-Velasco and Montalt-Resurrecció conducted a pilot study in which they used a survey and a focus group to collect both quantitative and qualitative data from actual readers of patient information guides; this they did in order to find out what kinds of illustrations best support patients' understanding of the information provided to them.Their findings suggest that illustrations do indeed play a role in the comprehensibility of these medical texts, and further research is needed to understand this comprehension process better.
Whereas the first two articles in this section employ both multimodal analysis methods and empirical research with users to explore the accessibility of medical texts, the other three articles concentrate more specifically on the processes of reading, interpreting and understanding translated multimodal texts.The first is Olli Philippe Lautenbacher's theoretical proposal for the manner in which multimodal texts are read and interpreted.The article investigates the function of redundancy in multimodal communication and the way the phenomenon of redundancy can be used as a crucial element in multimodal meaning-making.The methodological framework of the article is based on relevance theory and Kintsch's Construction-Integration Model.Lautenbacher proposes a model for a reading process for multimodal texts that is based on the detection of trigger stimuli that function partly through redundancy.This model has implications for the translation process, because any changes in the redundancy structure may also influence reception.
Eliisa Pitkäsalo's article explores the use of focus-group discussions as a datacollection method for examining the multimodal reading experience of comics or, more particularly, graphic novels.The data of the study were collected from readers from two cultural backgrounds: one data-collection session was held in Italy, where comics have a long tradition, and the other in Hungary, where comic-style literature has not traditionally been popular.Pitkäsalo's study indicates that the cultural background of comic readers, including the tradition of reading comics, may have an effect on the way readers understand the visual language of comics, which should also affect the work of comic translators.
The contribution of Antonio Javier Chica Núñez connects three axes around which Translation Studies might revolve in the future.First of all, multimodality and its semiotic implications at ST level may determine the macro-and micro-structural properties of the TT and influence its semantic features.Secondly, the implications of multimodality can be better perceived at what we may call the "translation event", namely the translator's production phase plus the reception phase on the side of the audience.Translators need to embrace a more complex and intertwined semiotic configuration of the ST.There, various communication modes shape a multilayered discourse whose meanings, nuances and interactions need to be fully conveyed into the TT.Finally, Cognitive Studies may constitute an indispensable theoretical and methodological paradigm according to which to conduct real and situated analyses of the multimodal translation process and to account for its intricacies.
In this sense, this article draws on several contributions from Social Neuroscience and Cognitive Neuroscience to focus on the analysis of emotions in AD.Those theoretical models allow Chica to establish a methodological approach based on psychophysics in order to measure the reaction of blind and visually impaired subjects to multimodal emotional stimuli in AD.As a result, his approach may contribute to assessing the validity and effectiveness of translation quantitatively, that is, to assess the capacity of any given AD to communicate emotions to recipients in cognitive terms.This type of approach could be applied equally to other audiovisual translation typologies where multimodality plays a decisive role.
The articles in this final section demonstrate just how many open questions remain about the ways in which translations and translators could best serve their target audiences, especially in situations where those audiences encounter the multi-layered meaning-making of a multimodal translation.It is, of course, impossible to know how individual viewers, readers or listeners interpret a message, but innovative theoretical and experimental approaches are crucial in increasing our understanding and providing new ways of exploring what actually happens when audiences process translations in a multimodal setting.

Epilogue
This collection of articles demonstrates how researchers are heeding the call for rigorous and innovative approaches that take into account all modes of multimodal texts and their translations, as well as the interaction between modes.This includes situations where one mode must take the place of another in translation in order to facilitate accessibility.A consistent focus on multimodality ensures that the verbal elements of a multimodal message are not the only ones analysed and that the special characteristics of multimodality are also taken into account.
Furthermore, one of our main concerns in selecting the articles for this special issue has been their explicit and reflective focus on research methods.In line with this, all the articles contain some critical discussion on questions concerning research methodology.However, articles on a methodological theme can assume multiple formats.Some of the articles are more theoretical in nature, exploring the background and the disciplinary underpinnings of a specific methodological approach, whereas others provide examples of how a certain method can be applied in practice and offer critical reflection on the usefulness and applicability of this method in a particular context.The key element in all of them is the transparent description and discussion of the methods used.
We hope that this combination of longstanding and new methods, theoretical and applied approaches, and product and process orientation will offer a well-rounded overview of the methodological multiplicity that characterises multimodal translation studies today.