Effects of thinking aloud on cognitive effort in translation

Sanjun Sun, Tian Li, Xiaoyan Zhou


This study investigates the effects of thinking aloud on cognitive effort in translation as a function of source-text difficulty level. It does so by considering time on task, duration of different translation phases (i.e., orientation, draft, revision), cognitive effort of processing the source and target texts, and translation quality. Twenty participants took part in an English–Chinese translation experiment, which comprised two matched sessions – translating while thinking aloud and translating silently. Their translation processes were recorded by means of an eye tracker and a key logger. An adapted NASA Task Load Index was employed to elicit their subjective assessments of translation difficulty levels. The quality of their translations was evaluated. The results of the study reveal a number of important effects of thinking aloud on cognitive effort during translation: on translation duration, cognitive effort, the perceived level of difficulty of a translation as measured by NASA-TLX, and on translating easier texts.


Think-aloud protocols; translation; cognitive effort; eye-tracking; key-logging

Full Text:



Angelone, E. (2010). Uncertainty, uncertainty management, and metacognitive problem solving in the translation task. In G. M. Shreve & E. Angelone (Eds.), Translation and cognition (pp. 17–40). John Benjamins. https://doi.org/10.1075/ata.xv.03ang

Block, R. A., Hancock, P. A., & Zakay, D. (2010). How cognitive load affects duration judgments: A meta-analytic review. Acta Psychologica, 134(3), 330–343. https://doi.org/10.1016/j.actpsy.2010.03.006

Bowles, M. A. (2010). The think-aloud controversy in second language research. Routledge. https://doi.org/10.4324/9780203856338

Bowles, M. A. (2018). Introspective verbal reports: Think-alouds and stimulated recall. In A. Phakiti, P. DeCosta, L. Plonsky, & S. Starfield (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 339–357). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-59900-1_16

Bowles, M. A., & Leow, R. P. (2005). Reactivity and type of verbal report in sla research methodology: Expanding the scope of investigation. Studies in Second Language Acquisition, 27(3), 415–440. https://doi.org/10.1017/S0272263105050187

Carl, M., Bangalore, S., & Schaeffer, M. (Eds.). (2016). New directions in empirical translation process research. Springer. https://doi.org/10.1007/978-3-319-20358-4

Carl, M., Dragsted, B., & Jakobsen, A. L. (2011). A taxonomy of human translation styles. Translation Journal, 16(2). https://translationjournal.net/journal/56taxonomy.htm

Carl, M., Schaeffer, M., & Bangalore, S. (2016). The CRITT translation process research database. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New directions in empirical translation process research (pp. 13–54). Springer. https://doi.org/10.1007/978-3-319-20358-4_2

Dragsted, B., & Carl, M. (2013). Towards a classification of translation styles based on eye-tracking and key-logging data. Journal of Writing Research, 5(1), 133–158. https://doi.org/10.17239/jowr-2013.05.01.6

Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. https://doi.org/10.1037/0033-295X.87.3.215

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. MIT Press. https://doi.org/10.7551/mitpress/5657.001.0001

Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive?: A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344. https://doi.org/10.1037/a0021663

Godfroid, A., & Spino, L. A. (2015). Reconceptualizing reactivity of think-alouds and eye tracking: Absence of evidence is not evidence of absence. Language Learning, 65(4), 896–928. https://doi.org/10.1111/lang.12136

Halverson, S. L. (2017). Multimethod approaches. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 195–201). Wiley Blackwell. https://doi.org/10.1002/9781119241485.ch11

Hansen, G. (2005). Experience and emotion in empirical translation research with think-aloud and retrospection. Meta, 50(2), 511–521. https://doi.org/10.7202/010997ar

Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139–183). North-Holland. https://doi.org/10.1016/S0166-4115(08)62386-9

Hvelplund, K. T. (2014). Eye tracking and the translation process: Reflections on the analysis and interpretation of eye-tracking data. MonTI. Monografías De Traducción E Interpretación, 2014(1), 201–223. https://doi.org/10.6035/MonTI.2014.ne1.6

Jääskeläinen, R. (2010). Think-aloud protocol. In Y. Gambier & L. van Doorslaer (Eds.), Handbook of translation studies (Vol. 1, pp. 371–373). John Benjamins. https://doi.org/10.1075/hts.1.thi1

Jääskeläinen, R. (2011). Back to basics: Designing a study to determine the validity and reliability of verbal report data on translation processes. In S. O’Brien (Ed.), Cognitive explorations of translation (pp. 15–29). Continuum.

Jääskeläinen, R. (2017). Verbal reports. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 213–231). Wiley-Blackwell. https://doi.org/10.1002/9781119241485.ch12

Jakobsen, A. L. (2003). Effects of think aloud on translation speed, revision, and segmentation. In F. Alves (Ed.), Triangulating translation (pp. 69–95). John Benjamins. https://doi.org/10.1075/btl.45.08jak

Jakobsen, A. L. (2017). Translation process research. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 21–49). Wiley Blackwell. https://doi.org/10.1002/9781119241485.ch2

Kohlberg, L. (1969). Stage and sequence: The cognitive-developmental approach to socialization. In D. Goslin (Ed.), Handbook of socialization theory and research (pp. 347–480). Rand McNally.

Krings, H. P. (2001). Repairing texts: empirical investigations of machine translation post-editing processes (G. Koby, G. Shreve, K. Mischerikow & S. Litzer, Trans.). Kent State University Press.

Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2019). Package lmerTest: Tests in linear mixed effects models. https://cran.r-project.org/web/packages/lmerTest/lmerTest.pdf

Lacruz, I. (2017). Cognitive effort in translation, editing, and post‐editing. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 386–401). Wiley-Blackwell. https://doi.org/10.1002/9781119241485.ch21

Leow, R. P., & Morgan-Short, K. (2004). To think aloud or not to think aloud: The issue of reactivity in SLA research methodology. Studies in Second Language Acquisition, 26(1), 35–57. https://doi.org/10.1017/S0272263104026129

Liu, Y., Zheng, B., & Zhou, H. (2019). Measuring the difficulty of text translation: The combination of text-focused and translator-oriented approaches. Target, 31(1), 125–149. https://doi.org/10.1075/target.18036.zhe

Muñoz Martín, R. (2012). Just a matter of scope: Mental load in translation process research. Translation Spaces, 1(1), 169–188. https://doi.org/10.1075/ts.1.08mun

Nitzke, J. (2019). Problem solving activities in post-editing and translation from scratch. Language Science Press. https://doi.org/10.4324/9780429030376-5

Nitzke, J., & Oster, K. (2016). Comparing translation and post-editing: An annotation schema for activity units. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New directions in empirical translation process research (pp. 293–308). Springer. https://doi.org/10.1007/978-3-319-20358-4_14

O'Donnell, R. D., & Block , F. T. (1986). Workload assessment methodology. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance: Vol. 2. Cognitive Processes and Performance (pp. 42-1–42-49). Wiley.

O’Brien, S. (2009). Eye tracking in translation-process research: Methodological challenges and solutions. In I. M. Mees, F. Alves, & S. Göpferich (Eds.), Methodology, technology and innovation in translation process research (pp. 251–266). Samfundslitteratur.

R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org

Smith, P., Kim, D., Vorobel, O., & King, J. R. (2019). Verbal reports in the reading processes of language learners: A methodological review. Review of Education, 8(1), 37–114. https://doi.org/10.1002/rev3.3170

Sun, S. (2011). Think-aloud-based translation process research: Some methodological considerations. Meta, 56(4), 928–951. https://doi.org/10.7202/1011261ar

Sun, S. (2015). Measuring translation difficulty: Theoretical and methodological considerations. Across Languages and Cultures, 16(1), 29–54. https://doi.org/10.1556/084.2015.16.1.2

Sun, S. (2019). Measuring difficulty in translation and post-editing: A review. In D. Li, V. L. C. Lei, & Y. He (Eds.), Researching cognitive processes of translation (pp. 139–168). Springer Singapore. https://doi.org/10.1007/978-981-13-1984-6_7

Sun, S., & Shreve, G. M. (2014). Measuring translation difficulty: An empirical study. Target, 26(1), 98–127. https://doi.org/10.1075/target.26.1.04sun

Toury, G. (1991). Experimentation in translation studies: Achievements, prospects and some pitfalls. In S. Tirkkonen-Condit (Ed.), Empirical research in translation and intercultural studies (pp. 45–66). Narr.

Tyler, S. W., Hertel, P. T., McCallum, M. C., & Ellis, H. C. (1979). Cognitive effort and memory. Journal of Experimental Psychology: Human Learning and Memory, 5(6), 607–617. https://doi.org/10.1037/0278-7393.5.6.607

Vieira, L. N. (2017). Cognitive effort and different task foci in post-editing of machine translation: A think-aloud study. Across Languages and Cultures, 18(1), 79–105. https://doi.org/10.1556/084.2017.18.1.4

Wierwille, W. W., & Williges, B. H. (1980). An annotated bibliography on operator mental workload assessment (Naval Air Test Center Report No. SY-27R-80). Naval Air Test Center.

Yang, C. (2019). The veridicality of think-aloud protocols and the complementary roles of retrospective verbal reports: A study from EFL writing. The Asia-Pacific Education Researcher, 28(6), 531–541. https://doi.org/10.1007/s40299-019-00453-5