Neural machine translation in AVT teaching in China: An in-depth analysis from the readability perspective


  • Qingran Wang China University of Political Science and Law
  • Jun Xu China University of Political Science and Law



neural machine translation, NMT, AVT teaching, MT, machine translation evaluation, readability test, Chinese–English translation


As audiovisual translation (AVT) becomes more complex and diverse, the need for advanced machine learning techniques has been increasing sharply, driving the widespread adoption of neural machine translation (NMT) technology in the field. This study contributes to the literature by evaluating the performance of NMT technology in AVT teaching. Based on readability theory, we constructed an evaluation framework with 12 indicators, built comparable corpora consisting of human and post-edited subtitle translations of corporate videos, and used them to examine the performance of four online NMT systems (Google Translate, Baidu Translate, Bing Translator, and Youdao Translate) in AVT teaching. Our statistical analyses and case studies show that Google Translate outperforms the other three platforms in all the readability tests, and it can enhance the readability of post-edited subtitles at five levels (word, syntax, textbase, situation model, genre and rhetorical). The performance of the other three platforms varies across different tests. Concrete examples are provided to substantiate the statistical analyses. Our study adds value to existing research both by examining the application and performance of NMT in AVT teaching and by suggesting potential directions for the refinement of current NMT systems.


Abdullah, Z., Md Nordin, S., & Abdul Aziz, Y. (2013). Building a unique online corporate identity. Marketing intelligence & planning, 31(5), 451–471.

Alva-Manchego, F., & Shardlow, M. (2022). Towards readability-controlled machine translation of COVID-19 texts. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pp. 287–288.

Banerjee S., & Lavie, A. (2005). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Valuation Measures for MT and/or Summarization, pp. 65–72.

Bellés-Calvera, L., & Quintana, R. C. (2021). Audiovisual translation through NMT and subtitling in the Netflix series ‘cable girls’. In Proceedings of the Translation and Interpreting Technology Online Conference, pp. 142–148.

Bywood, L., Georgakopoulou, P., & Etchegoyhen, T. (2017). Embracing the threat: Machine translation as a solution for subtitling. Perspectives, 25(3), 492–508.

Chatzikoumi, E. (2020). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, 26(2), 137–161.

Ciobanu, M., Dinu, L., & Pepelea, F. (2015). Readability assessment of translated texts. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pp. 97–103.

Courtis, K. (1995). Readability of annual reports: Western versus Asian evidence. Accounting, Auditing & Accountability Journal, 8(2), 4–17.

Graesser, A. C., & McNamara, D. S. (2011). Computational analysis of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398.

Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234.

Giménez, J. & Márquez, L. (2010). Linguistic measures for automatic machine translation evaluation. Machine Translation, 24, 209–240.

Han, Y., & Meng, S. (2022). Machine English translation evaluation system based on BP neural network algorithm. Computational Intelligence & Neuroscience, 2022, 1–10.

Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W., Li, M., Liu, S., Liu, T., Luo, R., Menezes, A., Qin, T., Seide, F., Tan, X., Tian, F., Wu, L., … Zhou, M. (2018). Achieving human parity on automatic Chinese to English news translation. arXiv:1803.05567.

Jia, Y., Carl, M., & Wang, X. (2019). Post-editing neural machine translation versus phrase-based machine translation for English-Chinese. Machine Translation, 33, 9–29.

Jiang, J., & Han, B. (2018). A study of reading text difficulty of CET6, TOEFL and IELTS based on Coh-Metrix. Foreign Languages in China, 3, 86–95.

Jones, D., Gibson, E., Shen, W., Granoien, N., Herzog, M., Reynolds, D., & Weinstein, C. (2005). Measuring human readability of machine generated text: Three case studies in speech recognition and machine translation. In Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 1009–1012.

Leong, K., Michael T., & Leyland, F. (2002). E-comprehension: Evaluating B2B websites using readability formulae. Industrial Marketing Management, 31(2), 125–131.

Loughran, T., & McDonald, B. (2014). Measuring readability in financial disclosures. Journal of Finance, 69(4), 1643–1671.

Matusov, E., Wilken, P., & Georgakopoulou, Y. (2019). Customizing a neural machine translation system for the translation of subtitles in the entertainment domain. In Proceedings of the Fourth Conference on Machine Translation, pp. 82–93.

McNamara, D., Graesser, A., McCarthy, P, & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.

Mobarakeh, M. D., & Sardareh, S. A. (2016). The effect of translation shifts on the level of readability of two Persian translations of novel “1984” by George Orwell. International Journal of Humanities and Cultural Studies, 2016, 1418–1427.

Olive, J., Christianson, C, & McCary, J. (2011). Handbook of natural language processing and machine translation. Springer Science & Business Media.

Östling, R., & Tiedemann, J. (2017). Neural machine translation for low-resource languages. arXiv:1708.05729.

Popovi?, M. (2017). Comparing language related issues for NMT and PBMT between German and English. The Prague Bulletin of Mathematical Linguistics, 108, 209–220.

Revanuru, K., Turlapaty, K., & Rao, S. (2017). Neural machine translation of Indian languages. In Proceedings of the 10th annual ACM India compute conference, pp. 11–20.

Sanchez-Torron, M., & Koehn, P. (2016). Machine translation quality and post-editor productivity. In Proceedings of the Association for Machine Translation in the Americas: MT Researchers' Track, pp. 16–26.

Scott, J. (2015). A re-examination of Fortune 500 homepage design practices. IEEE Transactions on Professional Communication, 58(1), 20–44.

Shi, X., & Shan, X. (2019). A corpus-based study on linguistic and cross-cultural adaptation of Chinese

corporate websites. Foreign Languages in China, 2, 71–80.

Smith, M., & Taffler, R. (1992). Readability and understandability: Different measures of the textual complexity of accounting narrative. Accounting, Auditing & Accountability Journal, 5(4), 84–98.

Song, K., Wang, K., Yu, H., Zhang, Y., Huang, Z., Luo, W., & Zhang, M. (2020). Alignment-enhanced transformer for constraining NMT with pre-specified translations. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8886–8893.

Tardel, A. (2021). Measuring effort in subprocesses of subtitling: The case of post-editing via pivot language. In M. Carl (Ed.), Explorations in empirical translation process research (pp. 81–110). Springer.

van Toledo, C., Schraagen, M., van Dijk, F., Brinkhuis, M., & Spruit, M. (2023). Readability metrics for machine translation in Dutch: Google vs. Azure & IBM. Applied Sciences, 13(7), 1–14.

Wang, L., & Liang, M. (2007). Applying WordSmith tools in studies of second languages. Technology Enhanced Foreign Language, 3, 3–7.

Wang, Q., & Xu, J. (2022). Analysis of the impact of machine translation technology on the language

service industry from the perspective of technological progress. Foreign Languages in China, 1,


Wang, Q., & Xu, J. (2023). Interdisciplinary research on translation teaching and talent training in the new era: Taking business translation teaching as an example. Foreign Languages in China, 4,





How to Cite

Wang, Q., & Xu, J. (2023). Neural machine translation in AVT teaching in China: An in-depth analysis from the readability perspective. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 22.