Neural machine translation in AVT teaching in China: An in-depth analysis from the readability perspective
DOI:
https://doi.org/10.52034/lans-tts.v22i.768Keywords:
neural machine translation, NMT, AVT teaching, MT, machine translation evaluation, readability test, Chinese–English translationAbstract
As audiovisual translation (AVT) becomes more complex and diverse, the need for advanced machine learning techniques has been increasing sharply, driving the widespread adoption of neural machine translation (NMT) technology in the field. This study contributes to the literature by evaluating the performance of NMT technology in AVT teaching. Based on readability theory, we constructed an evaluation framework with 12 indicators, built comparable corpora consisting of human and post-edited subtitle translations of corporate videos, and used them to examine the performance of four online NMT systems (Google Translate, Baidu Translate, Bing Translator, and Youdao Translate) in AVT teaching. Our statistical analyses and case studies show that Google Translate outperforms the other three platforms in all the readability tests, and it can enhance the readability of post-edited subtitles at five levels (word, syntax, textbase, situation model, genre and rhetorical). The performance of the other three platforms varies across different tests. Concrete examples are provided to substantiate the statistical analyses. Our study adds value to existing research both by examining the application and performance of NMT in AVT teaching and by suggesting potential directions for the refinement of current NMT systems.
References
Abdullah, Z., Md Nordin, S., & Abdul Aziz, Y. (2013). Building a unique online corporate identity. Marketing intelligence & planning, 31(5), 451–471. https://doi.org/10.1108/MIP-04-2013-0057
Alva-Manchego, F., & Shardlow, M. (2022). Towards readability-controlled machine translation of COVID-19 texts. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pp. 287–288. https://aclanthology.org/2022.eamt-1.33
Banerjee S., & Lavie, A. (2005). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Valuation Measures for MT and/or Summarization, pp. 65–72. https://aclanthology.org/W05-0909.pdf
Bellés-Calvera, L., & Quintana, R. C. (2021). Audiovisual translation through NMT and subtitling in the Netflix series ‘cable girls’. In Proceedings of the Translation and Interpreting Technology Online Conference, pp. 142–148. https://doi.org/10.26615/978-954-452-071-7_015
Bywood, L., Georgakopoulou, P., & Etchegoyhen, T. (2017). Embracing the threat: Machine translation as a solution for subtitling. Perspectives, 25(3), 492–508. https://doi.org/10.1080/0907676X.2017.1291695
Chatzikoumi, E. (2020). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, 26(2), 137–161. https://doi.org/10.1017/S1351324919000469
Ciobanu, M., Dinu, L., & Pepelea, F. (2015). Readability assessment of translated texts. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pp. 97–103. https://aclanthology.org/R15-1014.pdf
Courtis, K. (1995). Readability of annual reports: Western versus Asian evidence. Accounting, Auditing & Accountability Journal, 8(2), 4–17.
https://doi.org/10.1108/09513579510086795
Graesser, A. C., & McNamara, D. S. (2011). Computational analysis of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398. https://doi.org/10.1111/j.1756-8765.2010.01081.x
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234. https://doi.org/10.3102/0013189X11413260
Giménez, J. & Márquez, L. (2010). Linguistic measures for automatic machine translation evaluation. Machine Translation, 24, 209–240. https://link.springer.com/article/10.1007/s10590-011-9088-7
Han, Y., & Meng, S. (2022). Machine English translation evaluation system based on BP neural network algorithm. Computational Intelligence & Neuroscience, 2022, 1–10. https://doi.org/10.1155/2022/4974579
Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W., Li, M., Liu, S., Liu, T., Luo, R., Menezes, A., Qin, T., Seide, F., Tan, X., Tian, F., Wu, L., … Zhou, M. (2018). Achieving human parity on automatic Chinese to English news translation. arXiv:1803.05567. https://arxiv.org/abs/1803.05567
Jia, Y., Carl, M., & Wang, X. (2019). Post-editing neural machine translation versus phrase-based machine translation for English-Chinese. Machine Translation, 33, 9–29. https://doi.org/10.1007/s10590-019-09229-6
Jiang, J., & Han, B. (2018). A study of reading text difficulty of CET6, TOEFL and IELTS based on Coh-Metrix. Foreign Languages in China, 3, 86–95.
Jones, D., Gibson, E., Shen, W., Granoien, N., Herzog, M., Reynolds, D., & Weinstein, C. (2005). Measuring human readability of machine generated text: Three case studies in speech recognition and machine translation. In Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 1009–1012. https://ieeexplore.ieee.org/document/1416477
Leong, K., Michael T., & Leyland, F. (2002). E-comprehension: Evaluating B2B websites using readability formulae. Industrial Marketing Management, 31(2), 125–131. https://doi.org/10.1016/S0019-8501(01)00184-5
Loughran, T., & McDonald, B. (2014). Measuring readability in financial disclosures. Journal of Finance, 69(4), 1643–1671. https://doi.org/10.1111/jofi.12162
Matusov, E., Wilken, P., & Georgakopoulou, Y. (2019). Customizing a neural machine translation system for the translation of subtitles in the entertainment domain. In Proceedings of the Fourth Conference on Machine Translation, pp. 82–93. https://aclanthology.org/W19-5209.pdf
McNamara, D., Graesser, A., McCarthy, P, & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press. https://doi.org/10.1017/CBO9780511894664
Mobarakeh, M. D., & Sardareh, S. A. (2016). The effect of translation shifts on the level of readability of two Persian translations of novel “1984” by George Orwell. International Journal of Humanities and Cultural Studies, 2016, 1418–1427.
Olive, J., Christianson, C, & McCary, J. (2011). Handbook of natural language processing and machine translation. Springer Science & Business Media. https://doi.org/10.1007/978-1-4419-7713-7
Östling, R., & Tiedemann, J. (2017). Neural machine translation for low-resource languages. arXiv:1708.05729. https://doi.org/10.48550/arXiv.1708.05729
Popovi?, M. (2017). Comparing language related issues for NMT and PBMT between German and English. The Prague Bulletin of Mathematical Linguistics, 108, 209–220. https://doi.org/10.1515/pralin-2017-0021
Revanuru, K., Turlapaty, K., & Rao, S. (2017). Neural machine translation of Indian languages. In Proceedings of the 10th annual ACM India compute conference, pp. 11–20. https://doi.org/10.1145/3140107.3140111
Sanchez-Torron, M., & Koehn, P. (2016). Machine translation quality and post-editor productivity. In Proceedings of the Association for Machine Translation in the Americas: MT Researchers' Track, pp. 16–26. https://aclanthology.org/2016.amta-researchers.2.pdf
Scott, J. (2015). A re-examination of Fortune 500 homepage design practices. IEEE Transactions on Professional Communication, 58(1), 20–44. https://doi.org/10.1109/TPC.2015.2420371
Shi, X., & Shan, X. (2019). A corpus-based study on linguistic and cross-cultural adaptation of Chinese
corporate websites. Foreign Languages in China, 2, 71–80.
Smith, M., & Taffler, R. (1992). Readability and understandability: Different measures of the textual complexity of accounting narrative. Accounting, Auditing & Accountability Journal, 5(4), 84–98. https://doi.org/10.1108/09513579210019549
Song, K., Wang, K., Yu, H., Zhang, Y., Huang, Z., Luo, W., & Zhang, M. (2020). Alignment-enhanced transformer for constraining NMT with pre-specified translations. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8886–8893. https://doi.org/10.1609/aaai.v34i05.6418
Tardel, A. (2021). Measuring effort in subprocesses of subtitling: The case of post-editing via pivot language. In M. Carl (Ed.), Explorations in empirical translation process research (pp. 81–110). Springer. https://doi.org/10.1007/978-3-030-69777-8_4
van Toledo, C., Schraagen, M., van Dijk, F., Brinkhuis, M., & Spruit, M. (2023). Readability metrics for machine translation in Dutch: Google vs. Azure & IBM. Applied Sciences, 13(7), 1–14. https://doi.org/10.3390/app13074444
Wang, L., & Liang, M. (2007). Applying WordSmith tools in studies of second languages. Technology Enhanced Foreign Language, 3, 3–7.
Wang, Q., & Xu, J. (2022). Analysis of the impact of machine translation technology on the language
service industry from the perspective of technological progress. Foreign Languages in China, 1,
–29.
Wang, Q., & Xu, J. (2023). Interdisciplinary research on translation teaching and talent training in the new era: Taking business translation teaching as an example. Foreign Languages in China, 4,
–97.
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Qingran Wang, Jun Xu
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under the CC BY-NC 4.0 Deed that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. The material cannot be used for commercial purposes.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
