Machine translation output assessment and its impact on reading comprehension

Mirzaeian, V.R.

doi:10.22061/jte.2019.5473.2227

Document Type : Original Research Paper

Author

V.R. Mirzaeian

ELT Department, Alzahra University, Tehran, Iran

https://doi.org/10.22061/jte.2019.5473.2227

Abstract

Background and Objectives: Machine translation is now widely used everywhere; However, its role as a language learning tool has not been confirmed, as there are concerns about its quality. However, if we compare the machine translation output with the output produced ten years ago, we see a significant improvement in its quality, especially in terms of vocabulary and grammar. Machine translation can be defined as: the process by which, using electronic devices, input can be provided from one language and output delivered in another language. When machine translation became available on smartphones, it gained universal acceptance because of its benefits such as free and easy access. In the field of education, many learners use this technology every day for various personal as well as academic purposes. These goals mainly include understanding a text that is not written in the native language or translating different texts from different languages into other languages and delivering it as homework. Machine translation can help learners gain a quick understanding of a text written in a language other than their mother tongue by producing an incomplete version. The aim of this research was to assess the quality of machine translation and its impact on students’ reading comprehension.
Methods: Three types of texts were selected with varying levels of difficulty. These texts were translated once by a human translator and once by machine translation (Google Translator). Finally, six texts were obtained. The output of machine translation was evaluated and analyzed. Postgraduate students who happened to use machine translation more frequently were then randomly divided into six groups, each group reading one of these texts and answering multiple choice comprehension questions at the end of the text. The T-test was performed on the data and it was found that from the three types of texts, the two types of texts, despite having some lexical and grammatical problems, were able to compete with human translation.
Findings: The data showed that the quality of machine translation is improving and has now reached a degree of quality that can be used as a tool in educational environments. Some guidelines were also given on how to use this technology in the classroom.
Conclusion: This study attracts attention of language educators to MT and its use in language teaching. It suggests that language educators should be trained to use this tool to improve language learning among students. Considering that the type of text has a great impact on the quality of machine translation and very good scientific texts and very bad literary texts are machine translated, this point should also be considered in generalizing the results of this research. All three texts translated by Google were able to match the human translated text in terms of comprehension, but the number of unknown sentences in this text was more than the other two texts, which were expected to have a negative effect on students' comprehension, which was not observed. The issue of gender can also be examined to see if there is a relationship between gender and the type of reaction to machine translation or not.

Keywords

20.1001.1.20080441.1399.14.2.13.3

Main Subjects

Electronic learning- virtual

COPYRIGHTS
©2020 The author(s). This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, as long as the original authors and source are cited. No permission is required from the authors or the publishers.

References

[1] Alhaisoni E, Alhaysony M. An investigation of Saudi EFL university students’ attitudes towards the use of Google Translate. International Journal of English Language Education. 2017; 5(1): 72–82.

[2] Lee S. The impact of using machine translation on EFL students’ writing. Computer Assisted Language Learning. 2019.

[3] Mirzaeian V. A Comparative Evaluation of Two Persian MT Systems. Translation Studies. 2007; 5(17): 35-45.

[4] Mirzaeian V. Challenges of Machine Translation in Persian, using Three MT systems. Translation Studies. 2010; 7(28), 73-90.

[5] Mirzaeian V. Improving the translation of idioms by Google Translate. Translation Studies. 2011; 9(35): 15-25.

[6] Nino A. Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advantages and disadvantages. ReCALL. 2009; 21(2): 241-258.

[7] Sukkhwan A, Sripetpun W. Use of Google Translate: A survey of Songkhla Rajabhat University students. In Proceedings of L-SA Workshop and Colloquium: “Speaking” for ASEAN. 2014; 88–104.

[8] Amaral L, Meurers D. On using intelligent computer-assisted language learning in real-life foreign language teaching and learning. ReCALL. 2011; 23(01): 4-24.

[9] Correa M. Leaving the “peer” out of peer-editing: Online translators as a pedagogical tool in the Spanish as a second language classroom. Latin American Journal of Content and Language Integrated Learning. 2014; 7(1): 1-20.

[10] Garcia I., Pena M. Machine translation-assisted language learning: Writing for beginners. Computer Assisted Language Learning. 2011; 24(5): 471–487.

[11] Ali K, Alireza F. The effect of computer-assisted translation on L2 leaners’ mastery of writing. International Journal of Research Studies in Language Learning. 2014; 3: 29–44.

[12] Godwin-Jones R. Contributing, creating, curating: Digital literacies for language learners. Language Learning & Technology. 2015; 19(3): 8–20.

[13] Bernardini S. Discovery learning in the language for translation classroom. Cadernos de Traducao. 2016;36: 14–35.

[14] Wong T, Lee, J. Corpus-based learning of Cantonese for Mandarin speakers. ReCALL. 2016; 28(2): 187–206.

[15] Nation I. Learning vocabulary in another language. Cambridge: Cambridge University Press;2001.

[16] Chen M, Huang S, Chang J, Liou H. Developing a corpus-based paraphrase tool to improve EFL learners’ writing skills. Computer Assisted Language Learning. 2015;28(1): 22-40.

[17] Frodesen J. Linguistic challenges of summary and paraphrase. Paper presented at the 2007 Symposium on Academicdis Language Development. San Francisco, CA. 2007.

[18] Bahri H., Mahadi T. Google Translate as a supplementary tool for learning Malay: A case study at Universiti Sains Malaysia. Advances in Language and Literary Studies. 2016; 7(3): 161-167.

[19] Kliffer M. Post-editing machine translation as an FSL exercise. Porta Linguarum. 2008; 9: 53-67.

[20] Kliffer M. An experiment in MT post-editing by a class of intermediate/advanced French majors. In Proceedings EAMT 10th Annual Conference. 2005; 160-165.

[21] Jin L. Foreign language learners’ use and perception of online dictionaries: A survey study. MERLOT Journal of Online Learning and Teaching. 2013; 9(4): 513-533.

[22] Amores M. A new perspective on peer-editing. Foreign Language Annals. 1997; 30(4): 513-522.

[23] Shei C. Combining translation into the second language and second language learning: An integrated computational approach. [dissertation]. University of Edinburgh, Edinburgh. 2002.

[24] Shei C. Teaching MT through pre-editing: Three case studies. In Proceedings of 6th EAMT Workshop Teaching Machine Translation. 2002b; 89–98.

[25] Josefsson E. Contemporary approaches to translation in the classroom: A study of students’ attitudes and strategies. 2011.

[26] Groves M, Mundt, K. Friend or foe? Google Translate in language for academic purposes. English for Specific Purposes. 2015; 3: 112–121.

[27] Barr D. Embedding technology in translation teaching: Evaluative considerations for courseware integration. Computer Assisted Language Learning. 2013; 26(4): 295–310.

[28] Case M. Machine translation and the disruption of foreign language learning activities. eLearning Papers. 2015; 45: 4–16.

[29] Popovic M. Error classification and analysis for Machine Translation. Quality Assessment. 2018.

[30] Aiken M, Balan S. An analysis of Google Translate accuracy. Translation Journal. 2011; 16(2). Retrieved from.

[31] Belam J. Buying up to falling down: A deductive approach to teaching post-editing. In Proceedings of theWorkshop on Teaching Translation Technologies and Tools. 2003; 1–10.

[32] Clifford J, Merschel L, Munne, J. Surveying the landscape: What is the role of machine translation in language learning? The Acquisition of Second Languages and Innovative Pedagogies. 2013; 10: 108–121.

[33] Doherty S, Kenny D. The design and evaluation of a Statistical Machine Translation syllabus for translation students. The Interpreter and Translator Trainer. 2014; 8(2): 295–315.

[34] Enkin E, Mejias-Bikandi E. Using online translators in the second language classroom: Ideas for advanced-level Spanish. Latin American Journal of Content & Language Integrated Learning. 2016; 9(1): 138–158.

[35] Kumar A. Machine translation in Arabic-speaking ELT classrooms: Applications and implications. International Journal of Social Science and Humanity. 2012; 2(6): 442–445.

[36] Lewis D. Machine translation in a modern language curriculum. Computer Assisted Language Learning.1997; 10(3): 255–271.

[37] Nino A. Recycling MT: A course on foreign language writing via MT post-editing. In Proceedings of 7th Annual CLUK Research Colloquium. 2004; 179–187.

[38] Ruiz N, Federico M. Complexity of spoken versus written language for machine translation. In Proceedings of the Annual Conference of the European Association for Machine Translation. 2014; 173–180.

[39] Williams L. Web-based machine translation as a tool for promoting electronic literacy and language awareness. Foreign Language Annals. 2006; 39(4): 565–578.

[40] Nino A. Evaluating the use of machine translation post-editing in the foreign language class. Computer Assisted Language Learning. 2008; 21(1): 29–49.

Name *

Email Address *

Affiliation *

Comments *

Security Code *

Technology of Education Journal (TEJ)

Machine translation output assessment and its impact on reading comprehension

References

References

Send comment about this article

Volume 14, Issue 2 - Serial Number 54
April 2020
Pages 393-404

Machine translation output assessment and its impact on reading comprehension

References

References

Send comment about this article

Volume 14, Issue 2 - Serial Number 54April 2020Pages 393-404

Volume 14, Issue 2 - Serial Number 54
April 2020
Pages 393-404