Multimodal Meaning Making and Language Learning in Digital Multilingual Environments
DOI:
https://doi.org/10.71435/688502Keywords:
Multimodal Learning , Language Acquisition , Digital Environments , Speaking ProficiencyAbstract
Purpose: This study examines how multimodal meaning-making in digital multilingual environments supports language learning, with particular attention to vocabulary, grammar, speaking development, learner engagement, and challenges encountered during digital learning.
Subjects and Methods: A mixed-methods design was employed involving 120 university students, comprising 60 students in an experimental group and 60 in a control group. Participants were intermediate-level learners of English, Spanish, and Mandarin from diverse linguistic backgrounds. The experimental group engaged with video lessons, subtitles, visual aids, interactive games, digital storytelling, and speaking simulations, whereas the control group received traditional instruction. Quantitative data were obtained through pre-test and post-test assessments, while qualitative evidence was collected through surveys, interviews, focus groups, and classroom observations and analyzed thematically using NVivo.
Results: Descriptive findings showed greater improvements in the experimental group across vocabulary, grammar, and speaking than in the control group. Qualitative findings indicated that audiovisual input, visual support, repeated practice, interactive dialogue, feedback, and low-pressure speaking simulations enhanced contextual understanding and learner confidence. However, platform navigation difficulties, cognitive overload, distraction, and unfamiliar cultural references constrained learning experiences.
Conclusions: Multimodal learning offers a promising approach to multilingual language education when digital resources are systematically structured, pedagogically integrated, and culturally contextualized. The findings support a conditional rather than inherently beneficial view of multimodal learning and highlight the need for stronger experimental and longitudinal research.
Downloads
References
Aktaş, E. (2023). The effect of Web 2.0 based technology applications on speaking skills and speaking anxiety in teaching Turkish as a foreign language: Voki example. Frontiers in Psychology, 14, 1183037. https://doi.org/10.3389/fpsyg.2023.1183037
Bateman, J., C. R. McMillan, J. E. Kress, & G. Van Leeuwen. (2017). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Blair, A., Haneda, M., & Bose, F. N. (2018). Reimagining English‐medium instructional settings as sites of multilingual and multimodal meaning making. Tesol Quarterly, 52(3), 516-539. https://doi.org/10.1002/tesq.449
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Budach, G. (2013). From language choice to mode choice: How artefacts impact on language use and meaning making in a bilingual classroom. Language and Education, 27(4), 329-342. https://doi.org/10.1080/09500782.2013.788188
Castellano-Sanz, M., & Reyes-Torres, A. (2024). Multiliteracies in minority language contexts.: A multimodal learning path to promote learners' meaning-making process and aesthetic experiences. L1-Educational Studies in Language and Literature, 24(3), 1-29. https://doi.org/10.21248/l1esll.2024.24.3.602
Chaisiri, P. (2023). Promoting Thai EFL learners’ willingness to communicate in the virtual classroom through technology-mediated oral tasks. LEARN Journal: Language Education and Acquisition Research Network, 16(1), 97–120.
Chen, Y. C., Fan, K. K., & Fang, K. T. (2021). Effect of flipped teaching on cognitive load level with mobile devices: The case of a graphic design course. Sustainability, 13(13), 7092. https://doi.org/10.3390/su13137092
Chowdhury, M., Dixon, L. Q., Kuo, L.-J., Donaldson, J. P., Eslami, Z., Viruru, R., & Luo, W. (2024). Digital game-based language learning for vocabulary development. Computers and Education Open, 6, 100160. https://doi.org/10.1016/j.caeo.2024.100160
Coyle, D., Hood, P., & Marsh, D. (2010). Content and Language Integrated Learning: Learning, Teaching and Strategy. Inggris: Cambridge University Press.
Gee, J. P. (2007). What Video Games Have to Teach Us About Learning and Literacy. Computers in Entertainment, 1(1), 20. https://doi.org/10.1145/1233039.1233040
Han, Y. (2022). Understanding Multilingual Young Adults and Adolescents' Digital Literacies in the Wilds: Implications for Language and Literacy Classrooms. Issues and Trends in Learning Technologies, 9(1), 27-46.
Hasyim, M., & Arafah, B. (2023). Semiotic multimodality communication in the age of new media. Studies in Media and Communication, 11(1), 96-103. https://doi.org/10.11114/smc.v11i1.5865
Henry, A. (2023). Learner–environment adaptations in multiple language learning: Casing the ideal multilingual self as a system functioning in context. International Journal of Multilingualism, 20(2), 97-114. https://doi.org/10.1080/14790718.2020.1798969
Hidayat, D. N., Lee, J. Y., Mason, J., & Khaerudin, T. (2022). Digital technology supporting English learning among Indonesian university students. Research and Practice in Technology Enhanced Learning, 17, 23. https://doi.org/10.1186/s41039-022-00198-8
Huang, H.-T. D. (2023). Examining the effect of digital storytelling on English speaking proficiency, willingness to communicate, and group cohesion. TESOL Quarterly, 57(1), 242–269. https://doi.org/10.1002/tesq.3147
Jiang, M. Y.-C., Jong, M. S.-Y., Lau, W. W.-F., Chai, C.-S., & Wu, N. (2023). Effects of automatic speech recognition technology on EFL learners’ willingness to communicate and interactional features. Educational Technology & Society, 26(3), 37–52. https://doi.org/10.30191/ETS.202307_26(3).0004
Kern, R. (2006). Perspectives on technology in learning and teaching languages. Tesol Quarterly, 40(1), 183-210. https://doi.org/10.2307/40264516
Kim, Y., & Namkung, Y. (2024). Methodological characteristics in technology-mediated task-based language teaching research: Current practices and future directions. Annual Review of Applied Linguistics, 44, 56–78. https://doi.org/10.1017/S0267190524000096
Klimova, B., Pikhart, M., & Al-Obaydi, L. H. (2024). Exploring the potential of ChatGPT for foreign language education at the university level. Frontiers in Psychology, 15, 1269319. https://doi.org/10.3389/fpsyg.2024.1269319
Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Kress, G., & Van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. Inggris: Arnold Publishers.
Kristiawan, D., Bashar, K., & Pradana, D. A. (2024). Artificial intelligence in English language learning: A systematic review of AI tools, applications, and pedagogical outcomes. The Art of Teaching English as a Foreign Language, 5(2), 207–218. https://doi.org/10.36663/tatefl.v5i2.912
Lai, B., Tan, K. H., He, M., Mohd Said, N. E., & Muslim, N. (2022). The roles of non-textual elements in sustaining ESL and EFL learning: A scoping review. Sustainability, 14(16), 10292. https://doi.org/10.3390/su141610292
Lee, J. S., & Chiu, M. M. (2023). Modeling EFL learners’ willingness to communicate: The roles of face-to-face and digital L2 communication anxiety. Annual Review of Applied Linguistics, 43, 64–87. https://doi.org/10.1017/S0267190523000090
Lee, S. Y., Lo, Y. H. G., & Chin, T. C. (2021). Practicing multiliteracies to enhance EFL learners’ meaning making process and language development: a multimodal Problem-based approach. Computer Assisted Language Learning, 34(1-2), 66-91. https://doi.org/10.1080/09588221.2019.1614959
Ma, Q., & Zhang, Z. (2024). The use of digital storytelling in bilingual/multilingual students’ meaning-making: A systematic literature review. Globalisation, cultural diversity and schooling, 33-73. https://doi.org/10.1007/978-3-031-53219-1_3
Morgana, V. (2024). Fostering English speaking and writing subskills for the Cambridge B2 First through technology-mediated tasks. ReCALL, 36(2), 119–134. https://doi.org/10.1017/S0958344023000277
Mulisa, F. (2022). Sampling techniques involving human subjects: Applications, pitfalls, and suggestions for further studies. International Journal of Academic Research in Education, 8(1), 74-83. https://doi.org/10.17985/ijare.1225214
O'Hara, M., & Pritchard, S. (2013). Multimodal Learning and Language Development. Language and Education, 27(3), 238–251. https://doi.org/10.1080/09500782.2012.713416
Okumuş Dağdeler, K. (2023). A systematic review of mobile-assisted vocabulary learning research. Smart Learning Environments, 10, 19. https://doi.org/10.1186/s40561-023-00235-z
Pikhart, M., Klimova, B., & Al-Obaydi, L. H. (2024). Exploring university students’ preferences and satisfaction in utilizing digital tools for foreign language learning. Frontiers in Education, 9, 1412377. https://doi.org/10.3389/feduc.2024.1412377
Polakova, P., & Klimova, B. (2022). Vocabulary mobile learning application in blended English language learning. Frontiers in Psychology, 13, 869055. https://doi.org/10.3389/fpsyg.2022.869055
Pratiwi, D. I., Fitriati, S. W., Yuliasri, I., & Waluyo, B. (2024). Flipped classroom with gamified technology and paper-based method for teaching vocabulary. Asian-Pacific Journal of Second and Foreign Language Education, 9, 1. https://doi.org/10.1186/s40862-023-00222-4
Pun, J. K., & Cheung, K. K. C. (2023). Meaning making in collaborative practical work: A case study of multimodal challenges in a Year 10 chemistry classroom. Research in Science & Technological Education, 41(1), 271-288. https://doi.org/10.1080/02635143.2021.1895101
Rodríguez-Arancón, P. (2023). Developing L2 intercultural competence in an online context through didactic audiovisual translation. Languages, 8(3), 160. https://doi.org/10.3390/languages8030160
Sindoni, M. G., Moschini, I., Adami, E., & Karatza, S. (2021). The common framework of reference for intercultural digital literacies (CFRIDiL): Learning as meaning-making and assessment as recognition in English as an additional language contexts. In Multimodality in English language learning (pp. 221-237). Routledge. https://doi.org/10.4324/9781003155300
Suman, C. (2023). The effective integration of multimedia in foreign language teaching. A Peer Rewiev Research Journal of Language and Human Development, 1(1), 83-93. https://doi.org/10.13140/RG.2.2.14985.42085
Takona, J. P. (2024). Research design: qualitative, quantitative, and mixed methods approaches. Quality & Quantity, 58(1), 1011-1013. https://doi.org/10.1007/s11135-023-01798-2
Tang, K. S., Jeppsson, F., Danielsson, K., & Bergh Nestlog, E. (2022). Affordances of physical objects as a material mode of representation: A social semiotics perspective of hands-on meaning-making. International Journal of Science Education, 44(2), 179-200. https://doi.org/10.1080/09500693.2021.2021313
Teng, C., Heydarnejad, T., Hasan, M. K., Omar, A., & Sarabani, L. (2022). Mobile assisted language learning in learning English through social networking tools: An account of Instagram feed-based tasks on learning grammar and attitude among English as a foreign language learners. Frontiers in Psychology, 13, 1012004. https://doi.org/10.3389/fpsyg.2022.1012004
Urbieta, A. S., & Peñalver, E. A. (2021). Multimodal discourse in digital storytelling: An assessment tool proposal. Computer-Assisted Language Learning Electronic Journal, 22(2), 14-25.
Vishnyakova, O. D., & Vishnyakova, E. A. (2022). Linguistic and cultural knowledge acquisition in terms of the multimodal approach to EIL studies. Professional Discourse & Communication, 4(1), 81-92. https://doi.org/10.24833/2687-0126-2022-4-1-81-92
Vnucko, G., & Klimova, B. (2023). Exploring the potential of digital game-based vocabulary learning: A systematic review. Systems, 11(2), 57. https://doi.org/10.3390/systems11020057
Waang, P. (2023). Maximizing the potential of multimedia in Indonesia: Enhancing engagement, accessibility, and learning outcomes. Journal of Appropriate Technology, 9(3), 235-245. https://doi.org/10.37675/jat.2023.00409
Wang, X., Chen, J., & Zhang, T. (2021). Facilitating English grammar learning by a personalized mobile-assisted system with a self-regulated learning mechanism. Frontiers in Psychology, 12, 624430. https://doi.org/10.3389/fpsyg.2021.624430
Warschauer, M., & Whittaker, R. (2015). The Role of Digital Tools in the Language Learning Process. Language Learning & Technology, 19(2), 4–9. https://doi.org/10125/44464
Zhou, Y., & Wei, M. (2018). Strategies in technology-enhanced language learning. Studies in Second Language Learning and Teaching, 8(2), 471-495.
Zhu, T., Zhang, Y., & Irwin, D. (2024). Second and foreign language vocabulary learning through digital reading: A meta-analysis. Education and Information Technologies, 29, 4531–4563. https://doi.org/10.1007/s10639-023-11969-1
.jpg)
