Multimodal Meaning Making and Language Learning in Digital Multilingual Environments
DOI:
https://doi.org/10.71435/Keywords:
Multimodal Learning , Language Acquisition , Digital Environments , Speaking ProficiencyAbstract
Purpose: This study investigates the role of multimodal learning tools in language acquisition within digital multilingual environments. The primary goal was to assess how learners engage with semiotic resources like text, images, audio, and video in enhancing language skills
Subjects and Methods: A mixed-methods design was used, with quantitative data from pre- and post-tests measuring improvements in vocabulary, grammar, and speaking proficiency, and qualitative data from interviews and focus groups capturing learners’ perceptions and experiences.
Results: The results indicated that learners using multimodal resources showed significant improvements in all language skills, especially in speaking proficiency. Vocabulary and grammar scores in the experimental group increased by 15% and 12%, respectively, while speaking skills saw a 20% improvement. Learners expressed positive perceptions of the multimodal tools, particularly video-based lessons and interactive exercises, though challenges related to platform navigation and cognitive overload were noted. The study concludes that multimodal learning tools are effective in enhancing language acquisition, but emphasizes the need for careful design to minimize challenges.
Conclusions: Future research should explore ways to optimize digital tools for diverse learner needs, and further investigate the long-term impact of multimodal learning on language retention.
Downloads
References
Bateman, J., C. R. McMillan, J. E. Kress, & G. Van Leeuwen. (2017). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Blair, A., Haneda, M., & Bose, F. N. (2018). Reimagining English‐medium instructional settings as sites of multilingual and multimodal meaning making. Tesol Quarterly, 52(3), 516-539. https://doi.org/10.1002/tesq.449
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Budach, G. (2013). From language choice to mode choice: How artefacts impact on language use and meaning making in a bilingual classroom. Language and Education, 27(4), 329-342. https://doi.org/10.1080/09500782.2013.788188
Castellano-Sanz, M., & Reyes-Torres, A. (2024). Multiliteracies in minority language contexts.: A multimodal learning path to promote learners' meaning-making process and aesthetic experiences. L1-Educational Studies in Language and Literature, 24(3), 1-29. https://doi.org/10.21248/l1esll.2024.24.3.602
Chen, Y. C., Fan, K. K., & Fang, K. T. (2021). Effect of flipped teaching on cognitive load level with mobile devices: The case of a graphic design course. Sustainability, 13(13), 7092. https://doi.org/10.3390/su13137092
Coyle, D., Hood, P., & Marsh, D. (2010). Content and Language Integrated Learning: Learning, Teaching and Strategy. Inggris: Cambridge University Press.
Gee, J. P. (2007). What Video Games Have to Teach Us About Learning and Literacy. Computers in Entertainment, 1(1), 20. https://doi.org/10.1145/1233039.1233040
Han, Y. (2022). Understanding Multilingual Young Adults and Adolescents' Digital Literacies in the Wilds: Implications for Language and Literacy Classrooms. Issues and Trends in Learning Technologies, 9(1), 27-46.
Hasyim, M., & Arafah, B. (2023). Semiotic multimodality communication in the age of new media. Studies in Media and Communication, 11(1), 96-103. https://doi.org/10.11114/smc.v11i1.5865
Henry, A. (2023). Learner–environment adaptations in multiple language learning: Casing the ideal multilingual self as a system functioning in context. International Journal of Multilingualism, 20(2), 97-114. https://doi.org/10.1080/14790718.2020.1798969
Kern, R. (2006). Perspectives on technology in learning and teaching languages. Tesol Quarterly, 40(1), 183-210. https://doi.org/10.2307/40264516
Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Kress, G., & Van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. Inggris: Arnold Publishers.
Lai, B., Tan, K. H., He, M., Mohd Said, N. E., & Muslim, N. (2022). The roles of non-textual elements in sustaining ESL and EFL learning: A scoping review. Sustainability, 14(16), 10292. https://doi.org/10.3390/su141610292
Lee, S. Y., Lo, Y. H. G., & Chin, T. C. (2021). Practicing multiliteracies to enhance EFL learners’ meaning making process and language development: a multimodal Problem-based approach. Computer Assisted Language Learning, 34(1-2), 66-91. https://doi.org/10.1080/09588221.2019.1614959
Ma, Q., & Zhang, Z. (2024). The use of digital storytelling in bilingual/multilingual students’ meaning-making: A systematic literature review. Globalisation, cultural diversity and schooling, 33-73. https://doi.org/10.1007/978-3-031-53219-1_3
Mulisa, F. (2022). Sampling techniques involving human subjects: Applications, pitfalls, and suggestions for further studies. International Journal of Academic Research in Education, 8(1), 74-83. https://doi.org/10.17985/ijare.1225214
O'Hara, M., & Pritchard, S. (2013). Multimodal Learning and Language Development. Language and Education, 27(3), 238–251. https://doi.org/10.1080/09500782.2012.713416
Pun, J. K., & Cheung, K. K. C. (2023). Meaning making in collaborative practical work: A case study of multimodal challenges in a Year 10 chemistry classroom. Research in Science & Technological Education, 41(1), 271-288. https://doi.org/10.1080/02635143.2021.1895101
Sindoni, M. G., Moschini, I., Adami, E., & Karatza, S. (2021). The common framework of reference for intercultural digital literacies (CFRIDiL): Learning as meaning-making and assessment as recognition in English as an additional language contexts. In Multimodality in English language learning (pp. 221-237). Routledge. https://doi.org/10.4324/9781003155300
Skulmowski, A., & Xu, K. M. (2022). Understanding cognitive load in digital and online learning: A new perspective on extraneous cognitive load. Educational psychology review, 34(1), 171-196. https://doi.org/10.1007/s10648-021-09624-7
Suman, C. (2023). The effective integration of multimedia in foreign language teaching. A Peer Rewiev Research Journal of Language and Human Development, 1(1), 83-93. https://doi.org/10.13140/RG.2.2.14985.42085
Takona, J. P. (2024). Research design: qualitative, quantitative, and mixed methods approaches. Quality & Quantity, 58(1), 1011-1013. https://doi.org/10.1007/s11135-023-01798-2
Tang, K. S., Jeppsson, F., Danielsson, K., & Bergh Nestlog, E. (2022). Affordances of physical objects as a material mode of representation: A social semiotics perspective of hands-on meaning-making. International Journal of Science Education, 44(2), 179-200. https://doi.org/10.1080/09500693.2021.2021313
Urbieta, A. S., & Peñalver, E. A. (2021). Multimodal discourse in digital storytelling: An assessment tool proposal. Computer-Assisted Language Learning Electronic Journal, 22(2), 14-25.
Vishnyakova, O. D., & Vishnyakova, E. A. (2022). Linguistic and cultural knowledge acquisition in terms of the multimodal approach to EIL studies. Professional Discourse & Communication, 4(1), 81-92. https://doi.org/10.24833/2687-0126-2022-4-1-81-92
Waang, P. (2023). Maximizing the potential of multimedia in Indonesia: Enhancing engagement, accessibility, and learning outcomes. Journal of Appropriate Technology, 9(3), 235-245. https://doi.org/10.37675/jat.2023.00409
Warschauer, M., & Whittaker, R. (2015). The Role of Digital Tools in the Language Learning Process. Language Learning & Technology, 19(2), 4–9. https://doi.org/10125/44464
Zhou, Y., & Wei, M. (2018). Strategies in technology-enhanced language learning. Studies in Second Language Learning and Teaching, 8(2), 471-495.
