Visual Semiotics and Learner Engagement in Technology Mediated Language Education
DOI:
https://doi.org/10.71435/Keywords:
Visual Semiotics, Learner Engagement, Multimodal LearningAbstract
Purpose: This study aimed to investigate the role of visual semiotics in shaping learner engagement within technology-mediated language education. It sought to explore how visual elements such as images, icons, animations, and color-coded cues influence cognitive, emotional, and behavioral engagement, as well as how learners interpret and make meaning from these semiotic resources.
Subjects and Methods: The study employed a qualitative case study design involving 10–15 language learners participating in digital language learning activities. Data were collected through observations, semi-structured interviews, and analysis of digital learning materials. Observations focused on learners’ interactions with visual semiotic elements, interviews captured learners’ subjective experiences and interpretations, and material analysis identified semiotic features embedded in the digital platform. Data were analyzed using thematic coding and semiotic analysis to identify patterns in learners’ engagement and meaning-making.
Results: The findings revealed that visual semiotics serve as cognitive scaffolds, enhancing comprehension, reducing cognitive load, and supporting strategic learning. Emotional engagement was strengthened through the aesthetic, interactive, and meaningful design of visual materials, increasing motivation, confidence, and enjoyment. Behavioral engagement manifested in active participation, task completion, peer collaboration, and strategic attention to visual cues. Patterns of semiotic meaning-making emerged, showing that learners integrated visual, textual, and experiential information to construct understanding.
Conclusions: Visual semiotics are integral to learner engagement in technology-mediated language education, functioning as active mediators of cognitive, emotional, and behavioral processes. Strategic integration of semiotic resources in instructional design, teacher training, and institutional management can enhance engagement, optimize learning outcomes, and support inclusive, multimodal digital learning environments. These findings highlight the need to reconceptualize engagement frameworks to account for multimodal semiotic affordances.
Downloads
References
Afnan, A. (2023). Digital Literacies for English Language Learners: A Multimodal Social Semiotic Approach to Investigate Saudi Students’ Reading of English Tweets (Doctoral dissertation, Newcastle University).
Ahmed, V., & Opoku, A. (2022). Technology supported learning and pedagogy in times of crisis: the case of COVID-19 pandemic. Education and information technologies, 27(1), 365-405. https://doi.org/10.1007/s10639-021-10706-w
Bayne, S. (2008). Higher education as a visual practice: seeing through the virtual learning environment. Teaching in Higher Education, 13(4), 395-410. https://doi.org/10.1080/13562510802169665
Benz, C., Riefle, L., & Satzger, G. (2024). User engagement and beyond: a conceptual framework for engagement in information systems research. Communications of the Association for Information Systems, 54(1), 331-359. https://doi.org/10.17705/1CAIS.05412
Chen, T. H. (2023). Is a video worth a thousand words? Enhancing second language reading comprehension through video-based e-book design and presentation. English Teaching & Learning, 47(1), 69-93.
Choi, L., & Chung, S. (2021). Navigating online language teaching in uncertain times: Challenges and strategies of EFL educators in creating a sustainable technology-mediated language learning environment. Sustainability, 13(14), 7664. https://doi.org/10.3390/su13147664
Dahlström, H. (2022). Students as digital multimodal text designers: A study of resources, affordances, and experiences. British Journal of Educational Technology, 53(2), 391-407. https://doi.org/10.1111/bjet.13171
Gogawale, S., Deshpande, M., Kumar, P., & Ben-Gal, I. (2024). Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision. arXiv preprint arXiv:2412.00429. https://doi.org/10.48550/arXiv.2412.00429
Goldberg, P., Sümer, Ö., Stürmer, K., Wagner, W., Göllner, R., Gerjets, P., ... & Trautwein, U. (2021). Attentive or not? Toward a machine learning approach to assessing students’ visible engagement in classroom instruction. Educational Psychology Review, 33(1), 27-49. https://doi.org/10.1007/s10648-019-09514-z
Higgs, J. M., & Kim, G. M. (2022). Interpreting old texts with new tools: digital multimodal composition for a high school reading assignment. English Teaching: Practice & Critique, 21(2), 128-142. https://doi.org/10.1108/ETPC-07-2020-0079
Huang, Y., & Wang, C. (2024). Image, Symbol, and Philosophy: Meaning Construction and Semiotic Analysis in Graphic Design. Cultura: International Journal of Philosophy of Culture and Axiology, 21(2).
Hung, B. P., Pham, A. T. D., & Purohit, P. (2022). Computer mediated communication in second language education. New trends and applications in Internet of Things (IoT) and big data analytics, 45-60. https://doi.org/10.1007/978-3-030-99329-0_4
Knight, J. (2023). From digital tools to digital others: A social semiotic technologies approach for studying learner interactions with screen-based resources. In Frontiers in Technology-Mediated Language Learning (pp. 73-97). Routledge.
Le, C. D. (2021). Using technology-enhanced language learning environments to influence the communicative potential of adult learners of English as a foreign language in Vietnam (Doctoral dissertation, Victoria University).
Lu, B., & Hanim, R. N. (2024). Enhancing Learning Experiences through Interactive Visual Communication Design in Online Education. Eurasian Journal of Educational Research (EJER), (109). https://doi.org/10.14689/ejer.2024.109.009
Mukherjee, T., & Dhar, R. L. (2023). Unraveling the black box of job crafting interventions: A systematic literature review and future prospects. Applied Psychology, 72(3), 1270-1323. https://doi.org/10.1111/apps.12434
Rawat, S., Rodrigues, M., Sheregar, P., Wagaskar, K. A., & Tripathy, A. K. (2024, June). Computer vision based hybrid classroom attention monitoring. In 2024 IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS) (pp. 1-6). IEEE. https://doi.org/10.1109/ICITEICS61368.2024.10624965
Smith, B. A., Tan, S., Podlasov, A., & O'Halloran, K. L. (2011). Analysing multimodality in an interactive digital environment: software as a meta-semiotic tool. Social Semiotics, 21(3), 359-380. https://doi.org/10.1080/10350330.2011.564386
Suárez Giri, F., & Sánchez Chaparro, T. (2024). Unveiling the blackbox within ESG ratings' blackbox: Toward a framework for analyzing AI adoption and its impacts. Business Strategy & Development, 7(4), e70038. https://doi.org/10.1002/bsd2.70038
Thomas, M., Reinders, H., & Warschauer, M. (2013). Contemporary computer-assisted language learning: The role of digital media and incremental change. Contemporary computer-assisted language learning, 30-47.
Travere, A. (2023). The interplay of signs and visuals: Unveiling the symbiotic relationship between semiotics and visual communication. Journal of Linguistics and Communication Studies, 2(3), 28-40.
Tudini, V., & Liddicoat, A. J. (2024). Technology-mediated discourse and second language research. The Routledge handbook of second language acquisition and discourse, 297-310. https://doi.org/10.1007/978-3-030-99329-0_4
Tyrer, C. (2021). The voice, text, and the visual as semiotic companions: an analysis of the materiality and meaning potential of multimodal screen feedback. Education and Information Technologies, 26(4), 4241-4260. https://doi.org/10.1007/s10639-021-10455-w
Udris-Borodavko, N., Oliinyk, V., Bozhko, T., Budnyk, A., & Hordiichuk, Y. (2023). Aesthetics and semiotics in 21st century visual communications: Pedagogical and sociocultural aspects. Research Journal in Advanced Humanities, 4(4), 22-40. https://doi.org/10.58256/ntj89217
Vladova, G., Ullrich, A., Bender, B., & Gronau, N. (2021). Students’ acceptance of technology-mediated teaching–how it was influenced during the COVID-19 pandemic in 2020: a study from Germany. Frontiers in Psychology, 12, 636086. https://doi.org/10.3389/fpsyg.2021.636086
Wang, Q. (2024). Designing Technology-Mediated Learning Environments. https://doi.org/10.1007/978-981-96-0680-1
Zou, R., Dechsubha, T., & Wang, Y. (2024). Mapping the Semiotic Landscape in Education: Language, Multimodality, and Educational Transformation. Language Related Research, 15(5), 283-311. https://doi.org/10.48311/LRR.15.5.283
