摘要

近年來利用電子書提升第二語言學習日漸廣泛，但現存研究還未能找出最佳數位內容的呈現模式以及不同閱讀輔助對學習行為與成效之影響。因此，本研究以眼動追蹤法（eye-tracking）針對52位英語為第二語之高階學習者在兩種呈現模式（同時呈現、序列呈現）與六種閱讀輔助（三種局部理解：字彙字義註解、重點字彙剖析（包含型、義、語用資訊）、註腳；三種整體理解：插圖、資訊圖表、照片）不同閱讀情境下的眼動行為進行分析。本研究發現，當以第二語學習者以字彙習得為閱讀目的時，他們會花更多凝視時間聚焦在重點字彙剖析與字彙字意註解上；但是第二語學習者以理解為閱讀目的時，則花更多的凝視時間在插圖分析上。此外，序列呈現模式（數位內容逐步呈現）則引導第二語學習者花更多的凝視時間，聚焦在重點字彙剖析上；這也代表在此模式下，受試者比較注意局部輔助資訊，閱讀目地也偏向以字彙習得為主。整體而言，本研究證明電子書學習第二外語之有效性，同時也發現在兩種電子語料呈現模式和不同閱讀目的下，六種多媒體輔助對閱讀行為的影響有相對顯著的差異。最後，根據本研究成果，本論文提出針對教師、電子書使用者以及電子書出版商對於閱讀教學之建議、並針對未來研究提供建議。

關鍵字：眼動追蹤技術、多媒體學習、電子書、第二語言習得、字彙習得、閱讀理解

Abstract

Employing the eye-tracking technology, this study probed the online reading behaviors of advanced L2 English learners reading for vocabulary acquisition and comprehension. Fifty-two university students with advanced L2 (English) proficiency read an e-book containing six types of multimedia supports for either vocabulary acquisition or comprehension. The six multimedia supports consisted of three micro-level supports that provided information about specific words (glosses, vocabulary focus, and footnotes), and three macro-level supports that provided global or background information (illustrations, infographics, and photos). The participants read the e-book under two input presentation modes frequently seen in instructional e-books: 1) simultaneous mode: where digital input and supports were presented at the same time; and 2) sequential mode: where the digital content and supports were incrementally presented. Analyses of the participants' total fixation duration showed that vocabulary focus and glosses were significantly fixated on when reading for vocabulary acquisition, and illustrations were more intensely fixated on when reading for comprehension. Additionally, vocabulary focus received significantly higher total fixation duration when the digital content was incrementally presented. This suggests that reading under the sequential presentation mode has the potency to guide L2 learners' conscious attention toward micro-level supports that encompass lexical form, meaning, and use. This study confirmed the effectiveness of multimedia learning, while illustrating significant distinctions between the relative contributions of micro-level and macro-level multimedia supports under different literacy purposes and presentation modes. Based on the findings of this study, L2 learners' online e-book reading behaviors and pedagogical implications are discussed.

Key words: eye-tracking, multimedia learning, e-books, second language learning, vocabulary acquisition, comprehension

References

AbuSeileek, A. F. (2011). Hypermedia annotation presentation: The effect of location and type on the EFL learners' achievement in reading comprehension and vocabulary acquisition. Computers & Education, 57(1), 1281-1291.
Akbulut, Y. (2007). Effects of multimedia annotations on incidental vocabulary learning and reading comprehension of advanced learners of English as a foreign language. Instructional Science, 35(6), 499-517.
Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning & Technology, 5(1), 202-232.
Ally, B. A. (2012). Using pictures and words to understand recognition memory deterioration in amnestic mild cognitive impairment and Alzheimer's disease: a review. Current Neurology and Neuroscience Reports, 12(6), 687-694.
Ariew, R., & Ercetin, G. (2004). Exploring the potential of hypermedia annotations for second language reading. Computer Assisted Language Learning, 17(2), 237-259.
Aust, R., Kelley, M. J., & Roby, W. (1993). The use of hyper-reference and conventional dictionaries. Educational Technology Research and Development, 41(4), 63-74.
Author (2009). Publication information removed.
Author (2014). Publication information removed.
Author (2015). Publication information removed.
Brown, M. (2008). Marc Brown's Arthur Anniversary Collection. Retrieved from http://www.goodreads.com/
Carpenter, S. K., & Olson, K. M. (2012). Are pictures good for learning new vocabulary in a foreign language? Only if you think they are not. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(1), 92-101.
Chen, I. J., & Yen, J. C. (2013). Hypertext annotation: Effects of presentation formats and learner proficiency on reading comprehension and vocabulary learning in foreign languages. Computers & Education, 63, 416-423.
Chun, D. M., & Plass, J. L. (1996). Effects of multimedia annotations on vocabulary acquisition. The Modern Language Journal, 80(2), 183-198.
Ciampa, K. (2012). ICANREAD: The effects of an online reading program on grade 1 students' engagement and comprehension strategy use. Journal of Research on Technology in Education, 45(1), 27-59.
De Jong, M. T., & Bus, A. G. (2003). How well suited are electronic books to supporting literacy? Journal of Early Childhood Literacy, 3(2), 147-164.
Diaz, G. (2015, October). Fighting Cancer, With a Little Help. Advanced. Retrieved from http://www.studioclassroom.com/e-paper/Epaper_Read_AD.php?no=472
Ercetin, G. (2013). Exploring ESL learners' use of hypermedia reading glosses. CALICO Journal, 20(2), 261-283.
Field, J. (2004). An insight into listeners' problems: too much bottom-up or too much top-down? System, 32(3), 363-377.
Gascoigne, C. (2005). Toward an understanding of the relationship between L2 reading comprehension and grammatical competence. The Reading Matrix, 5(2), 1-14.
Gilbert, J. K. (2010). The role of visual representations in the learning and teaching of science: An introduction. Asia-Pacific Forum on Science Learning and Teaching, 11(1), 1-20.
Gweon, H., & Schulz, L. (2011). 16-month-olds rationally infer causes of failed actions. Science, 332(6037), 1524-1524.
Han, Z., & Liu, Z. (2013). Input processing of Chinese by ab initio learners. Second Language Research, 29(2), 145-164.
Han, Z., & Peverly, S. T. (2007). Input processing: A study of ab initio learners with multilingual backgrounds. International Journal of Multilingualism, 4(1), 17-37.
Han, Z., Park, E. S., & Combs, C. (2008). Textual enhancement of input: Issues and possibilities. Applied Linguistics, 29(4), 597-618.
Hoffman, J. L., & Paciga, K. A. (2014). Click, swipe, and read: sharing e-books with toddlers and preschoolers. Early Childhood Education Journal, 42(6), 379-388.
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.
Huang, H. C. (2013). E-reading and e-discussion: EFL learners' perceptions of an e-book reading program. Computer Assisted Language Learning, 26(3), 258-281.
Huang, H. C., Chern, C. L., & Lin, C. C. (2009). EFL learners' use of online reading strategies and comprehension of texts: An exploratory study. Computers & Education, 52(1), 13-26.
Huang, Y. M., & Liang, T. H. (2015). A technique for tracking the reading rate to identify the e‐book reading behaviors and comprehension outcomes of elementary school students. British Journal of Educational Technology, 46(4), 864-876.
Hulstijn, J. (1989). Implicit and incidental second language learning: Experiments in the processing of natural and partly artificial input, Interlingual Processes, 1, 49-73.
Hyltenstam, K. (1988). Lexical characteristics of near-native second-language learners of Swedish. Journal of Multilingual & Multicultural Development, 9(1-2), 67-84.
Jones, L. (2004). Testing L2 vocabulary recognition and recall using pictorial and written test items. Language Learning and Technology, 8, 122–143.
Koda, K. (2010). The role of reading in fostering transcultural competence. Reading in a Foreign Language, 22, 5-10.
Korat, O., Levin, I., Atishkin, S., & Turgeman, M. (2014). E-book as facilitator of vocabulary acquisition: support of adults, dynamic dictionary and static dictionary. Reading and Writing, 27(4), 613-629.
Kotval, X. P., & Goldberg, J. H. (1998). Eye movements and interface component grouping: an evaluation method. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(5), 486-490.
Lervåg, A., & Aukrust, V. G. (2010). Vocabulary knowledge is a critical determinant of the difference in reading comprehension growth between first and second language learners. Journal of Child Psychology and Psychiatry, 51(5), 612-620.
Lin, C. C., & Tseng, Y. F. (2012). Videos and Animations for Vocabulary Learning: A Study on Difficult Words. Turkish Online Journal of Educational Technology-TOJET, 11(4), 346-355.
Lin, C. C., & Yu, Y. C. (2016). Effects of presentation modes on mobile-assisted vocabulary learning and cognitive load. Interactive Learning Environments, 1-15.
Lin, L. F. (2010). A video‐based CALL program for proficient and less‐proficient L2 learners' comprehension ability, incidental vocabulary acquisition. Educational Media International, 47(3), 199-216.
Liou, H. C., Chang, J. S., Chen, H. J., Lin, C. C., Liaw, M. L., Gao, Z. M., & You, G. N. (2006). Corpora processing and computational scaffolding for a Web-based English learning environment: the CANDLE project. CALICO Journal, 77-95.
Liu, T. C., Fan, M. H. M., & Paas, F. (2014). Effects of digital dictionary format on incidental acquisition of spelling knowledge and cognitive load during second language learning: Click-on vs. key-in dictionaries. Computers & Education, 70, 9-20.
Lotherington, H., & Jenson, J. (2011). Teaching multimodal and digital literacy in L2 settings: New literacies, new basics, new pedagogies. Annual Review of Applied Linguistics, 31, 226-246.
Lwo, L., & Lin, M. C. T. (2012). The effects of captions in teenagers' multimedia L2 learning. ReCALL: The Journal of EUROCALL, 24(2), 188-208.
Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19.
Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-coding hypothesis. Journal of Educational Psychology, 83(4), 484.
Miller, E. B., & Warschauer, M. (2014). Young children and e-reading: research to date and questions for the future. Learning, Media and Technology, 39(3), 283-305.
Miyasako, N. (2002). Does text-glossing have any effects on incidental vocabulary learning through reading for Japanese senior high school students? Language Education & Technology, (39), 1-20.
Mol, L. (2011). The potential role for infographics in science communication. Master's thesis, Biomedical Sciences, Vrije Universiteit, Amsterdam, Netherlands.
Nassaji, H. (2003). Higher–level and lower–level text processing skills in advanced ESL reading comprehension. The Modern Language Journal, 87(2), 261-276.
Nassaji, H. (2006). The relationship between depth of vocabulary knowledge and L2 learners' lexical inferencing strategy use and success. The Modern Language Journal, 90(3), 387-401.
Nation, P., & Meara, P. (2002). Vocabulary. An Introduction to Applied Linguistics, 35-54.
Poole, A., & Ball, L. J. (2006). Eye tracking in HCI and usability research. Encyclopedia of Human Computer Interaction, 1, 211-219.
Press, P. P. (2011). Al Gore–Our Choice: A Plan to Solve the Climate Crisis. Available from http://ourchoicethebook.com/
Qian, D. D. (2002). Investigating the relationship between vocabulary knowledge and academic reading performance: An assessment perspective. Language Learning, 52(3), 513-536.
Rayner, K., Slattery, T. J., & Bélanger, N. N. (2010). Eye movements, the perceptual span, and reading speed. Psychonomic Bulletin & Review, 17(6), 834-839.
Rockinson-Szapkiw, A. J., Courduff, J., Carter, K., & Bennett, D. (2013). Electronic versus traditional print textbooks: A comparison study on the influence of university students' learning. Computers & Education, 63, 259-266.
Roskos, K., Brueck, J., & Widman, S. (2009). Investigating analytic tools for e-book design in early literacy learning. Journal of Interactive Online Learning, 8(3), 218-240.
Sakar, A., & Ercetin, G. (2005). Effectiveness of hypermedia annotations for foreign language reading. Journal of Computer Assisted Learning, 21(1), 28-38.
Schnotz, W. (2008). Why multimedia learning is not always helpful. In J.F. Rouet, R. Lowe & W. Schnotz (Eds.), Understanding multimedia documents (pp.17-41). New York: Springer.
Shalmani, H. B., & Sabet, M. K. (2010). Pictorial, textual, and picto-textual glosses in E-reading: A comparative study. English Language Teaching, 3(4), 195-203.
Slattery, T. J., & Rayner, K. (2010). The influence of text legibility on eye movements during reading. Applied Cognitive Psychology, 24(8), 1129-1148.
Smeets, D. J., & Bus, A. G. (2012). Interactive electronic storybooks for kindergartners to promote vocabulary growth. Journal of Experimental Child Psychology, 112(1), 36-55.
Smeets, D. J. H., & Bus, A. G. (2014). The interactive animated e-book as a word learning device for kindergartners. Applied Psycholinguistics, 22(1), 1-22.
Smiciklas, M. (2012). The power of infographics: Using pictures to communicate and connect with your audiences. Indianapolis, IN: Que Publishing.
Sweller, J. (2006). How the human cognitive system deals with complexity. Handling Complexity in Learning Environments: Theory and Research, 13-25.
Tsai, M. J., Hou, H. T., Lai, M. L., Liu, W. Y., & Yang, F. Y. (2012). Visual attention for solving multiple-choice science problem: An eye-tracking analysis. Computers & Education, 58(1), 375-385.
Tseng, S. S., Yeh, H. C., & Yang, S. H. (2015). Reading comprehension levels through online annotations. Computer Assisted Language Learning, 28 (1), 41-57.
Türk, E., & Erçetin, G. (2014). Effects of interactive versus simultaneous display of multimedia glosses on L2 reading comprehension and incidental vocabulary learning. Computer Assisted Language Learning, 27(1), 1-25.
Underwood, J. (1989). HyperCard and interactive video. CALICO Journal, 6, 7-20.
van der Laan, L. N., Hooge, I. T., De Ridder, D. T., Viergever, M. A., & Smeets, P. A. (2015). Do you like what you see? The role of first fixation and total fixation duration in consumer choice. Food Quality and Preference, 39, 46-55.
VanPatten, B. (1990). Attending to form and content in the input. Studies in Second Language Acquisition, 12(03), 287-301.
VanPatten, B. (2004). Processing instruction: Theory, research, and commentary. Mahwah, NJ: Erlbaum.
VanPatten, B. (2008). Processing matters in input enhancement. In Piske, T. & Young-Scholten, M. Input matters (pp. 47-61). Clevedon, UK: Multilingual Matters.
VanPatten, B., & Wong, W. (2004). Processing instruction and the French causative: Another replication. Processing Instruction: Theory, Research, and Commentary, 97-118.
Weiss, R. E., Knowlton, D. S., & Morrison, G. R. (2002). Principles for using animation in computer-based instruction: Theoretical heuristics for effective design. Computers in Human Behavior, 18(4), 465-477.
Yanguas, I. (2009). Multimedia glosses and their effect on L2 text comprehension and vocabulary learning. Language Learning & Technology, 13(2), 48-67.
Yoon, T. (2013). Exploring a literacy development in young Korean ELLs with online E-books. Journal of Language Teaching and Research, 4(2), 253-259.
Yoshii, M., & Flaitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture annotation types. CALICO journal, 33-58.