對於許多EFL高中學生來說，英語寫作可能是一個重大挑戰，他們因而尋求線上寫作工具的支援。如今，隨著科技的進步，有各種AI寫作助理可供語言學習者使用，以檢查和評估他們的寫作。鑑於這些AI寫作助理在英語寫作中的重要作用，本研究比較了兩種以ChatGPT驅動的寫作助理對EFL高中生的英語寫作的有效性，分別是Cool English Writing Assistant和Cool English AI Grammar Checker。此外，本研究旨在調查高中學生對這兩種寫作助理使用的看法。
本研究實施了前測和後測以收集量化數據，並發放問卷以探究質性資料。研究對象為來自臺灣南部一所高中的三個班級的112名學生，她們被招募來使用這兩種寫作助理來評估她們的文章。學生被分為兩組(73人和39人)，並分別使用Cool English Writing Assistant和Cool English AI Grammar Checker來檢測她們的寫作。在前測中，受試者被要求寫兩篇文章，這些文章由兩位高中老師評分。在實驗前，受試者接受這兩個寫作助理的使用教程。在實驗期間，參與者寫了五篇文章並使用這兩個工具檢查和修改他們的作品。在後測中，受試者以與前測相同的兩個題目寫了另外兩篇文章，這些文章也由前測相同的兩位老師評分。之後，參與者填寫了問卷以調查她們對於這兩個寫作助理的看法。
經過分析受試者的成績後，統計結果表明Cool English Writing Assistant和Cool English AI Grammar Checker皆有助於高中學生的英語寫作技能。使用Cool English Writing Assistant的學生在組內的後測成績上有顯著地提高。然而，兩組受試者在後測成績上沒有顯著差異，這表明這兩個工具對高中學生的影響相似且同樣有用。
根據問卷的結果，兩組受試者都認為這些工具在檢查他們的英語寫作方面是有用且有效的。她們認為這些工具的使用者介面是友善的，也稱讚其易用性以及對詞彙使用、句子和搭配詞修改上的幫助。這些都提高了她們對英語文法和寫作技巧的理解。許多學生表示願意向朋友推薦這些工具。然而，有學生指出處理速度和偶爾的系統故障的問題。對於兩個寫作助理的優點，受試者提出如友善的介面、在特定領域的有效性、操作的簡便性、有幫助的寫作建議和快速的處理速度。缺點則包括較慢的處理速度、登入過程較繁瑣、字數限制、改變受試者的句子原意及在使用前需要將手寫文章內容數位化的過程。最後，本研究根據結果提出了研究的教學意義和侷限性，並對未來的研究提出了一些建議。

For many EFL high school students, English writing can be a major challenge, prompting them to seek online writing support. The growing trend on writing assistants and technological advancements has led to the development and widespread use of various grammar-checking tools. Nowadays, with the advance of technology, various kinds of AI-powered writing assistants are available for language learners to check and evaluate their writing. Given the significant role of these AI-powered writing assistants in terms of English writing, this study compared the effectiveness of two ChatGPT-based writing assistants on EFL high school students’ English writing, namely Cool English Writing Assistant and Cool English AI Grammar Checker. In addition, this study aimed to investigate the perception of high school students toward the use of these two writing assistants.
In the present study, a pretest and posttest were conducted to collect the quantitative data, and questionnaires were then distributed to probe into the qualitative information. A total of 112 participants from three intact classes at a high school in southern Taiwan were recruited to apply these two writing assistants to the evaluation of their writing. The students were divided into two groups (n=73 and 39) and were assigned to use Cool English Writing Assistant and Cool English AI Grammar Checker to evaluate their writing respectively. In the pretest, participants were asked to write two essays, which were then graded by two high school teachers. Participants were then given a tutorial on how to utilize the two target writing assistants to check their writing before the experiment. During the experiment, the participants wrote five essays and used the two tools to check and revise their works. In the posttest, participants wrote two essays on the same two topics as the pretest, and the essays were also graded by the same two teachers in the pretest. Afterwards, a questionnaire was filled out by the participants to research into their perceptions toward the application of the two writing assistants.
After the grades were analyzed, the statistical results indicated that Cool English Writing Assistant and Cool English AI Grammar Checker can benefit high school students' English writing skills. Students using the Cool English Writing Assistant showed significant improvement in their posttest scores compared to their pretest scores. However, there was no significant difference in posttest performance between the two groups, suggesting that both tools have a similar impact and are equally useful for high school students.
According to the questionnaire results, students from both groups found the tools useful and effective for checking their English writing. They praised the tools for their user-friendly interfaces, ease of use, and helpful word, sentence, and collocation revisions, which improved their understanding of English grammar and writing techniques. Many students expressed a willingness to recommend these tools to friends. However, issues such as processing speed and occasional system failures were noted. Participants also highlighted strengths such as user-friendly interfaces, effectiveness in specific areas, ease of operation, helpful writing suggestions, and fast processing speeds. Weaknesses identified included low processing speed, cumbersome login processes, word count limits, unintended changes in sentence meanings, and the need to digitize handwriting before using the tools. Lastly, pedagogical implications and limitations of the study were yielded based on the analysis of the findings. Additionally, suggestions for future research were presented.

Abrams, L., & Davis, D. (2016). The tip-of-the-tongue phenomenon: Who, what, and why. Cognition, Language and Aging, 13–54. https://doi.org /10.1075/z.200
Aka, M. R. (2018). Students’ Problems in English Writing: An Analysis on Senior High School Students’ Writing Work (Doctoral dissertation, UIN AR-RANIRY).
Akbar, M. H. (2023). The Use of Grammarly as a Grammar Online Checker For Students’ Thesis Writing (Doctoral dissertation, IAIN Ponoroho).
AlAfnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3(2), 60-68.
Alharbi, W. (2023). AI in the foreign language classroom: A pedagogical overview of automated writing assistance tools. Education Research International, 2023, 1-15.
Alzoubi, K. A. A. (2023). The Effect of Using Mobile Applications in Treating Learning Difficulties Among Students in Jordan. International Journal of Interactive Mobile Technologies, 17(22).
Attali, Y. (2004). Exploring the feedback and revision features of Criterion. Journal of Second Language Writing, 14(3), 191-205.
Atwell, E. (1987). How to detect grammatical errors in a text without parsing it. In EACL’87 Proceedings of the third conference on European chapter of the Association for Computational Linguistics (pp. 38-45). The Association for Computer Linguistics.
Baskara, F. R. (2023). Integrating ChatGPT into EFL writing instruction: Benefits and challenges. International Journal of Education and Learning, 5(1), 44-55.
Banaruee, H., Khatin-Zadeh, O., & Ruegg, R. (2018). Recasts vs. direct corrective feedback on writing performance of high school EFL learners. Cogent Education, 5(1), 1455333.
Bibi, Z., & Atta, A. (2024). The Role of ChatGPT as AI English Writing Assistant: A Study of Student’s Perceptions, Experiences, and Satisfaction. Annals of Human and Social Sciences, 5(1), 433-443.
Bulatović, V., Mirović, I., & Kaurin, T. (2024). Analyzing Grammarly software for corrective feedback Teacher’s perspective on affordances, limitations and implementation. Focus on ELT Journal, 6(1), 74-86.
Cavaleri, M., & Dianati, S. (2016). You want me to check your grammar again?: The usefulness of an online grammar checker as perceived by students. Journal of Academic Language and Learning, 10(1), A223-A236.
Cao, S., & Zhong, L. (2023). Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation. arXiv preprint arXiv:2309.01645.
Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of second language writing, 12(3), 267-296.
Chen, C. F. E., & Cheng, W. Y. E. C. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes.
Chen, H. J. H. (2009, June). Evaluating two web-based grammar checkers-Microsoft ESL Assistant and NTNU Statistical Grammar Checker. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 14, Number 2, June 2009-Special Issue on Computer Assisted Language Learning.
Chen, H. J. H., Lai, S. L., Lee, K. Y., & Yang, C. T. Y. (2023). Developing and evaluating an academic collocations and phrases search engine for academic writers. Computer Assisted Language Learning, 36(4), 641-668.
Chen, S., Nassaji, H., & Liu, Q. (2016). EFL learners’ perceptions and preferences of written corrective feedback: a case study of university students from Mainland China. Asian-Pacific journal of second and foreign language education, 1, 1-17.
Chodorow, M., Tetreault, J., & Han, N. R. (2007, June). Detection of grammatical errors involving prepositions. In Proceedings of the fourth ACL-SIGSEM workshop on prepositions (pp. 25-30).
Chou, C. L. (2022) Evaluating Error Correction Performance of Five Grammar Checkers. (Unpublished master’s thesis). National Taiwan Normal University, Taipei, Taiwan.
Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Routledge.
Crossley, S. A., Bradfield, F., & Bustamante, A. (2019). Using human judgments to examine the validity of automated grammar, syntax, and mechanical errors in writing. Journal of Writing Research, 11(2), 251-270.
Daniels, P., & Leslie, D. (2013). Grammar software ready for EFL writers. OnCue Journal, 9(4), 391-401.
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y. S., Gašević, D., & Chen, G. (2023, July). Can large language models provide feedback to students? A case study on ChatGPT. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 323-325). IEEE.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340.
De Felice, R., & Pulman, S. (2008, August). A classifier-based approach to preposition and determiner error correction in L2 English. In Proceedings of the 22nd international conference on computational linguistics (Coling 2008) (pp. 169-176).
Dikli, S. (2006). An overview of automated scoring of essays. The Journal of Technology, Learning and Assessment, 5(1).
Ebadi, S., Gholami, M., & Vakili, S. (2023). Investigating the effects of using Grammarly in EFL writing: the case of articles. Computers in the Schools, 40(1), 85-105.
Ecke, P., & Hall, C. J. (2013). Tracking tip-of-the-tongue states in a multilingual speaker: Evidence of attrition or instability in lexical systems? International Journal of Bilingualism, 17(6), 734–751.
Ellis, R., Loewen, S., & Erlam, R. (2006). Implicit and explicit corrective feedback and the acquisition of L2 grammar. Studies in second language acquisition, 28(2), 339-368.
Fang, T., Yang, S., Lan, K., Wong, D. F., Hu, J., Chao, L. S., & Zhang, Y. (2023). Is ChatGPT a highly fluent grammatical error correction system? a comprehensive evaluation. arXiv preprint arXiv:2304.01746.
Fareed, M., Ashraf, A., & Bilal, M. (2016). ESL learners’ writing skills: Problems, factors and suggestions. Journal of education and social sciences, 4(2), 81-92.
Fitria, T. N. (2021). Grammarly as AI-powered English writing assistant: Students’ alternative for writing English. Metathesis: Journal of English Language, Literature, and Teaching, 5(1), 65-78.
Fitria, T. N. (2023). ProWritingAid as AI-Powered Writing Tools: The Performance in Checking Grammar and Spelling of Students’ Writing. Journal Polingua: Scientific Journal of Linguistics, Literature and Language Education, 12(2), 65-72.
Fitriana, K., & Nurazni, L. (2022). Exploring English Department Students’ Perceptions on Using Grammarly to Check the Grammar in Their Writing. Journal of English Teaching, 8(1), 15-25.
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. BioRxiv, 2022-12.
Gamon, M., Leacock, C., Brockett, C., Dolan, W. B., Gao, J., Belenko, D., & Klementiev, A. (2009). Using statistical techniques and web search to correct ESL errors. Calico Journal, 26(3), 491-511.
Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing Assistant’s impact on English language learners. Computers and Education: Artificial Intelligence, 3, 100055.
Ghufron, M. A., & Rosyida, F. (2018). The role of Grammarly in assessing English as a Foreign Language (EFL) writing. Lingua Cultura, 12(4), 395-403.
Griffiths, L., & Nicolls, B. (2010). e-Support4U: An evaluation of academic writing skills support in practice. Nurse Education in Practice, 10(6), 341-348.
Guo, Q., Feng, R., & Hua, Y. (2022). How effectively can EFL students use automated written corrective feedback (AWCF) in research writing?. Computer Assisted Language Learning, 35(9), 2312-2331.
Guo, K., & Wang, D. (2023). To resist it or to embrace it? Examining ChatGPT’s potential to support teacher feedback in EFL writing. Education and Information Technologies, 1-29.
Harunasari, S. Y. (2022). Examining the Effectiveness of AI-integrated Approach in EFL Writing: A Case of ChatGPT.
Hinkel, E., & Fotos, S. (2002). Teaching grammar in writing classes: Tenses and cohesion. New perspectives on grammar teaching in second language classrooms, 181-198.
Hoon, T. B. (2006). Online automated essay assessment: Potentials for writing development. Making a difference with Web technologies. Proceedings of AusWeb, 230-234.
Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: writing better scientific review articles. American journal of cancer research, 13(4), 1148.
Hyland, K., & Hyland, F. (2019). Contexts and issues in feedback on L2 writing. Feedback in second language writing: Contexts and issues, 1-22.
Japos, G. V. (2013). Effectiveness of coaching interventions using Grammarly software and plagiarism detection software in reducing grammatical errors and plagiarism of undergraduate researches. JPAIR Institutional Research, 1(1), 110-112.
John, P., & Woll, N. (2020). Using grammar checkers in an ESL context. Calico Journal, 37(2), 169-192.
Kaplan, N. (1991). Ideology, technology, and the future of writing instruction. Evolving perspectives on computers and composition studies, 11-42.
Karyuatry, L. (2018). Grammarly as a tool to improve students’ writing quality: Free online-proofreader across the boundaries. JSSH (Jurnal Sains Sosial dan Humaniora), 2(1), 83-89.
Kim, H. R., & Mathes, G. (2001). Explicit vs. implicit corrective feedback. The Korea TESOL Journal, 4(1), 57-72.
Khoshnevisan, B. (2019). The affordances and constraints of automatic writing evaluation (AWE) tools: A case for Grammarly. ARTESOL EFL Journal, 2(2), 12-25.
Li, M. (2005). A tentative study of e-grading mechanisms in the teaching of writing. Unpublished master’s thesis). National Tsinghua University, Beijing, China.
Li, Y. (2007). The application of computers in English writing teaching and learning. Journal of Hubei Normal University, 27(5), 139-141.
Liao, H. C. (2016). Enhancing the grammatical accuracy of EFL writing by using an AWE-assisted process approach. System, 62, 77-92.
Lim, F. V., & Phua, J. (2019). Teaching writing with language feedback technology. Computers and Composition, 54, 102518.
Luong, M. T., Sutskever, I., Le, Q. V., Vinyals, O., & Zaremba, W. (2015). Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206, 11-19
Mahboob, A. (2014). Epilogue: Understanding language variation: Implications for EIL pedagogy. The pedagogy of English as an international language: Perspectives from scholars, teachers, and students, 1, 257-265.
Marlina, R., & Giri, R. A. (Eds.). (2014). The pedagogy of English as an international language: Perspectives from scholars, teachers, and students (Vol. 1). Springer.
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2), 2236469.
McAlexander, P. J. (2000). Checking the grammar checker: Integrating grammar instruction with writing. Journal of Basic Writing, 124-140.
Milton, J. (2006). Resource-rich web-based feedback: Helping learners become independent writers. Feedback in second language writing: Contexts and issues, 123-139.
Mirzaii, M., & Aliabadi, R. B. (2013). Direct and indirect written corrective feedback in the context of genre-based instruction on job application letter writing. Journal of Writing Research, 5(2), 191-213.
Moon, D. (2021). Evaluating Corrective Feedback Generated by an AI-Powered Online Grammar Checker. International Journal of Internet, Broadcasting and Communication, 13(4), 22-29.
Mohammadi, M. (2009). Recast and metalinguistic feedback in teaching and learning L2 writing: A comparative study. Journal of Asia TEFL, 6(3).
Naber, D. (2003). A rule-based style and grammar checker.
Nagata, N., & Swisher, M. V. (1995). A study of consciousness‐raising by computer The effect of metalinguistic feedback on second language learning. Foreign Language Annals, 28(3), 337-347.
Navoda, M. R., Weerasooriya, O. W. R. Y., Siriwardhana, A. U. A., Sonali, L. D. A., Krishara, J., & Panduwawala, P. (2023). Automated spelling and grammar checker tool for sinhala. International Research Journal of Innovations in Engineering and Technology, 7(10), 131.
Nova, M. (2018). Utilizing Grammarly in evaluating academic writing: A narrative research on EFL students’ experience. Premise: Journal of English Education and Applied Linguistics, 7(1), 80-96.
ONeill, R., & Russell, A. (2019a). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1).
O’Neill, R., & Russell, A. M. (2019b). Grammarly: Help or hindrance? Academic learning advisors’ perceptions of an online grammar checker. Journal of Academic Language and Learning, 13(1), A88-A107.
Owu-Ewie, C., & Williams, M. R. (2017). Grammatical and lexical errors in students’ English composition writing: The case of three senior high schools (SHS) in the central region of Ghana. Sino-us English teaching, 14(8), 463-482.
Pablo, J. C. I., & Lasaten, R. C. S. (2018). Writing difficulties and quality of academic essays of senior high school students. Asia Pacific Journal of Multidisciplinary Research, 6(4), 46-57.
Park, J. (2019). An AI-based English Grammar Checker vs. Human Raters in Evaluating EFL Learners’ Writing. Multimedia-Assisted Language Learning, 22(1).
Perdana, I., & Farida, M. (2019). Online grammar checkers and their use for EFL writing. Journal of English Teaching, Applied Linguistics and Literatures (JETALL), 2(2), 67-76.
Pi-chen Tsai, & Yuh-show Cheng. (2009). The effects of rhetorical task type, English proficiency, and writing anxiety on senior high school students’ English writing performance. English Teaching & Learning, 33(3), 95-131.
Qassemzadeh, A., & Soleimani, H. (2016). The impact of feedback provision by Grammarly software and teachers on learning passive structures by Iranian EFL learners. Theory and practice in language studies, 6(9), 1884-1894.
Rezaei, S., & Derakhshan, A. (2011). Investigating recast and metalinguistic feedback in task-based grammar instruction. Journal of Language Teaching and Research, 2(3), 655.
Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing writing, 15(1), 18-39.
Roh, J., & Kim, T. (2019). Fostering learner autonomy through CALL and MALL in a Korean class: A case study. Journal of Interactive Learning Research, 30(2), 215-254.
Sahu, S., Vishwakarma, Y. K., Kori, J., & Thakur, J. S. (2020). Evaluating performance of different grammar checking tools. International Journal, 9(2), 2227-2233.
Schmidt-Fajlik, R. (2023). ChatGPT as a grammar checker for Japanese English language learners: A comparison with Grammarly and ProWritingAid. AsiaCALL Online Journal, 14(1), 105-119.
Sheehan, S. (2009). British Council ELT research papers (Vol. 1). British Council. Retrieved from https://www.teachingenglish.org.uk/article/ british-council-elt-research-papers-volume-1.
Stasenko, A., & Gollan, T. H. (2019). Tip of the tongue after any language: Reintroducing the notion of blocked retrieval. Cognition, 193, 104027. https://doi.org/10.1016/j.cognition.2019.104027
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., ... & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894.
Sumakul, D. T. Y., Hamied, F. A., & Sukyadi, D. (2022, February). Students’ Perceptions of the Use of AI in a Writing Class. In 67th TEFLIN International Virtual Conference & the 9th ICOELT 2021 (TEFLIN ICOELT 2021) (pp. 52-57). Atlantis Press.
Sun, Q., & Zhang, L. J. (2023). Examining the effects of English as a foreign language student-writers’ metacognitive experiences on their writing performance. Current Psychology, 42(27), 23743-23758.
Suzuki, W., Nassaji, H., & Sato, K. (2019). The effects of feedback explicitness and type of target structure on accuracy in revision and new pieces of writing. System, 81, 135-145.
Tariq, D. B., & Yusof, M. A. M. (2016). The secondary school students’ usage of English learning websites to self-correct writing errors. Asian Tefl, 1(1), 2016.
Tang, D. (2022) A Study on the Impact of Collocation Referencing Tools on EFL Learners’ Collocation Learning. (Unpublished master’s thesis). National Taiwan Normal University, Taipei, Taiwan.
Tossell, C. C., Tenhundfeld, N. L., Momen, A., Cooley, K., & de Visser, E. J. (2024). Student Perceptions of ChatGPT Use in a College Essay Assignment: Implications for Learning, Grading, and Trust in Artificial Intelligence. IEEE Transactions on Learning Technologies.
Ummah, L. K. (2022). EFL students’ perception of Grammarly premium’s feedback and how they deal with the inaccuracy (Doctoral dissertation, Universitas Islam Negeri Maulana Malik Ibrahim).
Vernon, A. (2000). Computerized grammar checkers 2000: Capabilities, limitations, and pedagogical possibilities. Computers and Composition, 17(3), 329-349.
Warschauer, M., & Kern, R. (Eds.). (2000). Network-based language teaching: Concepts and practice. Cambridge university press.
Wang, Y. J., Shang, H. F., & Briody, P. (2013). Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Computer Assisted Language Learning, 26(3), 234-257.
Ware, P. D., & Warschauer, M. (2006). Electronic feedback and second language writing. Feedback in second language writing: Contexts and issues, 2, 105-122.
Wolfersberger, M. (2003). L1 to L2 writing process and strategy transfer: A look at lower proficiency writers. TESL-EJ.
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q. L., & Tang, Y. (2023). A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5), 1122-1136.
Wu, H., Wang, W., Wan, Y., Jiao, W., & Lyu, M. (2023). Chatgpt or Grammarly? evaluating chatgpt on grammatical error correction benchmark. arXiv preprint arXiv:2303.13648.
Yamashita, T. (2022). Automated written corrective feedback: Error-correction performance and timing of delivery Jim Ranalli, Iowa State University. Language Learning & Technology, 26(1).
Yang, H. (2018). Efficiency of online grammar checker in English writing performance and students’ perceptions. 영어학, 18(3), 328-348.
Yi, X., Gao, J., & Dolan, W. B. (2008). A web-based English proofing system for English as a second language users. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II.
Zhan, H. (2015). Frequent Errors in Chinese EFL Learners’ Topic-Based Writings. English Language Teaching, 8(5), 72-81.
Zhang, J., Ozer, H., & Bayazeed, R. (2020). Grammarly vs. face-to-face tutoring at the writing center: ESL student writers’ perceptions.