簡易檢索 / 詳目顯示

研究生: 黃楨喻
Chen-Yu Huang
論文名稱: 英文學習者文章摘要結果自動化評分技術
Document Summarization Automatic Scoring for English Learners
指導教授: 柯佳伶
Koh, Jia-Ling
學位類別: 碩士
Master
系所名稱: 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年: 2014
畢業學年度: 102
語文別: 中文
論文頁數: 67
中文關鍵詞: 自動化評分文章摘要語意關係圖
英文關鍵詞: automatic scoring, document summarization, semantic graph
論文種類: 學術論文
相關次數: 點閱:163下載:9
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 英語為我國語文教學的一門重要科目。以往的研究顯示,大量的閱讀能增進語文能力,但學生在閱讀後是否理解內容則需要適當的評估方式。文章主角及內容摘要的非選擇問題可瞭解學生是否理解文章內容,但此類型的問答題,若由教師進行評分需花費許多時間,因此本研究將文章摘要問答題進行自動化評分,將可加速評估回饋並增加學生練習的機會。本研究從文章內容擷取特徵,使用機器學習的方法建立模型,進行文章類型自動分類,以挑選合適的文意理解問答題。針對學生回答的摘要結果自動化評分,本研究不需要教師提供答案,而是將英文文章及學生的摘要分別建立語意關係圖,運用語意關係圖計算出各字詞在文章及摘要內容中的重要性,並透過比對英文文章及學生摘要的語意關係圖,取出各種比對特徵,以機器學習的方法建立預測評分等第的分類模型,用來對學生回答的摘要進行語意符合程度自動化評分。實驗結果顯示,本研究所提出的方法在文章有明確的字詞表達文章重點時,可達到不錯的正確率。

    English is an important subject of language teaching in our country. Previous studies have shown that a lot of reading can enhance language ability, but it needs appropriate assessment methods to judge that whether students understand the contents after reading. The open questions about article's main role and article's summarization can evaluate whether students understand the content of an article. However, such kind of questions need a lot of time of scoring by teacher. The main goal of this study is to provide automatic scoring for the summarization questions of articles. Accordingly, the students can get evaluation feedback in short time such that it can provide more opportunities for students to practice. In our study, we extract the different features from the content of the article. After that, the machine learning method is used to establish classification model for two article types. According to the article type, suitable questions are selected to be the summarization questions. In the proposed system, teachers are not required to provide answers. Instead, the article and the students' summarizations are represented by semantic graphs in order to calculate the importance score of each word in the article and the students' summarizations, respectively. Then the semantic graphs of the article and the students' summarizations are compared to extract the matching features. Finally, the machine learning method is used to establish the classification model of automatic scoring for the given summarizations. The experiment results show that the proposed method can achieve high accuracy when the articles have distinguishable words to express its focus.

    附表目錄 i 附圖目錄 ii 第一章 緒論 1 1-1 研究動機及目的 1 1-2 研究的範圍與限制 2 1-3 研究方法 3 1-4 論文架構 4 第二章 文獻探討 5 2-1 文章分類方法 5 2-2 自動產生問題系統 5 2-3 概念圖理論 6 2-4 自動化評分方法 7 第三章 系統架構與流程 9 3-1 產生文意理解問答題 9 3-2 自動化評分 10 第四章 主角問答題自動評分 12 4-1 語意關係圖建立 12 4-2 重要性分數計算 20 4-3 主角問答題答案評分 25 4-3-1 主角詞彙挑選 25 4-3-2 主角詞彙比對評分 27 第五章 文意理解敘述題自動評估 29 5-1 教師評分的標準 29 5-2 文意理解敘述題答案評分方法 29 5-2-1 文章與答案之語意表示模型 29 5-2-2 比對特徵擷取 32 5-3 評分模型建立及預測 38 第六章 實驗結果與討論 39 6-1 文章類型自動分類實驗 39 6-1-1 實驗資料來源及評估方法 39 6-1-2 實驗結果 39 6-2 主角問答題實驗 40 6-2-1 實驗資料來源及評估方法 40 6-2-2 實驗結果 40 6-3 文意理解敘述題實驗 42 6-3-1 實驗資料來源及評估方法 42 6-3-2 實驗結果 42 【實驗一】文意理解敘述題自動評分效果 42 【實驗二】摘要預測評分特徵值選取 45 【實驗三】單向語意關係圖文意理解敘述題自動評分效果 46 第七章 結論與未來研究方向 51 參考文獻 52 附錄一 系統自動擷取主角詞彙結果 54 1-1 無方向性語意關係圖的主角詞彙結果 54 1-2 雙向語意關係圖的主角詞彙結果 55 1-3 單向語意關係圖的主角詞彙結果 57 附錄二 實驗資料(英文文章) 59

    [1] M. Agarwal and P. Mannen, “Automatic Gap-fill Question Generation from Text Books,” in Proceeding IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, Pages 56-64, 2011.
    [2] L. F. Bachman, N. Carr, G. Kamei, M. Kim, M. J. Pan, C. Salvador and Y. Sawaki, “A Reliable Approach to Automatic Assessment of Short Answer Free Responses,” in Proceeding of the 19th international conference on Computational linguistics - Volume 2, Pages 1-4, 2002.
    [3] C.-C. Chang and C.-J. Lin, “LIBSVM: A Library for Support Vector Machines,” National Taiwan University, Taipei, Taiwan, 2013.
    [4] S. Curto, A. C. Mendes and L. Coheur, “Exploring linguistically-rich patterns for question generation,” in Proceeding of the UCNLG+Eval: Language Generation and Evaluation Workshop, Pages 33-38, 2011.
    [5] R. Mitkov and L. A. Ha, “Computer-Aided Generation of Multiple-Choice Tests,” in Proceeding Natural Language Processing and Knowledge Engineering, 2003.
    [6] M. Mohler, R. Bunescu and R. Mihalcea, “Learning to Grade Short Answer Questions using Semantic SimilarityMeasures and Dependency Graph,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, Pages 752-762, 2011.
    [7] A. J. C. Joseph D. Novak, “The Theory Underlying Concept Maps and How to Construct and Use Them,” Technical Report IHMC CmapTools 2006-01 Rev 01-2008, Florida Institute for Human and Machine Cognition, 2008.
    [8] C. P. Ros´e, A. Roque, D. Bhembe and K. VanLehn, “A Hybrid Approach to Content Analysis for Automatic Essay Grading,” in Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers - Volume 2, Pages 88-90, 2003.

    [9] S. G. Pulman and J. Z. Sukkarieh, “Automatic Short Answer Marking,” in Proceedings of the second workshop on Building Educational Applications Using NLP, Pages 9-16, 2005.
    [10] E. Sumita, F. Sugaya and S. Yamamoto, “Measuring Non-native Speakers Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions,” in Proceedings of the 2nd Workshop on Building Educational Applications Using NLP, pages 61–68, 2005.
    [11] Y.-C. Yang, J.-F. Yang, J.-M. Chang and J. S. Chang, “電腦輔助閱讀測驗自動出題Development of a Computer Assisted Reading Comprehension Test,” in International Conference on English Instruction and Assessment, 2006.
    [12] K. Zubrinic, D. Kalpic2 and M. Milicevic, “The automatic creation of concept maps from documents written using morphologically rich languages,” Expert Systems with Applications, page 12709-12718, 2012.
    [13] R. Ziai, N. Ott and D. Meurers, “Short Answer Assessment: Establishing Links Between Research Strands, ” In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, page 190-200, 2012.
    [14] 張春興, 教育心理學-三化取向的理論與實踐, 台北市: 東華書局, 1989.
    [15] 李秀娟, “不同教學策略對國中生學習生物的影響,” 國立台灣師範大學科學教育研究所碩士論文, 1998.
    [16] Stanford Corenlp, http://nlp.stanford.edu/software/corenlp.shtml.
    [17] Wordnet, http://wordnet.princeton.edu/.

    下載圖示
    QR CODE