摘錄式語音摘要是根據事先定義的摘要比例，從語音文件中選取一些重要的語句來產生簡潔的摘要以代表原始文件的主旨或主題，在近幾年已成為一項非常熱門的研究議題。其中，使用語言模型(Language Modeling)架構結合庫爾貝克-萊伯勒差異量(Kullback-Leibler Divergence)來進行重要語句選取的方法，在一些文字與語音文件摘要任務上已展現不錯的效能。本論文延伸此一方法而三個主要貢獻。首先，基於所謂關聯性(Relevance)的概念，我們探索新穎的語句模型技術。透過不同層次(例如詞或音節)索引單位的使用所建立的語句模型能與文件模型進行比對，來估算候選摘要語句與語音文件的關係。再者，我們不僅使用了語音文件中所含有語彙資訊(Lexical Information)，也使用了語音文件中所含隱含的主題資訊(Topical Information)來建立各種語句模型。最後，為了改善關聯模型(Relevance Modeling)需要初次檢索的問題，本論文提出了詞關聯模型(Word Relevance Modeling)。語音摘要實驗是在中文廣播新聞上進行；相較於其它非監督式摘要方法，本論文所提出摘要方法似乎能有一定的效能提升。

Extractive speech summarization, aiming to select an indicative set of sentences from a spoken document so as to concisely represent the most important aspects of the document, has emerged as an attractive area of research and experimentation. A recent school of thought is to employ the language modeling (LM) framework along with the Kullback-Leibler (KL) divergence measure for important sentence selection, which has shown preliminary promise for extractive speech summarization. Our work in this paper continues this general line of research in three significant aspects. First, we explore a novel sentence modeling approach built on top of the notion of relevance, where the relationship between a candidate summary sentence and the spoken document to be summarized is discovered through various granularities of semantic context for relevance modeling. Second, not only lexical but also topical cues inherent in the spoken document are exploited for sentence modeling. Third, to counteract the shortcoming of the RM approach, need of resorting to a time-consuming retrieval procedure for relevance modeling, we present a word relevance modeling(WRM) approach. Experiments on broadcast news summarization seem to demonstrate the performance merits of our methods when compared to several existing unsupervised methods.

[Barzilay and Elhadad 1997] R. Barzilay and M. Elhadad, “Using lexical chains for text summarization,” Proceedings of Workshop on Intelligent Scalable Text Summarization, pp. 10-17, 1997.
[Baxendale 1958] P. Baxendale “Machine-made index for technical literature - an experiment,” IBM Journal of Research and Development, Vol. 2, No. 4, pp. 354-361, 1958.
[Blei et al. 2003] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” Journal of Machchine. Learning, vol. 3, pp. 993-1022, 2003.
[Boguraev and Neff 2000] B.-K. Boguraev and M.-S. Neff, “Lexical cohesion, discourse segmentation and document summarization,” Proceedings of the 6th International Conference on Content-Based Multimedia Information Access, pp. 962-979, 2000.
[Brin and Page 1998] S. Brin and L. Page, “The anatomy of a large-scale hypertextual web search engine,” Computer Networks and ISDN System, Vol. 30, No. 1-7, pp. 107-117, 1998.
[Carbonell and Goldstein 1998] J. Carbonell and J. Goldstein, “The use of MMR diversity-based reranking for reordering documents and producing summaries,” Proceedings of the 21th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1998), pp. 335-336, 1998.
[Chang and Chien 2009] Y.-L. Chang and J.-T. Chien, “Latent Dirichlet learning for document summarization,” Proceedings of the 34th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 1689-1692, 2009.
[Chen et al. 2009] Y.-T. Chen, B. Chen and H.-M. Wang, “A probabilistic generative framework for extractive broadcast news speech summarization,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 17, No. 1, pp. 95-106, 2009.
[Chen 2009] B. Chen, “Word topic models for spoken document retrieval and transcription,” ACM Transactions on Asian Language Information Processing, Vol. 8, No. 1, pp. 1-27 , 2009.
[Chen et al. 2011] P.-N. Chen, K.-Y. Chen and B. Chen, “Leveraging relevance cues for improved spoken document retrieval,” Proceedings of the 12th Annual Conference of the International Speech Communication Association (Interspeech 2011), pp. 929-932, 2011.
[Chen et al. 2011] Y.-N. Chen, Y. Huang, C.-F. Yeh and L.-S. Lee, “Spoken lecture summarization by random walk over a graph constructed with automatically extracted key terms,” Proceedings of the 12th Annual Conference of the International Speech Communication Association (Interspeech 2011), pp. 933-936, 2011.
[Chen et al. 2012] K.-Y. Chen, H.-C. Chang, B. Chen and H.-M. Wang, “Word relevance modeling for speech recognition,” Proceedings of the 13th Annual Conference of the International Speech Communication Association (Interspeech 2012), 2012.
[Christensen et al. 2008] H. Christensen, Y. Gotoh, and S. Renals, “A cascaded broadcast news highlighter,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 16, No. 1, pp. 151-161, 2008.
[Conroy and Leary 2001] J.-M. Conroy and D.-P. O’Leary, “Text summarization via hidden Markov models,” Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp. 406-407, 2001.
[Earl 1970] L. Earl “Experiments in automatic extracting and indexing,” Journal of Information Storage and Retrieval, Vol. 6, No. 4, pp. 313-330, 1970.
[Edmundson 1969] H.-P. Edmundson “New methods in automatic extracting,” Journal of the Association for Computer Machinery, Vol. 16, No. 2, pp. 264-285, 1969.
[Erkan and Radev 2004] G. Erkan and D.-R. Radev, “LexRank graph-based lexical centrality as salience in text summarization,” Journal of artificial intelligence research, Vol. 22, No. 1, pp. 457-479, 2004.
[Fattah and Ren 2009] M.-A. Fattah and F. Ren, “GMM-GA, MR, FFNN, PNN and GMM based models for automatic text summarization,” Computer Speech and Language, Vol. 23, No. 1, pp. 126-144, 2009.
[Ferrier 2001] L. Ferrier, “A maximum entropy approach to text summarization,” Proceedings of the 32th Association of Computational Linguistics conference (ACL 2001), 2001.
[Furui et al. 2004] S. Furui, T. Kikuchi, Y. Shinnaka, and C. Hori, “Speech to text and speech to speech summarization of spontaneous speech,” IEEE Transactions on Speech and Audio Processing, Vol. 12, No. 4, pp. 401-408, 2004.
[Gally et al. 2004] M. Galley, K. McKeown, J. Hirschberg, and E. Shriberg, “Identifying agreement and disagreement in conversational speech: use of bayesian networks to model pragmatic dependencies,” Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), pp. 669-676, 2004.
[Gally 2006] M. Galley, “A skip-chain conditional random field for ranking meeting utterances by importance,” Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pp. 364-372, 2006.
[Galley and McKeown 2007] ,M. Galley and K. McKeown, “Lexicalized Markov grammars for sentence compression,” Proceedings of Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics Annual Meeting, pp. 180-187, 2007.
[Garg et al. 2009] N. Garg, B. Favre, K. Reidhammer and D.H. Tur, “ClusterRank: A graph based method for meeting summarization,” Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp. 1499-1502, 2009.
[Gillick and Yur 2008] D. Gillick, B. Favre and D.H. Yur, “The ICSI summarization system at TAC 2008,” Proceedings of the Text Analysis Conference (TAC 2008).
[Gillick et al. 2009] D. Gillick, K. Riedhammer, B. Favre and D. H. Yur, “A global optimization framework for meeting summarization,” Proceedings of the 34th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 4769-4772, 2009.
[Gillick et al. 2009] D. Gillick, K. Riedhammer, B. Favre and D. H. Yur, “A global optimization framework for meeting summarization,” Proceedings of the 34th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 4769-4772, 2009.
[Gong and Liu 2001] Y. Gong and X. Liu, “Generic text summarization using relevance measure and latent semantic analysis,” Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp. 19-25, 2001.
[Harwath and Hazen 2012] D. Harwath and T.G. Hazen, “Topic identification based extrinsic evaluation of summarization techniques applied to conversational speech,” Proceedings of the 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), pp. 5073-5076, 2012.
[Hirschberg 2002] J. Hirschberg, “Communication and prosody: functional aspects of prosody,” Speech Communication, Vol. 36, No 2, pp. 31-43, 2002.
[Inoue et al. 2004] A. Inoue, T. Mikami and Y. Yamashita, “Improvement of speech summarization using prosodic information,” Proceedings of Speech Prosody, pp. 599-602, 2004.
[Joachims 2002] T. Joachims, “Learning to classify text using support vector machines: methods theory, and algorithms,” 2002: Kluwer Academic.
[Knight and Marcu 2000] K. Knight and D. Marcu, “Statistics-based summarization - step one: sentence compression,” Proceedings of National Conference of the American Association for Artificial Intelligence, pp. 703-710, 2000.
[Ko and Seo 2008] Y. Ko and J. Seo, “An effective sentence-extraction technique using contextual information and statistical approaches for text summarization,” Pattern Recognition Letters, Vol.29, No. 9, pp. 1366-1371, 2008.
[Kolcz et al. 2001] A. Kolcz, V. Prabakarmurthi and J. Kalita, “Summarization as feature selection for text categorization,” Proceedings of the 10th ACM International Conference on Information and Knowledge Management (CIKM 2001), pp. 365-307, 2001.
[Kong and Lee 2006] S.-Y. Kong and L.-S. Lee, “Improved spoken document summarization using probabilistic latent semantic analysis,” Proceedings of the 31th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2006), 2006.
[Koumpis and Renals 2000] K. Koumpis and S. Renals, “Transcription and summarization of voicemail speech,” Proceedings of International Conference on Spoken Language Processing, pp. 688-891, 2000.
[Kuo and Chen 2006] J.-J. Kuo and H-H. Chen, “Multi-document summary generation using informative and event words,” Journal of ACM Transactions on Asian Language Information Processing, Vol. 7, No. 1, pp. 550-557, 2006.
[Kupiec 1995] J. Kupiec “A trainable document summarizer,” Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1995), pp. 68-73, 1995.
[Lavrenko and Croft 2001] V. Lavrenko and W.-B. Croft, “Relevance based language models,” Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp. 120-127, 2001.
[Lee and Chen 2005] L.-S. Lee and B. Chen, “Spoken document understanding and organization,” IEEE Signal Processing Magazine, Vol. 22, No. 5, pp. 42-60, 2005.
[Lee et al. 2006] J.-H. Lee, S.-Y. Kong, Y- C. Pan, Y. S. Fu, and Y.-T. Huang, “Multilayered summarization of spoken document archive by information extraction and semantic structuring,” Proceedings of the 7th Annual Conference of the International Speech Communication Association (Interspeech 2006), pp. 1539-1542, 2006
[Lee et al. 2009] J.-H. Lee, S. Park, C.-M. Ahn and D. Kim, “Automatic generic document summarization based on non-negative matrix factorization,” Journal of Information Processing and Management, Vol. 45, No. 1, pp. 20-34, 2009.
[Li 2007] X. Li, “A new robust relevance model in the language model framework,” Journal of Information Processing and Management (IPM 2008), Vol. 44, No. 3, pp. 991-1007, 2008.
[Lin 2004] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” Proceedings of the ACL Workshop on Text Summarization Branches Out, pp. 74-81, 2004.
[Lin et al. 2009] H. Lin, J. Bilmes and S. Xie, “Graph based submodular selection for extractive summarization,” IEEE workshop on Automatic Speech Recognition and Understanding (ASRU 2009), pp. 381-386, 2009.
[Lin and Chen 2009] S.-H Lin and B. Chen, “Improved speech summarization with multiple-hypothesis representations and kullback-leibler divergence measures,” Proceeding of the 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp. 1847-1850, 2009.
[Lin et al. 2009] S.-H. Lin, B. Chen and H.-M. Wang, “A comparative study of probabilistic ranking models for Chinese spoken document summarization,” ACM Transactions on Asian Language Information Processing, Vol. 8, No 1, pp. 1-3, 2009
[Lin and Chen 2010] S.-H. Lin and B. Chen, “A survey on speech summarization techniques,” The Association for Computational Linguistics and Chinese Language Processing (ACLCLP 2010) Newsletter, Vol. 21, No. 1, pp. 4-16, February 2010.
[Lin et al. 2010] S.-H. Lin, Y.-M. Yeh, and B. Chen, “Extractive speech summarization from the view of decision theory,” Proceedings of the 11th Annual Conference of the International Speech Communication Association (Interspeech 2010), pp.1684-1987, 2010.
[Lin et al. 2010] S.-H. Lin, Y.-M. Chang, J.-W. Liu and B. Chen, “Leveraging evaluation metric-related training criteria for speech summarization,” Proceedings of the 35th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), pp. 5314-5317, 2010.
[Lin et al. 2011] S.-H. Lin, Y.-M. Yeh and B. Chen, “Leveraging kullback-leibler divergence measures and information-rich cues for speech summarization,” IEEE Transactions on Audio, Speech and Language Processing. Vol. 19, No. 4, pp. 871-882, 2011.
[Liu et al. 2006] Y. Liu, E. Shriberg, A. Stolcke, D. Hillard, M. Ostendorf, and M. Harper, “Enriching speech recognition with automatic detection of sentence boundaries and disfluencies,” IEEE Transactions on Audio, Speech and Language Processing, Vol 14, No 5, pp. 1526-1540, 2006.
[Liu and Xie 2008] Y. Liu and S. Xie, “Impact of automatic sentence segmentation on meeting summarization,” Proceedings of the 33th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2008), pp. 5009-5012, 2008.
[Liu and Liu 2009] F. Liu and Y. Liu, “From extractive to abstractive meeting summaries: can it be done by sentence compression,” Proceedings of Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing 2009.
[Liu and Hakkani-Tur 2011] Y. Liu and D. Hakkani-Tur, “Speech summarization,” In the spoken language understanding systems for extracting semantic information from speech,” 2011.
[Lo et al. 2012] Y.-T. Lo, S.-H. Lin and B. Chen, “Constructing effective ranking models for speech summarization,” Proceedings of the 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), pp. 5053-5056 , 2012.
[Lu and Zhai 2011] Y. Lu, Q. Mei and C.X. Zhai, “Investigating task performance of probabilistic topic models: an empirical study of PLSA and LDA,” Information Retrieval, Vol. 14, No. 2, pp. 178-203, 2011.
[Luhn, 1958] P. Luhn “The automatic creation of literature abstracts,” IBM Journal of Research and Development, Vol. 2, No. 2, pp.159-165, 1958.
[McKeown et al. 2005] K. McKeown, J. Hirschberg, M. Galley, and S. Maskey, “From text to speech summarization,” Proceedings of the 30th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), pp. 997- 1000, 2005.
[Mani and Maybury 1999] I. Mani and M. T. Maybury, “Advances in automatic text summarization”, Cambridge: MIT Press, 1999.
[Marcu 2000] D. Marcu, “The theory and practice of discourse parsing and summarization”, Cambridge: MIT Press, 2000.
[Maskey and Hirschberg 2003] S.R Maskey and J. Hirschberg, “Automatic summarization of broadcast news using structural features,” Proceedings of 8th European Conference on Speech Communication and Technology (Eurospeech 2003), pp. 2781-2784, 2003.
[Maskey and Hirschberg 2005] S.-R Maskey and J. Hirschberg, “Comparing lexical, acoustic/prosodic, discourse and structural features for speech summarization,” Proceedings of the 6th Annual Conference of the International Speech Communication Association (Interspeech 2005), pp. 621-624, 2005.
[Maskey and Hirschberg 2006] S.-R. Maskey and J. Hirschberg, “Summarizing speech without text using hidden markov models,” Proceedings of Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics Annual Meeting, pp. 89-92, 2006.
[Maskey et al. 2006] S.-R Maskey, B. Zhou, and Y. Gao, “A phrase-level machine translation approach for disfluency detection using weighted finite state transducers,” Proceedings of the 7th Annual Conference of the International Speech Communication Association (Interspeech 2006), pp. 749 -752, 2006.
[McKeown et al. 2005] K. McKeown, J. Hirschberg, M. Galley, and S. Maskey, “From text to speech summarization,” Proceedings of the 30th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), pp. 997-1000, 2005.
[Meij et al. 2009] E. Meij, D. Trieschnigg, M. Rijke and W. Kraaij, “Conceptual language models for domain-specific retrieval,” Journal of Information Processing and Management (IPM 2009), Vol. 46, No. 2, pp. 448-469, 2009.
[Mihalcea and Tarau 2004] R. Mihalcea and P. Tarau, “TextRank bringing order into texts,” Proceedings of Empirical Method in Natural Language Processing (EMNLP 2004), pp. 404-411, 2004.
[Mihalcea et al. 2005] G. Murray, S. Renals, and J. Carletta, “Extractive summarization of meeting recordings,” Proceedings of the 6th Annual Conference of the International Speech Communication Association (Interspeech 2005), pp. 593-596, 2005.
[Nenkova and McKeown 2011] A. Nenkova and K. McKeown, “Automatic summarization,” Foundations and Trends® in Information Retrieval, Vol 5, No 2-3, 2011.
[Paice 1990] C.-D. Paice, “Constructing literature abstracts by computer techniques and prospects,” Journal of Information Processing and Management (IPM 1990), Vol. 26, No. 1, pp. 171-186, 1990.
[Penn and Zhu 2008] G. Penn and X. Zhu, “A critical reassessment of evaluation baselines for speech summarization,” Proceedings of Annual Meeting of the Association for Computational Linguistics, pp. 470-478, 2008
[Shen et al. 2007] D. Shen, J.-T. Sun, H. Li, Q. Yang, and Z. Chen, “Document summarization using conditional random fields,” Proceedings of International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2862-2867, 2007.
[Sipos et al. 2012] R. Sipos, P. Shivaswamy and T. Joachims, “Large-margin learning of submodular summarization models,” Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2012), pp. 224-233, 2012.
[Strzalkowski et al. 1998] T. Strzalkowski, J. Wand, and B. Wise, “A robust practical text summarization,” Proceedings of AAAI Conference on Artificial Intelligence Spring Symposium on Intelligent Text Summarization, pp. 26-33, 1998
[Teufel and Moens 2002] S. Teufel and M. Moens, “Summarizing scientific articles: Experiments with relevance and rhetorical status,” Computational Linguistics, Vol 28, No 4, pp. 409-445, 2005
[Wan and Yang 2008] X. Wan and J. Yang, “Multi-document summarization using cluster-based link analysis,” Proceedings of the 31th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2008), pp. 299-306, 2008.
[Wan and Xiao 2008] X. Wan and J. Xiao, “Single document keyphrase extraction using neighborhood knowledge,” Proceedings of the 23nd Conference on Artificial Intelligence (AAAI 2008), Vol. 2, pp. 855-860, 2008.
[Wang et al. 2009] D. Wang, S. Zhu, T. Li, and Y. Gong, “Multi-document summarization using sentence-based topic models,” Proceedings of. Annual Meeting of the Association for Computational Linguistics, pp. 297-300, 2009.
[Witbrock and Mittal 2008] M. Witbrock and V. Mittal, “Ultra summarization: a statistical approach to generating highly condensed non-extractive summaries,” Proceedings of the 22th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1999), pp. 315-316, 1999.
[Wu et al. 2007] C. H. Wu, C. H. Hsieh, and C. L. Huang, “Speech sentence compression based on speech segment extraction and concatenation,” IEEE Transcations on Multimedia, Vol 9, No 2, pp. 434-437, 2007.
[Xie et al. 2009] S. Xie, B. Favre, D. Gillick, D. H. Yur and Y. Liu, “Leveraging sentence weights in a concept-based optimization framework for extractive meeting summarization,” Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp.1503-1506, 2009.
[Xie et al. 2010] S. Xie, H. Lin and Yang Liu, “Semi-supervised extractive speech summarization via co-training algorithm,” Proceedings of the 11th Annual Conference of the International Speech Communication Association (Interspeech 2010), pp. 2522-2525, 2010.
[Zhai and Lafferty 2001] C.X. Zhai and J. Lafferty, “A study of smoothing methods for language models applied to ad hoc information retrieval,” Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp.334-342, 2011.
[Zhang et al. 2007] J.-J. Zhang, H.-Y. Chan and P. Fung, “Improving lecture speech summarization using rhetorical information,” Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU 2007), pp. 195-200, 2007.
[Zhang et al. 2007] J.-J. Zhang, H.-Y. Chan and P. Fung, “A comparative study on speech summarization of broadcast news and lecture speech,” Proceeding of the 8th Annual Conference of the International Speech Communication Association (Interspeech 2007), pp. 2781-2784, 2007.
[Zhang et al. 2010] J.-J. Zhang, H.-Y. Chan and P. Fung, “Extractive speech summarization using shallow rhetorical structure modeling,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 18, No. 6, pp. 1147-1157, 2010.
[Zechner and Waibel 2000] K. Zechner and A. Waibel, “Diasumm: flexible summarization of spontaneous dialogues in unrestricted domains,” Proceeding of International Conference on Computational Linguistics (ICCL 2000), pp. 968-974, 2000.