人臉偵測與人臉辨識目前廣泛應用於門禁監視系統(surveillance)、智慧型電腦介面控制(HCI)、身分認證(BIA)、安全資訊存取(SIA)等相關應用上。然而人臉偵測又是人臉表情辨識與人臉追蹤前置處理之重要動作，因此探討利用臉部表情作為診斷學習困難度亦為一相當有趣的研究課題。
本文結合影像前處理、人臉偵測及人臉表情辨識技術，應用於診斷學習者的學習困難度。將WebCam所攝取的臉部影像，經過整合型濾波器抑制雜訊，再透過Sobel邊緣偵測取得影像邊緣特徵，接著利用邊緣強化加強影像輪廓;然後利用膚色偵測擷取眼睛和嘴巴的特徵，最後利用迴避度、專注度及愉悅度等向量進行困難度表情辨識，以自行拍攝40人，2040張之人臉資料庫的實驗結果發現學習困難度的辨識率可達92%。可見臉部表情對於診斷學習困難度已有可行性，將來可以應用在E-learning適性學習系統。

Facial detection and recognition have been utilized in areas such as surveillance system ,human controller interface(HCI), biometric identity authentification(BIA) and Security Information Access(SIA). However, Facial detection is an important preprocessing before facial expression recognition and facial tracing , Therefore ,it is an interesting research to design diagnosing the learning difficulty with facial expression recognition.
In this study, we combined the image processing technical ,included image preprocessing ,Facial detection and facial expression recognition. This paper discusses three phase .In the first phase ,It captured the facial picture from a WebCam , Removing noise from the facial picture through an integrated filter while preserving and enhancing edges is one of the most fundamental operations of image processing. Then, the second phase used the color of skin method to extract the eyes and mouth characteristic, Finally, based on the facial emotion vector theories (withdrawal,careness,happiness) , we used it to diagnose the learning difficulty with facial expression recognition.In our experiment , we captured 2040 piece of facial images by 40 students utilizing WebCam for facial recognition database.The experiment result showed an correctness rate of 92 percent . The experiment proved that we can utilizing the proposed method for E-learning system in the further .

一、中文部份
黃俊欽(1996)。灰階人臉辨識之研究，國立交通大學資訊科學研究所碩士論文。
潘奕安(2004)。低解析度影像序列知自動化表情辨識系統，成功大學資訊工程所碩士論文。
蔡騰興(2000)。影像分割技術之深入探討及摭拾，國立台灣海洋大學電機工程學系碩士論文。
簡逸聰(2001)。一個以膚色為基礎之互補人臉偵測策略，國立中央大學資訊工程研究所碩士論文。
吳瑞珍(2002)。人臉特徵自動抽取之演算法設計與應用，元智大學電機工程研究所碩士論文。
王晴右(2003)。表情線索與熟悉度對臉孔辨識歷程之影響，中原大學心理學系碩士論文。
黃泰祥(2004)。具備人臉追蹤與辨識功能的一個智慧型數位監視系統，中原大學電子工程所碩士論文。
吳明衛(2003)。自動化臉部表情分析系統，成功大學資訊工程所碩士論文。
連振昌(2006)。臉部表情辨識技術研究報告，中華大學資工系碩士論文。
張志永(2001)。利用類神經網路於臉部表情自動辨識系統, 國立交通大學電機與控制工程學系碩士論文。
繆紹綱(2005)。“數位影像處理”, 初版，東華書局，台北市，519-596。
溫蒂德里丹(1997)著，“理性情緒心理學入門”，武自珍譯，心理出版社。
張春興(1991)。“現代心理學”， 正大印書館。
楊國樞(2001)。“西爾格德心理學（下）”，桂冠圖書股份有限公司。
尾澤直志(1999)，“人物設計師NO.2感情表現”， 龍溪國際圖書有限公司。
王志良(2007)。學習者情緒空間定義及應用，北京科技大學信息工程學院。
黃健智(2003)。利用情緒模型設計學習代理人，中原大學資訊工程學系碩士論文。
莊謙本(1997)。“數位電路技術電腦化適性測驗之研究”，師大書苑。
王克蒂(1999)。數學遊戲教學之效益研究—以國小四年級學生為例，國立台灣師範大學科學教育研究所碩士論文。
孟昭蘭(1989)。“人類情緒”，上海人民出版社，上海。
王志良(2000)。人工心理學—關於更接近人腦工作模式的科學，北京科技大學學報。
解迎剛(2006)。王志良，人臉與眼部檢測及其在情感識別中的應用，第25屆中國控制會議，pp.1942-1946。

二、 英文部份
Chen, C. W. and Huang, C. L.(1992).” Human face recognition from a single front view”, International Journal of Pattern Recognition and Artificial Intelligence, Vol. 6, No. 4, pp.571-593
Otsu, N.(1979).“A threshold selection method from gray-level histogram”, IEEE Trans. System, Man, and Cybernetics, Vol. 9, No. 1, pp.62-66.
Alattar, A. M. and Rajala, S.(1999). “A..Facial feature localization in front view head and shoulders images”,Proceedings of IEEE International Conference on Acoustics Speech and Signal Processing, Vol. 6 pp.3557-3560.
L. Yin and R. Yang(1995). “Circuits and systems exposition — weighted median filters: a tutorial,” IEEE Trans., Circuit and Systems, vol. 42, no. 10, pp. 583-601, Oct.
T. Chen and H. R. Wu(2001). “Space variant median filters for the restoration of impulse noise corrupted images,” IEEE Trans. Circuit and Systems, vol. 48, no. 8, pp. 784-789, Aug.
P. Perona and J. Malik(1990).“Scale-space and edge detection using anisotropic diffusion,” IEEE Trans., pattern analysis and machine intelligence, vol. 12, no. 7, Jul.
P. S. Marc and J. S. Chen(1991).“Adaptive smoothing: a general tool for early vision,” IEEE Trans., pattern analysis and machine intelligence, vol 13, no. 6, Jun.
W. C. Kao and H. S. Tai(2006).“A Pipelined Architecture Design for Trilateral Noise Filtering,” IEEE.
P.E. Ng and K. K. Ma(2006).“A Switching Median Filter With Boundary Discriminative Noise Detection for Extremely Corrupted Images,” IEEE Trans., vol. 15, no. 6, pp. 1506–1515, June.
H.-L. Eng and K.-K. Ma(2001).“Noise adaptive soft-switching median filter, ”IEEE Trans. Image Process vol. 10, no. 2, pp. 242–251, Feb.
W. Ping, L. Junli L. Dongming, C. Gang, (2007). “A Fast and reliable switching median filter for highly corrupted images by impulse noise,” IEEE , pp. 3427–3430.
A. J. Bardos and J. Sangwine(1997).“Selective Vector Median Filtering Of Color Image ,” IEEE,IPA97, pp. 708–711,July.
M. Basu(2002).”Gaussian-based edge-detection methods―a survey,” IEEE Trans., Systems, man, and cybernetics, vol. 32, no. 3, pp. 252-260, Aug.
M. Yang(2002).“Detecting faces in images,” IEEE Trans. Pattern Anal. Machine Intell., vol. 24, no. 1, pp. 34-58, Jan.
F. Tsalakanidou, S. Malassiotis and M. G. Strintzis(2005).“Face localization and authentication using color and depth images,” IEEE Trans. Image processing, vol. 14, no. 2, pp. 152-168, Feb.
R. L. Hsu, M. A. M, A. K. Jain(1999).“Face detection in color images,” IEEE Trans. Pattern Anal., Machine Intell., vol. 24, no. 5, pp. 696-706, June.
C. H. Lin, J. L. Wu(1999).“Automatic Facial Feature Extraction by genetic algorithm,” IEEE Trans. Image Processing, vol. 8, no. 6, pp. 834-845, June.
M. Pantic, L. J. M. Rothkrantz(2004).“Facial action recognition for facial expression analysis from static face images,” IEEE Trans. System, Man, and Cybernetics, vol. 34, no. 3, pp. 1449-1461, June .
Y. Wang, B. Yuan(2001).“A novel approach for human face detection from color images under complex background,” Pattern Recognition, vol. 34, pp. 1983-1992 .
C. H. Lee, J. S. Kim, K. H. Park(1996).“Automatic human face location in a complex background using motion and color information,” Pattern Recognition, vol. 29, no. 11, pp. 1877-1889 .
M. Zobel, A. Gebhard, D. Paulus, J. Denzler, H. Niemann(2000).“Robust facial feature localization by coupled features,” Proc. IEEE conf. on Automatic Face and Gesture recognition, pp. 2-7, March.
K. M. Lam, H. Yan(1998).“An analytic-to-holistic approach for face recognition based on a single frontal view,” IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 7, pp. 673-686, July.
H. Yao, W. Gao(2001).“Face detection and location based on skin chrominance and lip chrominance transformation from color images,” Pattern Recognition, vol. 34, pp. 1555-1564 .
G. Wei, I. K. Sethi(1999).“Face detection for image annotation,” Pattern Recognition Letters, vol. 20, pp. 1313-1321.
S. H. Jeng, H. Y. M. Liao, C. C. Han, M, Y. Chern, Y. T. Liu(1998).“Facial featre detection using geometrical face model: an efficient approach,” Pattern Recognition, vol. 31, no. 3, pp. 273-282.
M. Hamouz, J. Kittler, J. -K, Kamarainen, P. Paalanen, H. Kalviainen, J. Matas(2005).“Feature-based affine-invariant localization of faces,” IEEE Trans. Pattern Anal. Machine Intell., vol. 27, no. 9, pp. 1490-1495, Sept.
F. Zuo, P. H. N. de With(2005).“Real-time embedded face recognition for smart home,” IEEE Trans. Consumer Electronics, vol. 51, no. 1, Feb.
Y. Dai, Y, Nakano(1996).“Face-texture model based on SGLD and its application in face detection in a color scene,” Pattern Recognition, vol. 29, no. 6, pp. 1007-1017.
J. Miao, B. Yin, K. Wang, L, Shen, X. Chen(1999).“A hierarchical multiscale and multiangle system for human face detection in a complex background using gravity-center template,” Pattern Recognition, vol. 32, pp. 1237-1248
H. Sako, M. WhitehouseA. Smith, A. Suherland(1994). “Real-time facial-feature tracking based on matching techniques and its applications,” Proc. Int. Conf. 12th IAPR, vol. 2, pp. 320-324, Oct.
Y. Hori, K. Shimizu, Y. Nakamura, T. Kuroda(2004).“A real-time multi face detection technique using positive-negative lines-of-face template,” Proc. Int. conf. ICPR’04, vol. 1, pp. 765-768, Aug.
J. W. Kim, B. H. Kang, P. M. Kim, M. S. Cho(1997).“Human face location in image sequences using genetic templates,” Proc. Int. conf. System, Man, cybernetics, vol. 3, pp. 2985-2988, Oct.
H. A. Rowley(1998).“Neural network-based face detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 1, pp. 23-38, Jan.
C. Garcia and G. Tziritas(1999).“Face detection using quantized skin color regions merging and wavelet packet analysis,” IEEE Trans. Meltimedia, vol. 1, no. 3, pp. 264-277, Sep.
K. K. Sung and T. Poggio(1998).“Example-based learning for view-based human face detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 1, pp. 39-51, Jan.
C. A. Waring and X. Liu(2005).“Face detection using spectral histograms and SVMs,” IEEE Trans. System, Man, Cybernetics, vol. 35, no. 3, June.
S. Z. Li, X. Lu, X. Hou, X. Peng, Q. Cheng(2005).“Learning multiview face subspaces and facial pose estimation using independent component analysis,” IEEE Trans. Image Processing, vol. 14, no. 6, pp. 705-712, June
V. P. Kumar, T. Poggio(2000).“Learning-based approach to real time tracking and analysis of faces,” Proc. Int. conf. on Automatic Face and Gesture recognition, pp. 96-101, March.
L. L. Huang, A. Shimizu, Y. Hagihara, H. Kobatake(2003).“Face detection from
cluttered images using polynomial neural network,” Pattern Recognition, vol. 51, pp. 197-211.
Y. Adini, Y. Moses, S. Ullman(1997).“Face recognition: the problem of compensating for changes in illumination direction,” IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 721-732, July.
P. N. Belhumeur, J. P. Hespanha, D. J. Kriegman(1997).“Eigenfaces vs. fisherfaces: recognition using class specific linear projection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 711-720, July.
A. N. Rajagopalan, R. Chellappa, N. T. Koterba(2005).“Background learning for
robust face recognition with PCA in the presence of cluster,” IEEE Trans. Image Processing, vol. 14, no. 6, June .
H. Ai, L. Liang, G. Xu(2001).“Face detection based on template matching and
support vector machines,” Proc. Int. conf. Image Processing, vol. 1, pp. 1006-1009, Oct.
B. Menser, F. Muller(1999).“Face detection in color images using principal components analysis,” Proc. Int. conf. Image Processing and its application, vol. 2, pp. 620-624, July.
B. Moghaddam(1997).A. Pentland, “Probabilistic visual learning for object representation,” IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 696-710, July.
J. Y. Chang and J. L. Chen(2001).“Automated Facial expression recognition system using neural network”, Journal of the Chinese Institure of Engineer , Vol. 24, No. 3,pp.345-356.
M. P. Loh, Y. P. Wong and C. O. Wong(2006).“Facial Expression Recognition for E-learning Systems using Gabor Wavelet & Neural Network”,.IEEE.
I. Kotsia and I. Pitas(2007).” Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines”, IEEE Transactions On Image Processing , Vol. 16, No. 1, pp.172-187.
Q. Xu, P. Zhang,W. Pei and Z. He(2007).“An Automatic Facial Expression Recognition Approach Based On Confusion-Crossed Support Vector Machine Tree”,ICASSP, pp. 625- 628.
H. Zhao, Z. Wang and J. Men(2007). “Facial Complex Expression Recognition
Based on Fuzzy Kernel Clustering and Support Vector Machines”,.ICNC .
J. W. Hong, M. J. Han, K. T. Song and F. Y. Chang(2007).“A Fast Learning Algorithm for Robotic Emotion Recognition”,.IEEE ,pp. 25 – 30.
Conati, C. (2002). Probabilistic assessment of user’s emotions in education games. Journal of Applied Artificial Intelligence, 16(Special Issue on managing cognition an Affect in HCI), pp.555-575.
Hyungil Ahn, Rosalind W.Picard(2005). Affective-Cognitive Learning and Decision Making: A Motivational Reward Framework For Affective Agents. First International Conference On Affective Computing and Intelligent Interaction. pp.866-873