經由報告顯示，續多交通事故都可歸咎於駕駛者的疲倦或勞累，因為疲倦會影響駕駛者的視野、警覺性與決策的能力，會使駕駛的效率與能力降低。
在本研究中，我們發展一視覺式的疲倦偵測與警告系統，可估計出駕駛者的潛在疲倦狀態，且做出一些警告。當假設駕駛者處於低疲倦狀態時，我們可以為駕駛者作一些干涉性較小的動作，例如：開啟空調系統、散播一些香水或打開收音機提供一些娛樂；當駕駛者處於高疲倦狀態時，我們可以開啟導航系統輔助或警告其他人本車駕駛處於高疲倦狀態。
疲倦程度的計算是藉由臉部影像來判斷，影像由置於車輛前方的攝影機取得。本系統主要包含了五大步驟：前處理、臉部特徵擷取、臉部追蹤、狀態參數的計算、疲倦程度的推測。
在前處理部份，我們首先降低輸入影像的維度以加快系統速度。接下來對光線作補償，以降低周遭環境光線的影響。最後我們對每像點計算其在不同色彩空間的色度值，提供之後臉部特徵擷取的使用。
臉部特徵擷取主要包含四個子步驟：膚色偵測、臉部定位、眼睛偵測、嘴巴偵測與特徵確認。膚色區域偵測由前處理所得到的色度值，經由膚色模型來計算。接下來臉部定位，我們搜尋最大的膚色區域，然而所得到的臉部區域通常都是不夠完整的，因此僅在不完整的膚色區域搜尋臉部特徵是不可靠的，所以我們實際上是在整張影像搜尋臉部特徵，而臉部區域則用來做確認臉部特徵的動作。
當臉部特徵確認後，我們即利用其作臉部的追蹤，直到臉部追蹤失敗，此時臉部特徵擷取才會再次被執行，因為臉部特徵擷取是較耗時的，而臉部追蹤是較快且較可靠的。
當系統處於臉部追蹤時，臉部的狀態參數會在此時計算出來，包括了：單位時間內閉眼所佔的比例、眨眼頻率、閉眼的時間長度、凝視狀態、嘴巴張開的時間長度與頭部的轉動。
最後我們利用這些狀態參數來推算駕駛的疲倦程度，使用的是模糊積分，其可將各種不同的參數統合起來，推測出一疲倦程度。
我們對許多不同的駕駛者與照明程度作測試，結果顯示我們系統可以在白天正常的運作，未來將把系統的使用性延伸至晚上，對於這點我們將納入紅外線攝影機來幫助系統的實現。

Many traffic accidents have been reported due to driver’s drowsiness/fatigue. Drowsiness degrades driving performance due to the declinations of visibility, situational awareness and decision-making capability. In this study, a vision-based drowsiness detection and warning system is presented, which attempts to bring to the attention of a driver to his/her own potential drowsiness. The information provided by the system can also be utilized by adaptive systems to manage noncritical operations, such as starting a ventilator, spreading fragrance, turning on a radio, and providing entertainment options. In high drowsiness situation, the system may initiate navigation aids and alert others to the drowsiness of the driver.
The system estimates the fatigue level of a driver based on his/her facial images acquired by a video camera mounted in the front of the vehicle. There are five major steps involved in the system process: preprocessing, facial feature extraction, face tracking, parameter estimation, and reasoning. In the preprocessing step, the input image is sub-sampled for reducing the image size and in turn the processing time. A lighting compensation process is next applied to the reduced image in order to remove the influences of ambient illumination variations. Afterwards, for each image pixel a number of chrominance values are calculated, which are to be used in the next step for detecting facial features.
There are four sub-steps constituting the feature extraction step: skin detection, face localization, eyes and mouth detection, and feature confirmation. To begin, the skin areas are located in the image based on the chrominance values of pixels calculated in the previous step and a predefined skin model. We next search for the face region within the largest skin area. However, the detected face is typically imperfect. Facial feature detection within the imperfect face region is unreliable. We actually look for facial features throughout the entire image. As to the face region, it will later be used to confirm the detected facial features. Once facial features are located, they are tracked over the video sequence until they are missed detecting in a video image. At this moment, the facial feature detection process is revoked again. Although facial feature detection is time consuming, facial feature tracking is fast and reliable.
During facial feature tracking, parameters of facial expression, including percentage of eye closure over time, eye blinking frequency, durations of eye closure, gaze and mouth opening, as well as head orientation, are estimated. The estimated parameters are then utilized in the reasoning step to determine the driver’s drowsiness level. A fuzzy integral technique is employed, which integrates various types of parameter values to arrive at a decision about the drowsiness level of the driver. A number of video sequences of different drivers and illumination conditions have been tested. The results revealed that our system can work reasonably in daytime. We may extend the system in the future work to apply in nighttime. For this, infrared sensors should be included.

[Ada00] Y. Adachi, A. Imai, M. Ozaki, and N. Ishii, “Extraction of Face Region by Using Characteristics of Color Space and Detection of Face Direction through an Eigenspace,” Proc. of 4th Int’l Conf. on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, vol. 1, pp. 393-396, Aug. 2000.
[Can02] J. L. Cantero, M. Atienza, and R. M. Salas, “Human Alpha Oscillations in Wakefulness, Drowsiness Period, and REM Sleep: Different Electroencephalographic Phenomena within the Alpha Band,” Neurophysiol Clin, 32, pp. 54–71, 2002.
[Cap07] O. Cappe, S. J. Godsill, and E. moulines, “An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo,” Proceedings of the IEEE, vol. 95, no. 5, pp. 899- 924, May 2007.
[Car00] B. C. Carmona, G. F. Capote, B. G. Botebol, L. P. García, A. A. Sánchez, G. J. Castillo, “Assessment of Excessive Day-Time Sleepiness in Professional Drivers with Suspected Obstructive Sleep Apnea Syndrome,” Arch Bronconeumol, 36(8), pp. 436-440, Sep. 2000.
[Cha99] D. Chai and K. N. Ngan, “Face Segmentation Using Skin-Color Map in Videophone Applications,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 9, no. 4, pp. 551-564, Jun. 1999.
[Che03] D. S. Chen and Z. K. Liu, “A Novel Approach to Detect and Correct Highlighted Face Region in Color Image,” Proc. Of IEEE Conf. on Advanced Video and Signal Based Surveillance, pp. 7-12, Jul. 2003.
[Con02] C. Conceicao, R. Silva, and R. Guerreiro, “Early Detection of Drowsiness Using a Neuro-fuzzy Detector,” Workshop on Biomedical Engineering 27 Setembro, pp. 82-86, Dec. 2002.
[Din98] D. F. Dinges, M. M. Mallis, G. M. Maislin and J. W. Powell, “Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and the Basis for Alertness Management,” Dept. Trans. Highway Safety, pub. 808 762, Apr. 1998.
[Dio03] J. J. de Dios and N. Garcia, “Face Detection Based on A New Color Space ,” Proc. of Int’l Conf. on Image Processing, vol. 3, pp. 909 -912, Sept. 2003.
[Dub82] D. Dubois and H. Prade, “Integration of Fuzzy Mappings: Part 2; Integration of Fuzzy Intervals: Part 3,” Fuzzy Sets Systems 8, pp. 105-116, 225-233, 1982.
[Esk03] A. Eskandarian and R. Sayed, “Driving Simulator Experiment: Detecting Driver Fatigue by Monitoring Eye and Steering Activity,” Proc. of NDIA Symp. on Annual Intelligent Vehicles Systems, Jun. 2003.
[Fan03] J. Fang and G. Qiu, “Human Face Detection Using Angular Radial Transform and Support Vector Machines,” Proc. Int’l Conf. on Image Processing, vol. 1, pp. I-669-72, Sep. 2003.
[Fid07] F. Fidan, M. Unlü, M. Sezer, and Z. Kara, “Relation Between Traffic Accidents and Sleep Apnea Syndrome in Truck Drivers,” Tuberk Toraks, 55(3), pp. 278-284, 2007.
[Gan05] P. H. Gander, N. S. Marshall, R. B. Harris, and P. Reid, “Sleep, Sleepiness and Motor Vehicle Accidents: A National Survey,” Journal of Public Health, 29(1), pp. 16-21, Feb. 2005.
[Gra98] R. Grace, V. E. Byrne, D. M. Bierman, J. M. Legrand, D. Gricourt, B. K. Davis, J. J. Staszewski, and B. Carnahan, “A Drowsy Driver Detection System for Heavy Vehicles,” Proc. of AIAA/IEEE/SAE 17th DASC Conf. on Digital Avionics Systems, vol. 2, pp. I36/1 -I36/8, Oct. 1998.
[Häk00] H. Häkkänen and H. Summala, “Sleepiness at Work Among Commercial Truck Drivers,” Sleep, 23(1), pp. 49-57, Feb. 2000.
[Hay02] T. Hayami, K. Matsunaga, K. Shidoji, and Y. Matsuki, “Detecting Drowsiness while Driving by Measuring Eye Movement - a Pilot Study,” Proc. of IEEE 5th Int'l Conf. on Intelligent Transportation Systems, pp. 156-161, Sep. 2002.
[Hor04] W. B. Horng, C. Y. Chen, Y. Chang, and C. Hai Fan, “Driver Fatigue Detection Based on Eye Tracking and Dynamic, Template Matching,” Proc. of IEEE Int’l Conf. on Networking, Sensing and Control, vol. 1, pp. 7-12, Mar. 2004.
[Hsu02] R.L. Hsu, A. M. Mottaleb and A. K. Jain, “Face Detection in Color Images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, May 2002.
[Ike03] O. Ikeda, “Segmentation of Faces in Video Footage Using HSV Color for Face Detection and Image Retrieval,” Proc. of 2003 Int’l Conf. on Image Processing, vol. 3, pp. III-913-6, Sep. 2003.
[Ito02] T. Ito, S. Mita, K. Kozuka, T. Nakano, and S. Yamamoto, “Driver Blink Measurement by the Motion Picture Processing and its Application to Drowsiness Detection,” Porc. of IEEE 5th Int’l Conf. on Intelligent Transportation Systems, pp. 168-173, Sep. 2002.
[Ji04] Q. Ji, Z. Zhu, and P. Lan, “Real-time Nonintrusive Monitoring and Prediction of Driver Fatigue,” IEEE Trans. Vehicular Technology, vol. 53, no. 4, pp. 1052-1068, Jul. 2004.
[Lie04] M. Lievin and F. Luthon, “Nonlinear Color Space and Spatiotemporal MRF for Hierarchical Segmentation of Face Features in Video,” IEEE Trans. on Image Processing, vol. 13, no. 1, pp. 63-71, Jan. 2004.
[Lui06] M. B. Luis, J. Nuevo, and M. A. Sotelo, “Real-Time System for Monitoring Driver Vigilance,” IEEE Trans. on Intelligent Transportation Systems, vol. 7, no. 1, pp. 63-77, 2006.
[Mit03] Y. Mitsukura, H. Takimoto, M. Fukumi and N. Akamatsu, “Face Detection and Emotional Extraction System Using Double Structure Neural Network,” Proc. of the International Joint Conference on Neural Networks, vol. 2, pp. 1253 -1257, Jul. 2003.
[Mor96] C. Moreno, M. Gou, E. K. Boukas and C. L. Nadeau, “Transparent Driver’s Drowsiness Detector: An Approach.” Montréal: Centre de Recherche sur les Transports, 1996.
[Ohn02] T. Ohno, N. Mukawa, and A. Yoshikawa, “Freegaze: A Gaze Tracking System for Everyday Gaze Interaction,” Eye Tracking Research Applications Symposium, New Orleans, LA, pp. 25-27, 2002.
[Pop03] J. C. Popieul, P. Simon and P. Loslever, “Using Driver's Head Movements Evolution as a Drowsiness Indicator,” Proc. of IEEE on Intelligent Vehicles Symposium, pp. 616 -621, Jun. 2003.
[Poy96] C. A. Poynton, A Technical Introduction to Digital Video, John Wiley & Sons, INC, New York, 1996.
[Sab96] E. Saber and A. M. Tekalp, “Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry-based Cost Functions,” Proc. of the 13th Int’l Conf. on Pattern Recognition, vol. 3, pp. 654-658, Aug. 1996.
[Sab05] S. Sabbagh-Ehrlich, L. Friedman, E. D. Richter, “Working Conditions and Fatigue in Professional Truck Drivers at Israeli Ports,” Inj Prev., 11(2), pp. 110-114, Apr. 2005.
[Sag99] F. Sagberg, “Road Accidents Caused by Drivers Falling Asleep,” Accident Analysis and Prevention, 31(6), pp. 639-649, Nov. 1999.
[Sai92] S. Saito, “Does Fatigue Exist in a Quantitative of Eye Movement,” Ergonomics, vol. 35, pp. 607-615, 1992.
[Smi00] P. Smith, M. Shah, and N. da Vitoria Lobo, “Monitoring Head/Eye Motion for Driver Alertness with One Camera,” Proc. of 15th Int'l Conf on Pattern Recognition, vol. 4, pp. 636-642, Sep. 2000.
[Sug77] M. Sugeno, “Fuzzy Measure and Fuzzy Integrals: A Survey,” Fuzzy Automatic and Decision Processes, Eds. By M. M. Gupta, G. N. Saridis, and B. R. Gaines, North Holland, Amsterdam, pp. 89-102, 1977.
[Tak03] I. Takai, K. Yamamoto, K. Kato, K. Yamada, and M. Andoh, “Robust Detection Method of the Driver’s Face and Eye Region for Driving Support System,” Proc. of 16th Int’l Conf. on Vision Interface, Halifax, Canada, pp. 148 -153, Jun. 2003.
[Uen94] H. Ueno, M. Kaneda, and M. Tsukino, “Development of Drowsiness Detection System,” Proc. of Conf. on Vehicle Navigation and Information Systems, pp. 15-20, Aug. 1994.
[Vie06] C. A. Viegas and H. W. de Oliveira, “Prevalence of Risk Factors for Obstructive Sleep Apnea Syndrome in Interstate Bus Drivers,” Journal of Bras Pneumol, 32(2), pp. 144-149, 2006.
[Vuc02] A. Vuckovic, V. Radivojevic, A. C. N. Chen, and D. Popovic, “Automatic Recognition of Alertness and Drowsiness from EEG by An Artificial Neural Network,” Journal of Medical Engineering and Physics, vol. 24, no. 5, pp. 349 -360, June 2002.
[Wan92] Z. Wang and G. J. Klir, Fuzzy Measure Theory, Plenum Press, New York, 1992.
[Wel95] G. Welch and B. Gary, “An Introduction to the Kalman Filter,” University of North Carolina at Chapel Hill, Dept. of Computer Sci., TR95-041, 1995.
[Wil02] G. Wilson, “An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures,” International Journal of Aviation Psychology, vol. 12, no. 1, pp. 3-18, 2002.
[Wu04] Y. Wu, H. Liu and H. Zha “A New Method of Detecting Human Eyelids Based on Deformable Templates,” IEEE Int’l Conf. on Systems, Man and Cybernetics, vol. 1, pp. 604 -609, Oct. 2004.
[Zim91] H. J. Zimmermann, Fuzzy Set Theory and Its Applications, 2nd Ed., Kluwer Academic, Boston, 1991.