簡易檢索 / 詳目顯示

研究生: 張弘凱
Chang, Hung-Kai
論文名稱: 以CenterNet 演算法則及數位彎曲感測器為基礎的即時手勢辨識系統之研究
Real-time gesture recognition system based on CenterNet algorithm and digital flex sensor
指導教授: 黃文吉
Hwang, Wen-Jyi
口試委員: 張寶基 周賜福
口試日期: 2021/08/03
學位類別: 碩士
Master
系所名稱: 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年: 2021
畢業學年度: 109
語文別: 中文
論文頁數: 37
中文關鍵詞: 智慧手套穿戴式裝置手指手勢辨識系統數位彎曲感測器
英文關鍵詞: CenterNet, Sensor based
DOI URL: http://doi.org/10.6345/NTNU202101300
論文種類: 學術論文
相關次數: 點閱:109下載:7
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 本論文將實現以數位彎曲感測器作為手指手勢的即時辨識系統,包含硬體方面的智慧手套之設計與製作;以及軟體方面,使用CenterNet演算法則來將偵測出手勢並且辨識類別。從硬體到軟體,形成一個完整的即時手勢辨識系統。
    另外本論文也解決了當今以感測器的手勢辨識幾乎都不重視前背景分離,都是以辨識的正確率為主。本論文成功的使用CenterNet來將前背景訊號分離,而不須額外的觸發,解決了在時間序列中前景手勢與背景手勢難以分離的問題,實現即時的手勢辨識系統。

    第一章 緒論 1 1.1 研究背景與動機 1 1.2 研究目的 6 1.3 研究貢獻 7 第二章 基礎理論與技術背景 8 2.1 CenterNet概念 8 2.2 CenterNet架構及訓練 10 第三章 系統架構與實驗方法 13 3.1 智慧手套製作 13 3.1.1 電子元件探討 14 3.1.2 電路設計 16 3.1.3 智慧手套設計 18 3.2 手勢辨識系統流程 20 3.3 演算法則介紹 22 3.3.1 模型訓練與架構 22 3.3.2 資料後處理 25 第四章 實驗結果及討論 26 4.1 手指手勢介紹 26 4.2 手勢偵測 27 4.3 手勢辨識 32 第五章 結論 34 參考文獻 35

    [1] A. Amir, "A low power, fully event-based gesture recognition system," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
    [2] C. Ng and S. Ranganath, "Real-time gesture recognition system and application," Image and Vision computing, vol. 20, p. 993–1007, 2002.
    [3] S. Rautaray and A. Agrawal, "Vision based hand gesture recognition for human computer interaction: a survey," Artificial intelligence review, vol. 43, p. 1–54, 2015.
    [4] J. Suarez and R. R. Murphy, "Hand gesture recognition with depth images: A review," in 2012 IEEE RO-MAN: the 21st IEEE international symposium on robot and human interactive communication, 2012.
    [5] W. Cheng, "Jointly network: a network based on CNN and RBM for gesture recognition," Neural Computing and Applications, vol. 1, p. 309–323, 2019.
    [6] G. Li, "Dynamic gesture recognition in the internet of things," IEEE Access, vol. 7, p. 23713–23724, 2018.
    [7] K. Bhaskaran and Abhijith, "Smart gloves for hand gesture recognition: Sign language to speech conversion system," in 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA), IEEE, 2016, p. 1–6.
    [8] T. Chouhan, "Smart glove with gesture recognition ability for the hearing and speech impaired," in IEEE Global Humanitarian Technology Conference-South Asia Satellite (GHTC-SAS), IEEE, 2014. 2014, p. 105–110.
    [9] T. Schlömer, B. Poppinga, N. Henze and S. Boll, "Gesture recognition with a Wii controller," in Proceedings of the 2nd international conference on Tangible and embedded interaction, 2008. February.
    [10] J. Liu, "Accelerometer-based personalized gesture recognition and its applications," Pervasive and Mobile Computing, vol. 5, p. 657–675, 2009.
    [11] Z. Zhang, Tian, Zengshan and M. Zhou, "Latern: Dynamic continuous hand gesture recognition using FMCW radar sensor," IEEE Sensors Journal, vol. 18, p. 3278–3289, 2018.
    [12] Q. Wan, "Gesture recognition for smart home applications using portable radar sensors," IEEE, p. 6414–6417, 2014. 2014.
    [13] J. Qi, Intelligent human-computer interaction based on surface EMG gesture recognition, vol. 7, Ieee Access, 2019, p. 61378–61387.
    [14] A. Memo, Minto, Ludovico and P. Zanuttigh, Exploiting silhouette descriptors and synthetic data for hand gesture recognition, 2015.
    [15] A. Memo and P. Zanuttigh, "Head-mounted gesture controlled interface for human-computer interaction," Multimedia Tools and Applications, vol. 77, p. 27–53, 2018.
    [16] X. Lu, "MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection," in Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK: Springer International Publishing, August 23-28, 2020. 2020.
    [17] S. Ren, "Faster r-cnn: Towards real-time object detection with region proposal networks," in Advances in neural information processing systems, vol. 28, 2015, p. 91–99.
    [18] A. Bochkovskiy, Wang, Chien-Yao, H.-Y. Liao and Mark, "Yolov4: Optimal speed and accuracy of object detection," 2020.
    [19] W. Liu, "Single shot multibox detector," in European conference on computer vision, Cham, Springer, 2016, p. 21–37.
    [20] Hochreiter, Sepp and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, p. 1735–1780, 1997.
    [21] K. Cho, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," 2014.
    [22] J. Ducloux, "Accelerometer-based hand gesture recognition system for interaction in digital TV," in IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings. IEEE, 2014. 2014.
    [23] Lee, Daeha, Yoon, Hosub and J. Kim, "Continuous gesture recognition by using gesture spotting," in 16th International Conference on Control, Automation and Systems (ICCAS), IEEE, 2016. 2016, p. 1496–1498.
    [24] S. Agrawal, "Using mobile phones to write in air," in Proceedings of the 9th international conference on Mobile systems, applications, and services, 2011.
    [25] S. Zhang, "Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020.
    [26] X. Zhou, D. Wang and P. Krähenbühl, "Objects as points," 2019.
    [27] H. Law, J. Deng and Cornernet, "Detecting objects as paired keypoints," in Proceedings of the European conference on computer vision (ECCV), 2018.
    [28] K. He, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
    [29] Yu and Fisher, "Deep layer aggregation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
    [30] T.-Y. Lin, "Focal loss for dense object detection," in Proceedings of the IEEE international conference on computer vision, 2017.
    [31] W.-C. Chuang, "Continuous finger gesture recognition based on flex sensors," Sensors, vol. 19, p. 3986, 2019.
    [32] Pathak, Vishal, Mongia, Sushant and G. Chitranshi, "A framework for hand gesture recognition based on fusion of Flex, Contact and accelerometer sensor," in Third International Conference on Image Information Processing (ICIIP), IEEE, 2015. 2015, p. 312–319.
    [33] O. Nisar, "Performance optimization of a Flex sensor based glove for hand gestures recognition and translation," International Journal of Engineering Research & Technology, vol. 3, p. 1565–1570, 2014.
    [34] O. Ronneberger, Fischer, Philipp and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, Cham, Springer, 2015, p. 234–241.
    [35] O. Köpüklü, "Real-time hand gesture detection and classification using convolutional neural networks," in 14th IEEE International Conference on Automatic Face & Gesture Recognition, IEEE, 2019. 2019. 2019, p. 1–8.
    [36] C. Zhu and W. Sheng, "Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living," in IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 3, 2011, p. 569–573.
    [37] Y. Yu, Time series outlier detection based on sliding window prediction. Mathematical problems in Engineering, 2014. 2014.

    下載圖示
    QR CODE