簡易檢索 / 詳目顯示

研究生: 謝俊毅
Sie, Jyun-Yi
論文名稱: 基於深度強化學習之自動充電系統電腦視覺與機械手臂控制系統發展
Development of Computer Vision and Robotic Arm Control System for an Autonomous Charging System Based on Deep Reinforcement Learning
指導教授: 陳瑄易
Chen, Syuan-Yi
口試委員: 陳瑄易
Chen, Syuan-Yi
李俊賢
Lee, Jin-Shyan
陳永耀
Chen, Yung-Yao
鄭穎仁
Chen, Ying-Jen
口試日期: 2024/10/16
學位類別: 碩士
Master
系所名稱: 電機工程學系
Department of Electrical Engineering
論文出版年: 2024
畢業學年度: 113
語文別: 中文
論文頁數: 104
中文關鍵詞: 自動充電系統電動車自主移動式充電機器人深度強化式學習機械手臂視覺系統
英文關鍵詞: Autonomous Charging Service System, Electric Vehicle, Autonomous Mobile Robot, Deep Reinforcement Learning, Robotic Arm, Vision System
研究方法: 實驗設計法
DOI URL: http://doi.org/10.6345/NTNU202401957
論文種類: 學術論文
相關次數: 點閱:13下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 隨著科技與電動車技術的蓬勃發展,近年來電動車獲得更多關注,建設更為便利的充電設施已然是電動車主們一大需求。因此本論文提出一主僕式架構之自動充電系統,提供自動充電服務,整體架構由自主移動式充電機器人與電源拖車組成,並搭配本研究設計之充電行為流程、充電插座姿態辨識與使用深度強化學習(Deep Reinforcement Learning, RL)於機械手臂運動控制,最終實現可提供充電服務之自動充電系統。
    本論文於充電插座自動辨識上使用Yolo、PnP等電腦視覺技術,並搭配類神經網路進行座標補償。在機械手臂控制策略上使用深度強化學習之深度確定策略梯度(Deep Deterministic Policy Gradient, DDPG)與近端策略最佳化(Proximal Policy Optimization, PPO)進行模擬實驗,並最終使用PPO搭配PID補償器進行實作,此設計架構可有效補償PPO輸出之穩態誤差,在機械手臂運動控制方面,滿足系統執行自動充電服務的需求。

    With the rapid advancement of technology and electric vehicle (EV) technology, EVs have gained significant attention in recent years. The demand for more convenient charging facilities has become a major requirement for EV owners. Therefore, this thesis proposes an automated charging system based on a master-slave architecture, providing automatic charging services. The overall structure consists of a mobile charging robot and a power trailer, combined with a charging behavior process, charging socket posture recognition, and deep reinforcement learning (RL) for motion control of the robotic arm, ultimately achieving an automated charging system that offers charging services.
    In this thesis, automatic recognition of the charging socket is performed using computer vision techniques such as Yolo and PnP, along with coordinate compensation through neural networks. For the robotic arm control strategy, Deep Reinforcement Learning algorithms like Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) were used for simulation experiments. PPO, combined with a PID compensator, was finally implemented. This design framework effectively compensates for steady-state errors in the PPO output, fulfilling the system’s requirements for executing automated charging services in robotic arm motion control.

    第一章 緒論 1 1.1 研究背景與動機 1 1.2 文獻探討 9 1.2.1 充電插座視覺定位 9 1.2.2 強化學習應用於機械手臂運動控制 12 1.3 研究目的 15 1.4 研究架構 16 第二章 自動充電系統之系統架構與設計 17 2.1 自動充電系統 17 2.2 自主移動式充電機器人系統架構設計 19 2.2.1 自主移動平台 19 2.2.2 機械手臂模組 22 2.2.3 相機模組整合分離式充電槍機構設計 27 2.2.4 控制核心 29 2.2.5 電源模組 31 2.3 電源拖車系統架構設計 33 2.3.1 電源拖車硬體設備說明 34 2.3.2 電源拖車各式機構設計與實作 38 第三章 自動充電系統之行為模式設計 40 3.1 充電行為設計 40 3.2 自主移動平台功能開發 43 3.2.1 混和導航控制流程開發 45 3.2.2 返程行為設計 47 3.3 自動充電系統充電流程設計 48 第四章 自主移動式充電機器人之視覺系統設計 50 4.1 Yolo物件辨識與影像處理 51 4.2 Aruco標籤定位辨識 53 4.3 基於類神經網路之充電插座姿態補償 54 4.4 相機與自主移動式充電機器人之座標轉換 55 第五章 基於強化學習之機械手臂運動控制 57 5.1 狀態價值函數與動作價值函數 57 5.2 策略梯度 58 5.3 深度確定策略梯度演算法 59 5.4 近端策略最佳化演算法 61 5.5 狀態、動作與獎勵值設計 64 5.6 基於比例、積分、微分演算法之強化學習動作補償 65 第六章 實驗結果與討論 67 6.1 自動充電系統於場域運作測試 67 6.1.1 自主移動式充電機器人接取電源拖車執行流程與結果 69 6.1.2 充電服務執行流程與結果 73 6.1.3 自主移動式充電機器人返程執行流程與結果 77 6.2 視覺系統運作之實驗結果 79 6.2.1 充電插座辨識之實驗結果 79 6.2.2 充電插座姿態補償之實驗結果 81 6.2.3 充電插座姿態捕捉之實驗結果 88 6.3 強化學習運作測試 89 6.3.1 PPO強化學習暨PID補償於機械手臂控制模擬 91 6.3.2 PPO強化學習暨PID補償於機械手臂控制實作 94 第七章 結論及未來展望 98 7.1 結論 98 7.2 未來展望 98 參考文獻 99

    [1] Y. Liu, H. Chen, Y. Li, J. Gao, K. Dave, J. Chen, and R. Tu, “Exhaust and non-exhaust emissions from conventional and electric vehicles: A comparison of monetary impact values,” Journal of Cleaner Production, vol. 331, p. 129965, 2022.
    [2] A. Albatayneh, M. N. Assaf, D. Alterman, and M. Jaradat, “Comparison of the overall energy efficiency for internal combustion engine vehicles and electric vehicles,” Rigas Tehniskas Universitates Zinatniskie Raksti, vol. 24, no. 1, pp. 669-680, 2020.
    [3] J. A. Sanguesa, V. Torres-Sanz, P. Garrido, F. J. Martinez, and J. M. Marquez-Barja, “A review on electric vehicles: Technologies and challenges,” Smart Cities, vol. 4, no. 1, pp. 372-404, Mar. 2021.
    [4] Z. A. Lashari, J. Ko, and J. Jang, “Consumers’ intention to purchase electric vehicles: Influences of user attitude and perception,” Sustainability, vol. 13, no. 12, p. 6778, 2021.
    [5] “Global EV Outlook 2023 Policy developments” Available: https://www.iea.org/reports/global-ev-outlook-2023/policy-developments
    [6] S. Afshar, P. Macedo, F. Mohamed, and V. Disfani, “Mobile charging stations for electric vehicles—A review,” Renewable and Sustainable Energy Reviews, vol. 152, 2021.
    [7] T. Chen, X. P. Zhang, J. Wang, J. Li, C. Wu, M. Hu, and H. Bian, “A review on electric vehicle charging infrastructure development in the UK,” Journal of Modern Power Systems and Clean Energy, vol. 8, no. 2, pp. 193-205, 2020.
    [8] “KUKA Charging Assistant Charging assistant for electric vehicles,” Available: https://ifdesign.com/en/winner-ranking/project/kuka-charging-assistant/282642
    [9] “Autonomous charging of electric vehicles with robotics: How it works,” Available: https://www.rocsys.com/newsroom/autonomous-charging-of-electric-vehicles-with-robotics-how-it-works/
    [10] “Electric Vehicle charging with an autonomous robot powered by force sensing|BotaSystems,”Available: https://youtu.be/SrtU32lWu9s?si=dYUT1YE2IzZ2PQSz
    [11] “Hyundai Motor Group Shows Newly Developed Automatic Charging Robot for Electric Vehicles,” Available: https://www.hyundai.com/worldwide/en/newsroom/detail/hyundai-motor-group-shows-newly-developed-automatic-charging-robot-for-electric-vehicles-0000000192
    [12] “Meet 'CARL' – the Autonomous EV Charging Robot,” Available: https://youtu.be/OOsFz5c8WaU?si=QWXJUPeREmYgNdB3
    [13] “Autev Launches Autonomous Charging Robot for EVs,” Available: https://youtu.be/mzUpWVkdoas?si=3ohM_EIBHMYO2EjM
    [14] “EVAR Parky (English version),” Available: https://youtu.be/PUOG1_MGlts?si=utWe3NksCZ5_5DQL
    [15] “E-HERO移動式充電機器人” Available: https://www.chander.com.tw/e-hero5.html
    [16] “汽車充電樁規格懶人包:充電樁種類&架構一篇看懂!,” Available: https://www.zerovatech.com/zh-hant/technical-knowledge/ev-charging-connector-types/
    [17] “電動車充電樁的規格、安裝、充電時間及常見問題,完整解析!,” Available: https://www.fubon.com/insurance/blog/information/charging-station.html
    [18] “【電動車充電原理】2大充電樁類型規格要了解,快充體驗超方便!,” Available: https://www.zerovatech.com/zh-hant/technical-knowledge/main-types-of-ev-charger-and-application/
    [19] “電動車充電系統-介面,” Available: https://www.bsmi.gov.tw/wSite/
    [20] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
    [21] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proceedings of the 2011 International Conference on Computer Vision (ICCV), pp. 2564-2571, 2011.
    [22] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580-587, 2014.
    [23] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
    [24] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in Computer Vision–ECCV 2016: 14th European Conference on Computer Vision, vol. 14, pp. 21-37, 2016.
    [25] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, 2016.
    [26] H. Perez and J. H. Tah, “Towards automated measurement of as-built components using computer vision,” Sensors, vol. 23, no. 16, p. 7110, 2023.
    [27] “Perspective-n-Point (PnP) pose computation,” Available: https://docs.opencv.org/4.x/d5/d1f/calib3d_solvePnP.html
    [28] M. Pan, C. Sun, J. Liu, and Y. Wang, “Automatic recognition and location system for electric vehicle charging port in complex environment,” IET Image Processing, vol. 14, no. 10, pp. 2263-2272, Oct. 2020.
    [29] P. Zhao, X. Chen, S. Tang, Y. Xu, M. Yu, and P. Xu, “Fast recognition and localization of electric vehicle charging socket based on deep learning and affine correction,” in 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2140-2145, 2022.
    [30] H. H. Huang, C. K. Cheng, Y. H. Chen, and H. Y. Tsai, “The robotic arm velocity planning based on reinforcement learning,” International Journal of Precision Engineering and Manufacturing, vol. 24, no. 9, pp. 1707-1721, 2023.
    [31] A. V. Duka, “Neural network based inverse kinematics solution for trajectory tracking of a robotic arm,” Procedia Technology, vol. 12, pp. 20-27, 2014.
    [32] T. Zhang, M. Xiao, Y. B. Zou, J. D. Xiao, and S. Y. Chen, “Robotic curved surface tracking with a neural network for angle identification and constant force control based on reinforcement learning,” International Journal of Precision Engineering and Manufacturing, vol. 21, pp. 869-882, 2020.
    [33] Y. Matsuo, Y. LeCun, M. Sahani, D. Precup, D. Silver, M. Sugiyama, ... and J. Morimoto, “Deep learning, reinforcement learning, and world models,” Neural Networks, vol. 152, pp. 267-275, 2022.
    [34] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE Transactions on Systems, Man, and Cybernetics, no. 5, pp. 834-846, 1983.
    [35] A. Romero, Y. Song, and D. Scaramuzza, “Actor-critic model predictive control,” in 2024 IEEE International Conference on Robotics and Automation (ICRA), May 2024, pp. 14777-14784. IEEE.
    [36] R. Liu, F. Nageotte, P. Zanne, M. de Mathelin, and B. Dresp-Langley, “Deep reinforcement learning for the control of robotic manipulation: A focused mini-review,” Robotics, vol. 10, no. 1, p. 22, 2021.
    [37] T. Zhang and H. Mo, “Reinforcement learning for robot research: A comprehensive review and open issues,” International Journal of Advanced Robotic Systems, vol. 18, no. 3, 2021.
    [38] M. Matulis and C. Harvey, “A robot arm digital twin utilizing reinforcement learning,” Computers & Graphics, vol. 95, pp. 106-114, 2021.
    [39] T. Lindner and A. Milecki, “Reinforcement learning-based algorithm to avoid obstacles by the anthropomorphic robotic arm,” Applied Sciences, vol. 12, no. 13, p. 6629, 2022.
    [40] “MODBUS APPLICATION PROTOCOL SPECIFICATION V1.1b3,” Available: https://www.modbus.org/
    [41] O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., & Walsh, J. (2020). Deep learning vs. traditional computer vision. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1 (pp. 128-144). Springer International Publishing.
    [42] C. Y. Wang, A. Bochkovskiy, and H. Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464-7475, 2023.
    [43] G. Terzakis and M. Lourakis, “A consistently fast and globally optimal solution to the perspective-n-point problem,” in Computer Vision–ECCV 2020: 16th European Conference on Computer Vision, vol. 16, pp. 478-494, 2020.
    [44] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280-2292, 2014.
    [45] J. Jiang, X. Luo, Q. Luo, L. Qiao, and M. Li, “An overview of hand-eye calibration,” The International Journal of Advanced Manufacturing Technology, vol. 119, no. 1, pp. 77-97, 2022.
    [46] I. Enebuse, M. Foo, B. S. K. K. Ibrahim, H. Ahmed, F. Supmak, and O. S. Eyobu, “A comparative review of hand-eye calibration techniques for vision guided robots,” IEEE Access, vol. 9, pp. 113143-113155, 2021.
    [47] T. P. Lillicrap, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
    [48] J. Schulman, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
    [49] J. Ren, Y. Lan, X. Xu, Y. Zhang, Q. Fang, and Y. Zeng, “Deep reinforcement learning using least‐squares truncated temporal‐difference,” CAAI Transactions on Intelligence Technology, vol. 9, no. 2, pp. 425-439, 2024.
    [50] R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 12, 1999.

    無法下載圖示 電子全文延後公開
    2029/10/29
    QR CODE