簡易檢索 / 詳目顯示

研究生: 楊祐銓
YANG, YU-CHUAN
論文名稱: 應用強化式學習策略之分數階比例積分微分控制於X-Y-Y棒狀線性馬達定位平台
Fractional-Order PID Control for a X-Y-Y Tubular Linear Motors-based Positioning Stage Using Reinforcement Learning Strategy
指導教授: 陳瑄易
Chen, Syuan-Yi
口試委員: 陳瑄易
Chen, Syuan-Yi
談光雄
Tan, Kuang-Hsiung
藍建武
Lan, Chien-Wu
李政道
Lee, Jeng-Dao
口試日期: 2024/01/10
學位類別: 碩士
Master
系所名稱: 電機工程學系
Department of Electrical Engineering
論文出版年: 2024
畢業學年度: 112
語文別: 中文
論文頁數: 134
中文關鍵詞: 棒狀線性馬達分數階微積分PID控制器強化式學習Q學習深度學習類神經網路
英文關鍵詞: Tubular Linear Motors, fractional calculus, PID controller, reinforcement learning control strategy, Q learning, deep learning, neural network
研究方法: 實驗設計法準實驗設計法主題分析比較研究觀察研究
DOI URL: http://doi.org/10.6345/NTNU202401088
論文種類: 學術論文
相關次數: 點閱:108下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 目錄 誌謝 i 摘要 ii ABSTRACT iii 目錄 v 表目錄 viii 圖目錄 x 第一章 緒論 1 1.1 研究背景與動機 1 1.2 文獻探討 2 1.3 研究目的 6 1.4 研究方法 7 1.5 研究架構 7 第二章 X-Y-Y棒狀線性馬達實驗平台介紹 9 2.1 棒狀線性馬達結構及運作原理 9 2.2 棒狀線性馬達模型分析 12 第三章 應用分數階PID控制於棒狀線性馬達系統 15 3.1 PID控制系統 15 3.1.1 比例控制器(Proportional Controller) 15 3.1.2 積分控制器(Integral Controller) 16 3.1.3 微分控制器(Derivative Controller) 16 3.2 分數階PID控制設計 17 3.2.1 分數階定義 17 3.2.2 離散分數階近似 18 第四章 應用強化式學習之最佳化分數階PID控制設計 21 4.1 強化式學習控制策略 21 4.1.1 馬爾可夫決策 22 4.1.2 獎勵值設計 25 4.1.3 動作策略 26 4.2 強化式學習Q學習控制策略 27 4.3 深度Q學習DQN控制策略 31 4.3.1 目標網路 33 4.3.2 經驗回放策略 34 4.3.3 DQN-FOPID控制設計 35 第五章 模擬結果與討論 37 5.1 X-Y-Y棒狀線性馬達定位控制模擬 37 5.2 FOPID模擬結果 41 5.3 QL-FOPID模擬結果 47 5.4 DQN-FOPID模擬結果 61 5.5 結果討論 74 第六章 實驗結果與討論 81 6.1 實驗設置 81 6.2 FOPID實作結果 86 6.3 QL-FOPID實作結果 92 6.4 DQN-FOPID實作結果 105 6.5 結果討論 118 第七章 結論與未來展望 125 7.1 結論 125 7.2 未來展望 126 參考文獻 127 自傳 132 學術成就 134

    [1] C. Wen, J. Liu, W. Wang, J. Liu, Z. Zhao, and J. Liu, “Research on improved permanent magnet linear synchronous motor for direct-drive application,” IEEE Transactions on Magnetics, vol. 55, no. 10, pp. 1-7, 2019.
    [2] 黃敬鈞,棒狀線性馬達之設計與製作,國立聯合大學機械工程學系碩士班碩士論文,苗栗縣,2016。
    [3] 楊振煒,基於改良式蜂群演算法之最佳化雙自由度 PID控制器於棒狀線性馬達運動控制之應用,國立臺北科技大學自動化科技研究所碩士論文,台北市,2018。
    [4] 魏佑鈞,基於交叉耦合分數階自抗擾控制之X-Y-Y棒狀線性馬達定位平台,國立臺灣師範大學電機工程學系碩士論文,台北市,2022。
    [5] J. Zhang, and L. Guo, “Theory and design of PID controller for nonlinear uncertain systems,” IEEE Control Systems Letters, vol. 3, no. 3, pp. 643-648, 2019.
    [6] C.-H. Hsu, “Fractional order PID control for reduction of vibration and noise on induction motor,” IEEE Transactions on Magnetics, vol. 55, no. 11, pp. 1-7, 2019.
    [7] A. Mahdy, A. Shaheen, R. El-Sehiemy, A. Ginidi, and S. F. Al-Gahtani, “Single- and Multi-Objective Optimization Frameworks of Shape Design of Tubular Linear Synchronous Motor,” Energies, vol. 16, no. 5, pp. 2409, 2023.
    [8] Z. Li, S. Zhou, Y. Xiao, and L. Wang, “Sensorless vector control of permanent magnet synchronous linear motor based on self-adaptive super-twisting sliding mode controller,” IEEE Access, vol. 7, pp. 44998-45011, 2019.
    [9] G. Brasó, and L. Leal-Taixé, “Learning a neural solver for multiple object tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6247-6257.
    [10] Y. Liu, W. Sun, and H. Gao, “High precision robust control for periodic tasks of linear motor via B-spline wavelet neural network observer,” IEEE Transactions on Industrial Electronics, vol. 69, no. 8, pp. 8255-8263, 2021.
    [11] R. P. Borase, D. Maghade, S. Sondkar, and S. Pawar, “A review of PID control, tuning methods and applications,” International Journal of Dynamics and Control, vol. 9, pp. 818-827, 2021.
    [12] A. U. Rehman, M. U. Khan, M. T. Rehman, W. Shehzad, S. Zaman, and M. W. Khan, “Mathematical Modelling and Implementation of 2DOF Standard, Parallel & Series PID Controllers,” in 2021 6th International Multi-Topic ICT Conference (IMTIC), 2021, pp. 1-6.
    [13] E. Guney, and M. Demir, “A comparative velocity control study of permanent magnet tubular linear DC motor by using PID and fuzzy-PID controllers,” in 2017 International Conference on Control, Automation and Diagnosis (ICCAD), 2017, pp. 390-395.
    [14] L. Wang, PID control system design and automatic tuning using MATLAB/Simulink: John Wiley & Sons, 2020.
    [15] A. Mughees, and S. A. Mohsin, “Design and control of magnetic levitation system by optimizing fractional order PID controller using ant colony optimization algorithm,” IEEE Access, vol. 8, pp. 116704-116723, 2020.
    [16] H. Sun, A. Chang, Y. Zhang, and W. Chen, “A review on variable-order fractional differential equations: mathematical foundations, physical models, numerical methods and applications,” Fractional Calculus and Applied Analysis, vol. 22, no. 1, pp. 27-59, 2019.
    [17] A. Tepljakov, B. B. Alagoz, C. Yeroglu, E. Gonzalez, S. H. HosseinNia, and E. Petlenkov, “FOPID controllers and their industrial applications: A survey of recent results,” IFAC-PapersOnLine, vol. 51, no. 4, pp. 25-30, 2018.
    [18] M. Yaghi, and M. Ö. Efe, “H2/H-Neural-Based FOPID Controller Applied for Radar-Guided Missile,” IEEE Transactions on Industrial Electronics, vol. 67, no. 6, pp. 4806-4814, 2019.
    [19] R. M. Batyha, “Optimal design of fractional-order PID controllers using bacterial foraging optimization algorithm,” Int. J. Adv. Soft Comput. Appl, vol. 13, pp. 136-149, 2021.
    [20] M. A. George, D. V. Kamat, and C. P. Kurian, “Electronically tunable ACO based fuzzy FOPID controller for effective speed control of electric vehicle,” IEEE Access, vol. 9, pp. 73392-73412, 2021.
    [21] W. Fu, and Q. Lu, “Multiobjective optimal control of FOPID controller for hydraulic turbine governing systems based on reinforced multiobjective harris hawks optimization coupling with hybrid strategies,” Complexity, vol. 2020, pp. 1-17, 2020.
    [22] A. Sikander, P. Thakur, R. C. Bansal, and S. Rajasekar, “A novel technique to design cuckoo search based FOPID controller for AVR in power systems,” Computers & Electrical Engineering, vol. 70, pp. 261-274, 2018.
    [23] M. Guerrero-Luis, F. Valdez, and O. Castillo, “A review on the cuckoo search algorithm,” Fuzzy Logic Hybrid Extensions of Neural and Optimization Algorithms: Theory and Applications, pp. 113-124, 2021.
    [24] T. M. Shami, A. A. El-Saleh, M. Alswaitti, Q. Al-Tashi, M. A. Summakieh, and S. Mirjalili, “Particle swarm optimization: A comprehensive survey,” IEEE Access, vol. 10, pp. 10031-10061, 2022.
    [25] A.-T. Nguyen, T. Taniguchi, L. Eciolaza, V. Campos, R. Palhares, and M. Sugeno, “Fuzzy control systems: Past, present and future,” IEEE Computational Intelligence Magazine, vol. 14, no. 1, pp. 56-68, 2019.
    [26] B. Kiumarsi, K. G. Vamvoudakis, H. Modares, and F. L. Lewis, “Optimal and autonomous control using reinforcement learning: A survey,” IEEE transactions on neural networks and learning systems, vol. 29, no. 6, pp. 2042-2062, 2017.
    [27] D. Bertsekas, Abstract dynamic programming: Athena Scientific, 2022.
    [28] S. Zou, T. Xu, and Y. Liang, “Finite-sample analysis for sarsa with linear function approximation,” Advances in neural information processing systems, vol. 32, 2019.
    [29] C. J. Watkins, and P. Dayan, “Q-learning,” Machine learning, vol. 8, pp. 279-292, 1992.
    [30] J. Fan, Z. Wang, Y. Xie, and Z. Yang, “A theoretical analysis of deep Q-learning,” in Learning for dynamics and control, 2020, pp. 486-489.
    [31] E. Fouché, J. Komiyama, and K. Böhm, “Scaling multi-armed bandit algorithms,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 1449-1459.
    [32] E. Altman, Constrained Markov decision processes: Routledge, 2021.
    [33] Z. Ding, Y. Huang, H. Yuan, and H. Dong, “Introduction to reinforcement learning,” Deep reinforcement learning: fundamentals, research and applications, pp. 47-123, 2020.
    [34] W. Dabney, G. Ostrovski, and A. Barreto, “Temporally-extended {epsilon}-greedy exploration,” arXiv preprint arXiv:2006.01782, 2020.
    [35] I. Carlucho, M. De Paula, S. A. Villar, and G. G. Acosta, “Incremental Q-learning strategy for adaptive PID control of mobile robots,” Expert Systems with Applications, vol. 80, pp. 183-199, 2017.
    [36] Y. Shan, B. Zheng, L. Chen, L. Chen, and D. Chen, “A reinforcement learning-based adaptive path tracking approach for autonomous driving,” IEEE Transactions on Vehicular Technology, vol. 69, no. 10, pp. 10581-10595, 2020.
    [37] H. F. Bassani, R. A. Delgado, J. N. d. O. L. Junior, H. R. Medeiros, P. H. Braga, M. G. Machado, L. H. Santos, and A. Tapp, “A framework for studying reinforcement learning and sim-to-real in robot soccer,” arXiv preprint arXiv:2008.12624, 2020.
    [38] J. F. Hernandez-Garcia, and R. S. Sutton, “Understanding multi-step deep reinforcement learning: A systematic study of the DQN target,” arXiv preprint arXiv:1901.07510, 2019.
    [39] I. Boldea, L. N. Tutelea, W. Xu, and M. Pucci, “Linear electric machines, drives, and MAGLEVs: An overview,” IEEE Transactions on Industrial Electronics, vol. 65, no. 9, pp. 7504-7515, 2017.
    [40] M. Rahmani, A. Darabi, and F. P. Deylami, “Impact of the Stator Coil Pitch on Acoustic Noise and Vibration of Squirrel Cage Induction Motors,” IEEE Transactions on Energy Conversion, 2023.
    [41] W. Ma, L. Xi, Y. Zhang, Z. He, X. Zhang, and Z. Cai, “Design and comparison of linear induction motor and superconducting linear synchronous motor for electromagnetic launch,” in 2021 13th International Symposium on Linear Drives for Industry Applications (LDIA), 2021, pp. 1-5.
    [42] D. Li, Y. Zhang, Y. Jing, D. Xie, and C.-S. Koh, “Core Loss and Deformation Computation in Permanent Magnet Linear Motors Considering the Effect of Stress on Magnetic and Magnetostrictive Properties,” in 2021 13th International Symposium on Linear Drives for Industry Applications (LDIA), 2021, pp. 1-5.
    [43] K. Hollaus, J. Schöberl, and M. Schöbinger, “MSFEM and MOR to minimize the computational costs of nonlinear eddy-current problems in laminated iron cores,” IEEE Transactions on Magnetics, vol. 56, no. 2, pp. 1-4, 2020.
    [44] X. Li, W. Xu, C. Ye, and I. Boldea, “Comparative study of transversal-flux permanent-magnetic linear oscillatory machines for compressor,” IEEE Transactions on Industrial Electronics, vol. 65, no. 9, pp. 7437-7446, 2017.
    [45] A. Boduroglu, Y. Demir, B. Cumhur, and M. Aydin, “A novel track structure of double-sided linear PM synchronous motor for low cost and high force density applications,” IEEE Transactions on Magnetics, vol. 57, no. 2, pp. 1-5, 2020.
    [46] Z. Pei, J. Zhao, J. Song, K. Zong, Z. He, and Y. Zhou, “Temperature field calculation and water-cooling structure design of coreless permanent magnet synchronous linear motor,” IEEE Transactions on Industrial Electronics, vol. 68, no. 2, pp. 1065-1076, 2020.
    [47] H. Chen, and Z. Li, “Flux characteristics analysis of single-phase tubular permanent magnet linear motor,” IEEE Transactions on Applied Superconductivity, vol. 28, no. 3, pp. 1-6, 2017.
    [48] Q. Tan, X. Huang, L. Li, and M. Wang, “Analysis of flux linkage and detent force for a modular tubular permanent magnet synchronous linear motor with large slots,” IEEE Transactions on Energy Conversion, vol. 34, no. 3, pp. 1532-1541, 2019.
    [49] "FileZilla FTP 伺服器," https://filezilla-project.org/.
    [50] "IMP-2 Intelligent Motion control Platform, 智慧型運動控制平台," https://www.epcio.com.tw/product/IMP-2.aspx.
    [51] D1系列驅動器. https://www.hiwinmikro.tw/zh/product/controller-drive/drive-d1.
    [52] "雙軸定位平台," https://www.hiwinmikro.tw/zh/product/linear-motor-system/dual-axis-standard.
    [53] "棒狀線性馬達," https://www.hiwinmikro.tw/zh/product/linear-motor/tubular-motors.
    [54] 楊孟晨,基於最佳化分數階模糊PID控制之X-Y音圈馬達定位平台,國立臺灣師範大學電機工程學系碩士論文,台北市,2019。
    [55] S.-W. Seo, and H. H. Choi, “Digital implementation of fractional order PID-type controller for boost DC–DC converter,” IEEE Access, vol. 7, pp. 142652-142662, 2019.
    [56] M. S. Ayas, and E. Sahin, “FOPID controller with fractional filter for an automatic voltage regulator,” Computers & Electrical Engineering, vol. 90, pp. 106895, 2021.
    [57] W. Zhenbin, W. Zhenlei, C. Guangyi, and Z. Xinjian, “Digital implementation of fractional order PID controller and its application,” Journal of Systems Engineering and Electronics, vol. 16, no. 1, pp. 116-122, 2005.
    [58] 鄭珮彣,基於強化式學習應用於旋轉式倒立單擺系統之控制器設計,國立陽明交通大學電控工程研究所碩士論文,新竹市,2021。
    [59] T. Doan, S. Maguluri, and J. Romberg, “Finite-time analysis of distributed TD (0) with linear function approximation on multi-agent reinforcement learning,” in International Conference on Machine Learning, 2019, pp. 1626-1635.
    [60] Y. Rioual, J. Laurent, and J.-P. Diguet, “Reinforcement-learning approach guidelines for energy management,” Journal of Low Power Electronics, vol. 15, no. 3, pp. 283-293, 2019.
    [61] Z. Chen, H. Hu, Y. Wu, R. Xiao, J. Shen, and Y. Liu, “Energy management for a power-split plug-in hybrid electric vehicle based on reinforcement learning,” Applied sciences, vol. 8, no. 12, pp. 2494, 2018.
    [62] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
    [63] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, and G. Ostrovski, “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529-533, 2015.
    [64] D. He, Y. Zou, J. Wu, X. Zhang, Z. Zhang, and R. Wang, “Deep Q-learning based energy management strategy for a series hybrid electric tracked vehicle and its adaptability validation,” in 2019 IEEE Transportation Electrification Conference and Expo (ITEC), 2019, pp. 1-6.
    [65] S.-Y. Chen, H.-R. Lin, M.-C. Yang, and Z.-Y. Shen, “Fractional-order fuzzy pid contouring control for a vcms-based xy motion stage,” in 2020 6th International Conference on Control, Automation and Robotics (ICCAR), 2020, pp. 236-241.

    無法下載圖示 電子全文延後公開
    2029/01/20
    QR CODE