本論文針對一具有鋰電池與超級電容之複合電力電動巴士(Hybrid Electric Bus, HEB)設計一套能量管理系統(Energy Management System, EMS)，期望利用所研究之控制策略，有效調度雙電力之間的功率流向，同時平滑鋰電池輸出電流以達到保護鋰電池作用。為評估所提方法之能源使用效益，本研究建立一套HEB能源模擬軟體，並設計與比較不同控制策略之成本。首先以基本規則庫控制策略與全域搜索控制策略在不同需求功率和超級電容殘電量的情況下，計算出最佳功率分配比以達到最小化電能消耗；傳統頻率解耦控制策略利用低通濾波器將需求電能解耦為高頻與低頻兩頻段，使雙電力源各自依據其特性滿足需求功率。為使低通濾波參數依據當前環境自適應調整，本研究進一步發展強化式頻率解耦控制策略，使 EMS 可於系統運作過程中，不斷學習以找到最佳解。而在學習方法上本論文採用Q-Learning 策略並將其與頻率解耦控制策略結合，利用Q-Learning 動態調整低通濾波器之濾波參數以平滑化鋰電池輸出電流，達到保護電池之目的。
結果比較中，以全球輕型車輛測試規範(Worldwide Harmonized Light Vehicles Test Procedures, WLTP)行車形態為例，基本規則庫控制策略為比較基準，使用全域搜索控制策略的總能耗改善率達20.62%，標準差改善率達10.90%；傳統頻率解耦控制策略的總能耗改善率達5.62%，標準差改善率達13.19%；強化式頻率解耦控制策略的總能耗改善率達8.88%，鋰電池輸出標準差改善率達最高的16.71%。

This thesis focuses on designing an Energy Management System (EMS) for a Hybrid Electric Bus (HEB) equipped with lithium battery and supercapacitor. The objective is to utilize the proposed control strategies to efficiently manage the power flow between the two energy sources while smoothing the output current of the lithium battery, thereby protecting them. To evaluate the energy efficiency of the proposed method, this research developed an HEB energy simulation software and studied different control strategies. These include a Rule-Based Control (RBC) strategy and a Global Search Control (GSA) strategy, which calculate the optimal power distribution ratio under varying power demands and supercapacitor State of Charge (SOC) to minimize energy consumption. The Frequency Decoupling Control (FDC) strategy is used to decouple the power demand, enabling more efficient energy release or storage between the two energy sources. Furthermore, this study develops a Reinforcement Learning Frequency Decoupling Control (RLFDC) strategy, which allows the system to continuously learn and find optimal solutions during operation, offering greater flexibility compared to conventional algorithms. RLFDC adopts the Q-Learning strategy as its learning method. Q-Learning is used to dynamically adjust the filter parameters of the low-pass filter to smooth the output current of the lithium battery, thereby achieving the goal of battery protection.
In the comparison of results using the WLTP driving cycle as an example, with the RBC serving as the benchmark, the GSA control strategy achieved a total energy consumption improvement rate of 23.18% and a standard deviation improvement rate of 14.41%. The traditional FDC strategy resulted in a total energy consumption improvement rate of 4.36% and a standard deviation improvement rate of 12.84%. The RLFDC strategy achieved a total energy consumption improvement rate of 8.61% and the highest improvement in lithium battery output standard deviation at 15.45%.

[1] 方俊德，台灣淨零碳排路徑初探，臺灣經濟研究月刊，第46卷第1期，頁13-20，2023 年 1月。
[2] P. Willemsen, F. Ji, and M. Nalbach, “Key Items for Future Hybrid Applications: Energy Storage and Power Electronics for Micro Hybrids up to Full Hybrids and EVs,” Proc. FISITA 2012 World Automotive Congress, in Future Automotive Powertrains I, vol. 3, 2013, pp. 355-364.
[3] D. Lemian, and F. Bode, “Battery-Supercapacitor Energy Storage Systems for Electrical Vehicles: A Review,” Energies, vol. 15, no. 15, pp. 5683, 2022.
[4] M. S. Rajabi, P. Beigi, and S. Aghakhani, “Drone Delivery Systems and Energy Management: A Review and Future Trends,” Handbook of smart energy systems, pp. 1-19, 2023.
[5] J. M. A. Curti, X. Huang, R. Minaki, and Y. Hori, (2012, October), “A Simplified Power Management Strategy for a Supercapacitor/Battery Hybrid Energy Storage System Using the Half-Controlled Converter,” IECON 2012-38th annual conference on IEEE industrial electronics society, Montreal, QC, Canada, 2012, pp. 4006-4011.
[6] J. A. Manzolli, J. P. Trovao, and C. H. Antunes, “A Review of Electric Bus Vehicles Research Topics–Methods and Trends,” Renewable and Sustainable Energy Reviews, vol. 159, pp. 112211, 2022.
[7] S. S. Perumal, R. M. Lusby, and J. Larsen, “Electric Bus Planning & Scheduling: A Review of Related Problems and Methodologies,” European Journal of Operational Research, vol. 301, no. 2, pp. 395-413, 2022.
[8] Y. He, Z. Liu, and Z. Song, “Integrated Charging Infrastructure Planning and 65 Charging Scheduling for Battery Electric Bus Systems,” Transportation Research Part D: Transport and Environment, vol. 111, pp. 103437, 2022.
[9] J. Ji, Y. Bie, and L. Wang, “Optimal Electric Bus Fleet Scheduling for a Route with Charging Facility Sharing,” Transportation Research Part C: Emerging Technologies, vol. 147, pp. 104010, 2023.
[10] Ş. Yıldırım, and B. Yıldız, “Electric Bus Fleet Composition and Scheduling,” Transportation Research Part C: Emerging Technologies, vol. 129, pp. 103197, 2021.
[11] Y. Zhou, Q. Meng, and G. P. Ong, “Electric Bus Charging Scheduling for a Single Public Transport Route Considering Nonlinear Charging Profile and Battery Degradation Eeffect,” Transportation Research Part B: Methodological, vol. 159, pp. 49-75, 2022.
[12] V. Sidharthan Panaparambil, Y. Kashyap, and R. Vijay Castelino, “A Review on Hybrid Source Energy Management Strategies for Electric Vehicle,” International Journal of Energy Research, vol. 45, no. 14, pp. 19819-19850, 2021.
[13] C. Jia, J. Zhou, H. He, J. Li, Z. Wei, K. Li, and M. Shi, “A Novel Energy Management Strategy for Hybrid Electric Bus with Fuel Cell Health and Battery Thermal-and Health-Constrained Awareness,” Energy, vol. 271, pp. 127105, 2023.
[14] J. Shi, B. Xu, Y. Shen, and J. Wu, “Energy Management Strategy for Battery/Supercapacitor Hybrid Electric City Bus Based on Driving Pattern Recognition,” Energy, vol. 243, pp. 122752, 2022.
[15] D. Min, Z. Song, H. Chen, T. Wang, and T. Zhang, “Genetic Algorithm Optimized Neural Network Based Fuel Cell Hybrid Electric Vehicle Energy Management Strategy under Start-Stop Condition,” Applied Energy, vol. 306, pp. 118036, 2022.
[16] N. D. Nguyen, C. Yoon, Y. I., and Lee, “A Standalone Energy Management System 66 of Battery/Supercapacitor Hybrid Energy Storage System for Electric Vehicles Using Model Predictive Control,” IEEE Transactions on Industrial Electronics, vol. 70, no. 5, pp. 5104-5114, 2022.
[17] Q. Zhang, L.Wang, G. Li, and Y. Liu, “A Real-Time Energy Management Control Strategy for Battery and Supercapacitor Hybrid Energy Storage Systems of Pure Electric Vehicles,” Journal of Energy Storage, vol. 31, pp.101721, 2020.
[18] X. Sun, Z. Jin, M. Xue, and X. Tian, “Adaptive ECMS with Gear Shift Control by Grey Wolf Optimization Algorithm and Neural Network for Plug-in Hybrid Electric Buses,” IEEE Transactions on Industrial Electronics, vol. 71, no. 1, pp. 667-677,2023.
[19] D. Qiu, Y. Wang, W. Hua, and G. Strbac, “Reinforcement Learning for Electric Vehicle Applications in Power Systems: A Critical Review,” Renewable and Sustainable Energy Reviews, vol. 173, pp. 113052, 2023.
[20] Q. Li, X. Meng, F. Gao, G. Zhang, and W. Chen, “Approximate Cost-Optimal Energy Management of Hydrogen Electric Multiple Unit Trains Using Double QLearning Algorithm,” IEEE Transactions on Industrial Electronics, vol. 69, no. 9, pp. 9099-9110, Sept. 2022
[21] B. Xu, J. Shi, S. Li, H. Li, and Z. Wang, “Energy Consumption and Battery Aging Minimization Using a Q-learning Strategy for a Battery/Ultracapacitor Electric Vehicle,” Energy, vol. 229, pp. 120705, 2021.
[22] J. Shi, B. Xu, X. Zhou, and J. Hou, “A Cloud-Based Energy Management Strategy for Hybrid Electric City Bus Considering Real-Time Passenger Load Prediction,” Journal of Energy Storage, vol. 45, pp. 103749, 2022.
[23] P. Dai, W. Sun, H. Xie, Y. Lv, and Z. Han, “Stability Analysis of FCHEV Energy System Using Frequency Decoupling Control Method,” Journal of Power Electronics, vol. 17, no. 2, pp. 490-500, 2017. 67
[24] H. -A. Trinh, H. -V. -A. Truong, and K. -K. Ahn, “Energy Management Strategy for Fuel Cell Hybrid Power System Using Fuzzy Logic and Frequency Decoupling Methods,” 2021 24th International Conference on Mechatronics Technology (ICMT), Singapore, 2021, pp. 1-6,
[25] F. Tao, L. Zhu, Z. Fu, P. Si, and L. Sun, “Frequency Decoupling-Based Energy Management Strategy for Fuel Cell/Battery/Ultracapacitor Hybrid Vehicle Using Fuzzy Control Method,” IEEE Access, vol. 8, pp. 166491-166502, 2020.
[26] H. Abdelaty, and M. Mohamed, “A Prediction Model for Battery Electric Bus Energy Consumption in Transit,” Energies, vol. 14, no. 10, pp. 2824, 2021.
[27] H. Hinz, “Comparison of Lithium-ion Battery Models for Simulating Storage Systems in Distributed Power Generation,” Inventions, vol. 4, no. 3, pp. 41, 2019.
[28] V. Mounica, and Y. P. Obulesu, “Hybrid Power Management Strategy with Fuel Cell, Battery, and Supercapacitor for Fuel Economy in Hybrid Electric Vehicle Application,” Energies, vol. 15, no. 12, pp. 4185, 2022.
[29] A. H. Ganesh, and B. Xu, “A Review of Reinforcement Learning Based Energy Management Systems for Electrified Powertrains: Progress, Challenge, and Potential Wolution,” Renewable and Sustainable Energy Reviews, vol. 154, pp. 111833, 2022.
[30] A. K. Gautam, M. Tariq, J. P. Pandey, K. S. Verma, and S. Urooj, “Hybrid Sources Powered Electric Vehicle Configuration and Integrated Optimal Power Management Strategy,” IEEE Access, vol. 10, pp. 121684-121711, 2022.
[31] S. Y. Chen, and C. H. Chang, “Optimal Power Flows Control for Home Energy Management with Renewable Energy and Energy Storage Systems,” IEEE Transactions on Energy Conversion, vol. 38, no. 1, pp. 218-229, March 2023,
[32] S. Zhou, Z. Chen, D. Huang, and T. Lin, “Model Prediction and Rule Based Energy Management Strategy for a Plug-in Hybrid Electric Vehicle With Hybrid Energy 68 Storage System,” IEEE Transactions on Power Electronics, vol. 36, no. 5, pp. 5926-5940, May 2021
[33] P. Xie, J. M. Guerrero, S. Tan, N. Bazmohammadi, J. C. Vasquez, M. Mehrzadi, and Y. Al-Turki, “Optimization-Based Power and Energy Management System in Shipboard Microgrid: A Review,” IEEE systems journal, vol. 16, no. 1, pp. 578590, 2021.
[34] H. Dong, H. Dong, Z. Ding, S. Zhang, and T. Chang, “Introduction to Reinforcement Learning,” Deep Reinforcement Learning, Singapore: Springer Singapore, 2020, pp. 47-70.
[35] S. Xie, Z. Zhang, H. Yu, and X. Luo, “Recurrent Prediction Model for Partially Observable MDPs,” Information Sciences, vol. 620, pp. 125-141, 2023.
[36] 羅祥瑜，基於強化式學習之複合電力電動機車能量管理系統，國立臺灣師範大學電機工程學系，碩士論文，台北市，2022。
[37] X. Wang, Z. Yang, G. Chen, and Y. Liu, “A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error,” Electronics, vol. 12, no. 19, pp. 4176, 2023.
[38] J. Clifton, and E. Laber, “Q-Learning: Theory and Applications,” Annual Review of Statistics and Its Application, vol. 7, no. 1, pp. 279-301, 2020.
[39] 楊祐銓，應用強化式學習策略之分數階比例積分微分控制於X-Y-Y棒狀線性馬達定位平台，國立臺灣師範大學電機工程學系，碩士論文，台北市，2023。
[40] 李政言，基於模糊Q學習之心率調節系統於電動輔助自行車之研究，國立臺北科技大學電機工程學系，碩士論文，台北市，2024。
[41] H. Achour, and A. G. Olabi, “Driving Cycle Developments and Their Impacts on Energy Consumption of Transportation,” Journal of Cleaner Production, vol. 112, pp. 1778-1788, 2016.
[42] P. Chen, “Optimization of a Heavy-Duty Hybrid Transit Bus Operated over 69 Transient Cycles,” Department of Mechanical and Aerospace Engineering, West Virginia University, Graduate Theses, Dissertations, and Problem Reports, 2010.
[43] V. Lystianingrum, A. Irawan, I. B Santoso, I. M. Y. Negara, and A. Priyadi, “On Feasibility of Ultracapacitor Full Electric Transit Bus for Jakarta, Indonesia,” 2021 International Conference on Technology and Policy in Energy and Electric Power (ICT-PEP), Jakarta, Indonesia, 2021, September, pp. 102-106.