車牌辨識已經是一門成熟的技術，廣泛被應用於停車場之車輛管理、道路收費系統、道路交通監測等領域。隨著深度學習的蓬勃發展，現已有許多能夠出色完成車牌辨識之網路，然而在訓練中不可或缺之車牌資料，於資料之獲取及準備階段相當耗費時間跟人力資源，甚至根據地區或狀況上的差異，需要之車牌資料會有字母、字型、角度、環境等不同的組合，若符合欲研究條件之車牌資料不足，也會有需要重新拍攝合適之車牌資料並重新進行標注的可能。因此本論文提出了一種車牌資料的合成方法，結合3D圖像軟體Unreal Engine 4以及CycleGAN，嘗試在不使用任何真實車牌影像輔助之條件下製作出可使用於車牌辨識訓練之合成車牌資料，以減少車牌辨識之相關研究用於資料收集及標記整理之時間與人力成本並強化車牌辨識效果。合成出來之車牌影像於視覺上與真實車牌相似，並且被證明能有效地提升作為辨識網路之YOLOv4之準確度。以3179張之真實車牌影像所訓練出來之YOLOv4為比較基準，我們所提出之合成方法所製作出的車牌影像能將原本97.00%之mAP提升至98.04%。

With the development of deep learning in image recognition, automatic license plate recognition is widely used in urban parking lots, roadside toll systems, and road traffic monitoring. In the study of automatic license plate recognition, we are faced with many challenges such as complex road environment, blurred license plates, diverse shooting angles, and scarce license plates. In order to train a powerful model, a sufficient license plate data set is essential. However, it is very difficult to build a sufficient license plate data set. In addition to sufficient quantity, it is also necessary to collect all kinds of license plate texts and images taken in various environments. This is a very labor-intensive and time-consuming task. Therefore, in this thesis, we propose to generate synthetic license plate images using Unreal 4 engine and CycleGAN. In order to reduce the time and labor cost of license plate collection, we intend to generate synthetic license plate data that can be used in training license plate recognition model without using any real data. The synthetic images we generated not only look similar to the real license plates, but are effective for training Yolo's license plate character recognition model. Taking the model trained by YOLOv4 from 3,179 real license plate images as a comparison benchmark, by adding the synthetic data we generate, the mAP of YOLOv4 can be increased from 97.00% to 98.04%.

[1] SANDERS, Andrew. “An introduction to Unreal engine 4.” CRC Press, 2016.
[2] ZHU, Jun-Yan, et al. “Unpaired image-to-image translation using cycle-consistent adversarial networks.” In: Proceedings of the IEEE international conference on computer vision. 2017. p. 2223-2232.
[3] DERAKHSHANI, Dariush. “Introducing Autodesk Maya 2013.” John Wiley & Sons, 2012.
[4] BOCHKOVSKIY, Alexey; WANG, Chien-Yao; LIAO, Hong-Yuan Mark. “Yolov4: Optimal speed and accuracy of object detection.” arXiv preprint arXiv:2004.10934, 2020.
[5] SARFRAZ, Muhammad; AHMED, Mohammed Jameel; GHAZI, Syed A. “Saudi Arabian license plate recognition system.” In: 2003 International Conference on Geometric Modeling and Graphics, 2003. Proceedings. IEEE, 2003. p. 36-41.
[6] PALIY, Ihor, et al. “Approach to recognition of license plate numbers using neural networks.” In: 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541). IEEE, 2004. p. 2965-2970.
[7] BABU, C. Nelson Kennedy; NALLAPERUMAL, Krishnan, “An efficient geometric feature based license plate localization and recognition,” Int. J. Imaging Sci. Eng, 2008, 2.2: 189-194.
[8] HONGLIANG, Bai; CHANGPING, Liu. “A hybrid license plate extraction method based on edge statistics and morphology.” In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. IEEE, 2004. p. 831-834.
[9] DUAN, Tran Duc; DUC, Duong Anh; DU, Tran Le Hong. “Combining Hough transform and contour algorithm for detecting vehicles' license-plates.” In: Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, 2004. IEEE, 2004. p. 747-750.
[10] BUSCH, Christoph, et al. “Feature based recognition of traffic video streams for online route tracing.” In: VTC'98. 48th IEEE Vehicular Technology Conference. Pathway to Global Wireless Revolution (Cat. No. 98CH36151). IEEE, 1998. p. 1790-1794.
[11] CHANG, Shyang-Lih, et al. “Automatic license plate recognition.” IEEE transactions on intelligent transportation systems, 2004, 5.1: 42-53.
[12] XIA, Huadong; LIAO, Dongchu. “The study of license plate character segmentation algorithm based on vetical projection.” In: 2011 International Conference on Consumer Electronics, Communications and Networks (CECNet). IEEE, 2011. p. 4583-4586.
[13] PATEL, Hetal, et al. “License plate detection using harris corner and character segmentation by integrated approach from an image.” Procedia Computer Science, 2016, 79: 419-425.
[14] ZHENG, Danian; ZHAO, Yannan; WANG, Jiaxin. “An efficient method of license plate location. Pattern recognition letters,” 2005, 26.15: 2431-2438.
[15] NUKANO, T.; FUKUMI, M.; KHALID, M. “Vehicle license plate character recognition by neural networks.” In: Proceedings of 2004 International Symposium on Intelligent Signal Processing and Communication Systems, 2004. ISPACS 2004. IEEE, 2004. p. 771-775.
[16] SHAPIRO, Vladimir; GLUHCHEV, Georgi. “Multinational license plate recognition system: Segmentation and classification.” In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. IEEE, 2004. p. 352-355.
[17] JAIN, Vishal, et al. “Deep automatic license plate recognition system.” In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing. 2016. p. 1-8.
[18] RADZI, Syafeeza Ahmad; KHALIL-HANI, Mohamed. “Character recognition of license plate number using convolutional neural network.” In: International Visual Informatics Conference. Springer, Berlin, Heidelberg, 2011. p. 45-55.
[19] PARASURAMAN, Kumar; SUBIN, P. S. “SVM based license plate recognition system.” In: IEEE International Conference on Computational Intelligence and Computing Research. 2010.
[20] LIN, Cheng-Hung; SIE, Yi-Sin. “Two-Stage License Plate Recognition System Using Deep learning.” In: 2019 8th International Conference on Innovation, Communication and Engineering (ICICE). IEEE, 2019. p. 132-135.
[21] REDMON, Joseph, et al. “You only look once: Unified, real-time object detection.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 779-788.
[22] GIRSHICK, Ross, et al. “Rich feature hierarchies for accurate object detection and semantic segmentation.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. p. 580-587.
[23] GIRSHICK, Ross. “Fast r-cnn.” In: Proceedings of the IEEE international conference on computer vision. 2015. p. 1440-1448.
[24] REDMON, Joseph; FARHADI, Ali. “YOLO9000: better, faster, stronger.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 7263-7271.
[25] REN, Shaoqing, et al. “Faster R-CNN: towards real-time object detection with region proposal networks.” IEEE transactions on pattern analysis and machine intelligence, 2016, 39.6: 1137-1149.
[26] REDMON, Joseph; FARHADI, Ali. “Yolov3: An incremental improvement.” arXiv preprint arXiv:1804.02767, 2018.
[27] HE, Kaiming, et al. “Deep residual learning for image recognition.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770-778.
[28] LIN, Tsung-Yi, et al. “Feature pyramid networks for object detection.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 2117-2125.
[29] WANG, Chien-Yao, et al. “CSPNet: A new backbone that can enhance learning capability of CNN.” In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020. p. 390-391.
[30] HUANG, Gao, et al. “Densely connected convolutional networks.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 4700-4708.
[31] HE, Kaiming, et al. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE transactions on pattern analysis and machine intelligence, 2015, 37.9: 1904-1916.
[32] LIU, Shu, et al. “Path aggregation network for instance segmentation.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. p. 8759-8768.
[33] GOODFELLOW, Ian, et al. “Generative adversarial nets.” Advances in neural information processing systems, 2014, 27.
[34] MIRZA, Mehdi; OSINDERO, Simon. “Conditional generative adversarial nets.” arXiv preprint arXiv:1411.1784, 2014.
[35] RADFORD, Alec; METZ, Luke; CHINTALA, Soumith. “Unsupervised representation learning with deep convolutional generative adversarial networks.” arXiv preprint arXiv:1511.06434, 2015.
[36] HAN, Byung-Gil, et al. “License plate image generation using generative adversarial networks for end-to-end license plate character recognition from a small set of real images.” Applied Sciences, 2020, 10.8: 2780.
[37] LEDIG, Christian, et al. “Photo-realistic single image super-resolution using a generative adversarial network.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 4681-4690.
[38] ZHU, Jun-Yan, et al. “Generative visual manipulation on the natural image manifold.” In: European conference on computer vision. Springer, Cham, 2016. p. 597-613.
[39] ENGIN, Deniz; GENÇ, Anil; KEMAL EKENEL, Hazim. “Cycle-dehaze: Enhanced cyclegan for single image dehazing.” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018. p. 825-833.
[40] XIAO, Yuxuan, et al. “Single image colorization via modified CycleGAN.” In: 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019. p. 3247-3251.
[41] WU, Changhao, et al. “How many labeled license plates are needed?.” In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Springer, Cham, 2018. p. 334-346.
[42] NEWELL, Martin E.; BLINN, James F. “The progression of realism in computer generated images.” In: Proceedings of the 1977 annual conference. 1977. p. 444-448.
[43] EDLER, Dennis; KEIL, Julian; DICKMANN, Frank. “From na pali to earth—an ‘unreal’engine for modern geodata?.” In: Modern approaches to the visualization of landscapes. Springer VS, Wiesbaden, 2020. p. 279-291.
[44] PROENÇA, Pedro F.; GAO, Yang. “Deep learning for spacecraft pose estimation from photorealistic rendering.” In: 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020. p. 6007-6013.
[45] WANG, Shubo, et al. “Construction of a virtual reality platform for UAV deep learning.” In: 2017 Chinese Automation Congress (CAC). IEEE, 2017. p. 3912-3916.
[46] YANG, Xinglong, et al. “An Efficient Simulation Platform for Testing and Validating Autonomous Navigation Algorithms for Multi-rotor UAVs Based on Unreal Engine.” In: China Satellite Navigation Conference. Springer, Singapore, 2019. p. 527-539.
[47] RICHTER, Stephan R., et al. “Playing for data: Ground truth from computer games.” In: European conference on computer vision. Springer, Cham, 2016. p. 102-118.
[48] CORDTS, Marius, et al. “The cityscapes dataset.” In: CVPR Workshop on the Future of Datasets in Vision. 2015.