研究生: |
蔡旻原 Tsai, Min-Yuan |
---|---|
論文名稱: |
對於支撐向量機中Truncated Pinball損失的平滑化函數 Smoothing Functions of the Truncated Pinball Loss for Support Vector Machines |
指導教授: |
陳界山
Chen, Jein-Shan |
口試委員: |
杜威仕
Du, Wei-Shih 柯春旭 Ko, Chun-Hsu 張毓麟 Chang, Yu-Lin 朱亮儒 Chu, Liang-Ju 陳界山 Chen, Jein-Shan |
口試日期: | 2021/06/22 |
學位類別: |
碩士 Master |
系所名稱: |
數學系 Department of Mathematics |
論文出版年: | 2021 |
畢業學年度: | 109 |
語文別: | 英文 |
論文頁數: | 40 |
中文關鍵詞: | Truncated Pinball 損失函數 、平滑化函數 、可微分的最佳化問題 |
英文關鍵詞: | The truncated pinball loss function, Smoothing function, Differentiable optimization problem |
DOI URL: | http://doi.org/10.6345/NTNU202100786 |
論文種類: | 學術論文 |
相關次數: | 點閱:95 下載:21 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
我們的研究目的是探討truncated pinball 損失函數P_(τ,s) (x)以及它的平滑化函數φ_(τ,s) (x,μ)。我們推導了P_(τ,s) (x)可以被寫為絕對值函數跟仿射函數的和: -τ/2 |x+s|+(1+τ)/2 |x|+(τs+x)/2。再者,我們使用了來自於多篇參考文獻中的絕對值函數|x|的平滑化函數φ_abs^k (x,μ) (k=1,2,...,10)來產出我們關於truncated pinball損失函數的平滑化函數φ_(τ,s)^k (x,μ) (k=1,2,...,10)性質的主要結果。因此,我們可以把原先的最佳化問題min_(w,b)〖1/2 ‖w‖^2+C∑_(i=1)^l〖P_(τ,s) (1-y_i (w^T Φ(x_i )+b))〗〗中的P_(τ,s) (x)替換成φ_(τ,s)^k (x,μ)來得到可微分的最佳化問題。我們得出的結論是當μ趨近於0^+的時候我們的可微分的最佳化問題min_(w,b)〖1/2 ‖w‖^2+C∑_(i=1)^l〖φ_(τ,s)^k (1-y_i (w^T Φ(x_i )+b),μ)〗〗就變回原問題。更進一步地說,尋找可微分的最佳化問題的解將引出原問題的解。
The objective of our research was to investigate the truncated pinball loss function P_(τ,s) (x) and its smoothing function φ_(τ,s) (x,μ). We derived P_(τ,s) (x) can be rewritten as the sum of absolute value functions and an affine function: -τ/2 |x+s|+(1+τ)/2 |x|+(τs+x)/2. Moreover, we used the results of smoothing functions φ_abs^k (x,μ) (k=1,2,...,10) of absolute value function |x| from many references to produce our main results about properties of smoothing functions φ_(τ,s)^k (x,μ) (k=1,2,...,10) of the truncated pinball loss function P_(τ,s) (x). Hence, we can replace P_(τ,s) (x) with φ_(τ,s)^k (x,μ) for the original minimization problem min_(w,b)〖1/2 ‖w‖^2+C∑_(i=1)^l〖P_(τ,s) (1-y_i (w^T Φ(x_i )+b))〗〗 to obtain a differentiable minimization problem. We concluded that as μ approaches 0^+ our differentiable minimization problem, min_(w,b)〖1/2 ‖w‖^2+C∑_(i=1)^l〖φ_(τ,s)^k (1-y_i (w^T Φ(x_i )+b),μ)〗〗, becomes the original one. Furthermore, finding solutions to the differentiable minimization problem will lead to solution to the original one.
[1] Y.J. Bagul, A smooth transcendental approximation to |x|, International Journal of Mathematical Sciences and Engineering Applications(IJMSEA), vol. 11, pp. 213–217, 2017.
[2] Y.J. Bagul and B.K. Khairnar, A note on smooth transcendental approximation
to |x|, https://www.preprints.org/manuscript/201902.0190/download/final_file, 2019.
[3] Y.J. Bagul and C. Chesneau, Sigmoid functions for the smooth approximation
to the absolute value function, Moroccan Journal of Pure and Applied Analysis, vol.
7, pp. 12–19, 2021.
[4] A. Beck and M. Teboulle, Smoothing and first order methods: A unified framework, SIAM Journal on Optimization, vol. 22, pp. 557-580, 2012.
[5] C.-H. Chen and O.L. Mangasarian, A class of smoothing functions for nonlinear
and mixed complementarity problems, Computational Optimization and Applications, vol. 5, pp. 97–138, 1996.
[6] F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and
Complementarity Problems, Springer-Verlag New York, vol. 2, 2003.
[7] W.-Z. Gu, W.-P. Chen, C.-H. Ko, Y.-J. Lee, and J.-S. Chen, Two smooth
support vector machines for ε-insensitive regression, Computational Optimization
and Applications, vol. 70, pp. 171–199, 2018.
[8] J. Han and C. Moraga, The influence of the sigmoid function parameters on
the speed of backpropagation learning, International Workshop on Artificial Neural Networks, pp. 195–201, 1995.
[9] X.-L. Huang, L. Shi, and J. A.K. Suykens, Support vector machine classifier
with pinball loss, IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 36, pp. 984–997, 2013.
[10] Y. Lin, A note on margin-based loss functions in classification, Statistics & Probability Letters, vol. 68, pp. 73-82, 2004.
[11] C.T. Nguyen, B. Saheya, Y.-L. Chang, and J.-S. Chen, Unified smoothing
functions for absolute value equation associated with second-order cone, Applied
Numerical Mathematics, vol. 135, pp. 206–227, 2019.
[12] R.T. Rockafellar, Convex Analysis, Princeton University Press, 1970.
[13] B. Saheya, C.-H. Yu, and J.-S. Chen, Numerical comparisons based on four
smoothing functions for absolute value equation, Journal of Applied Mathematics and Computing, vol. 56, pp. 131–149, 2018.
[14] B. Saheya, C.T. Nguyen, J.-S. Chen, Neural network based on systematically
generated smoothing functions for absolute value equation, Journal of Applied Mathematics and Computing, vol. 61, pp. 533–558, 2019.
[15] X. Shen, L.-F. Niu, Z.-Q. Qi, and Y.-J. Tian, Support vector machine classifier
with truncated pinball loss, Pattern Recognition, vol. 68, pp.199-210, 2017.
[16] I. Steinwart, Sparseness of Support Vector Machines—some asymptotically sharp bounds, Advances in Neural Information Processing Systems, vol. 16, pp. 1069–1076, 2004.
[17] I. Steinwart, and A. Christmann, Support Vector Machines, Springer, 1st
edition, 2008.
[18] K.B. Stolarsky, Hölder Means, Lehmer Means, and x^−1 log cosh x, Journal of
mathematical analysis and applications, vol. 202, pp. 810-818, 1996.
[19] A. Tharwat, Behavioral analysis of support vector machine classifier with Gaussian kernel and imbalanced data, arXiv preprint arXiv:2007.05042, 2020.
[20] H. Tuy, Convex Analysis and Global Optimization, Springer International Publishing, 2nd, 2016.
[21] L. Vandenberghe, Smoothing, seas.ucla.edu/~vandenbe/236C/lectures/
smoothing.pdf, Spring 2013-14.
[22] V. Vapnik, The Nature of Statical Learning Theory, Springer-Verlag New York,
2nd, 2000.
[23] S. Voronin, G. Ozkaya, and D. Yoshida, Convolution based smooth approximations to the absolute value function with application to non-smooth regularization, arXiv preprint arXiv:1408.6795 [math.NA], 2014.
[24] X.-G. Zhang, Using class-center vectors to build support vector machines, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No. 98TH8468), pp. 3–11, 1999.