簡易檢索 / 詳目顯示

研究生: 蔡旻原
Tsai, Min-Yuan
論文名稱: 對於支撐向量機中Truncated Pinball損失的平滑化函數
Smoothing Functions of the Truncated Pinball Loss for Support Vector Machines
指導教授: 陳界山
Chen, Jein-Shan
口試委員: 杜威仕
Du, Wei-Shih
柯春旭
Ko, Chun-Hsu
張毓麟
Chang, Yu-Lin
朱亮儒
Chu, Liang-Ju
陳界山
Chen, Jein-Shan
口試日期: 2021/06/22
學位類別: 碩士
Master
系所名稱: 數學系
Department of Mathematics
論文出版年: 2021
畢業學年度: 109
語文別: 英文
論文頁數: 40
中文關鍵詞: Truncated Pinball 損失函數平滑化函數可微分的最佳化問題
英文關鍵詞: The truncated pinball loss function, Smoothing function, Differentiable optimization problem
DOI URL: http://doi.org/10.6345/NTNU202100786
論文種類: 學術論文
相關次數: 點閱:95下載:21
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 我們的研究目的是探討truncated pinball 損失函數P_(τ,s) (x)以及它的平滑化函數φ_(τ,s) (x,μ)。我們推導了P_(τ,s) (x)可以被寫為絕對值函數跟仿射函數的和: -τ/2 |x+s|+(1+τ)/2 |x|+(τs+x)/2。再者,我們使用了來自於多篇參考文獻中的絕對值函數|x|的平滑化函數φ_abs^k (x,μ) (k=1,2,...,10)來產出我們關於truncated pinball損失函數的平滑化函數φ_(τ,s)^k (x,μ) (k=1,2,...,10)性質的主要結果。因此,我們可以把原先的最佳化問題min_(w,b)⁡〖1/2 ‖w‖^2+C∑_(i=1)^l〖P_(τ,s) (1-y_i (w^T Φ(x_i )+b))〗〗中的P_(τ,s) (x)替換成φ_(τ,s)^k (x,μ)來得到可微分的最佳化問題。我們得出的結論是當μ趨近於0^+的時候我們的可微分的最佳化問題min_(w,b)⁡〖1/2 ‖w‖^2+C∑_(i=1)^l〖φ_(τ,s)^k (1-y_i (w^T Φ(x_i )+b),μ)〗〗就變回原問題。更進一步地說,尋找可微分的最佳化問題的解將引出原問題的解。

    The objective of our research was to investigate the truncated pinball loss function P_(τ,s) (x) and its smoothing function φ_(τ,s) (x,μ). We derived P_(τ,s) (x) can be rewritten as the sum of absolute value functions and an affine function: -τ/2 |x+s|+(1+τ)/2 |x|+(τs+x)/2. Moreover, we used the results of smoothing functions φ_abs^k (x,μ) (k=1,2,...,10) of absolute value function |x| from many references to produce our main results about properties of smoothing functions φ_(τ,s)^k (x,μ) (k=1,2,...,10) of the truncated pinball loss function P_(τ,s) (x). Hence, we can replace P_(τ,s) (x) with φ_(τ,s)^k (x,μ) for the original minimization problem min_(w,b)⁡〖1/2 ‖w‖^2+C∑_(i=1)^l〖P_(τ,s) (1-y_i (w^T Φ(x_i )+b))〗〗 to obtain a differentiable minimization problem. We concluded that as μ approaches 0^+ our differentiable minimization problem, min_(w,b)⁡〖1/2 ‖w‖^2+C∑_(i=1)^l〖φ_(τ,s)^k (1-y_i (w^T Φ(x_i )+b),μ)〗〗, becomes the original one. Furthermore, finding solutions to the differentiable minimization problem will lead to solution to the original one.

    1 Introduction 1 2 Preliminaries 11 3 Main Results 24 4 Conclusion 38 References 38

    [1] Y.J. Bagul, A smooth transcendental approximation to |x|, International Journal of Mathematical Sciences and Engineering Applications(IJMSEA), vol. 11, pp. 213–217, 2017.
    [2] Y.J. Bagul and B.K. Khairnar, A note on smooth transcendental approximation
    to |x|, https://www.preprints.org/manuscript/201902.0190/download/final_file, 2019.
    [3] Y.J. Bagul and C. Chesneau, Sigmoid functions for the smooth approximation
    to the absolute value function, Moroccan Journal of Pure and Applied Analysis, vol.
    7, pp. 12–19, 2021.
    [4] A. Beck and M. Teboulle, Smoothing and first order methods: A unified framework, SIAM Journal on Optimization, vol. 22, pp. 557-580, 2012.
    [5] C.-H. Chen and O.L. Mangasarian, A class of smoothing functions for nonlinear
    and mixed complementarity problems, Computational Optimization and Applications, vol. 5, pp. 97–138, 1996.
    [6] F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and
    Complementarity Problems, Springer-Verlag New York, vol. 2, 2003.
    [7] W.-Z. Gu, W.-P. Chen, C.-H. Ko, Y.-J. Lee, and J.-S. Chen, Two smooth
    support vector machines for ε-insensitive regression, Computational Optimization
    and Applications, vol. 70, pp. 171–199, 2018.
    [8] J. Han and C. Moraga, The influence of the sigmoid function parameters on
    the speed of backpropagation learning, International Workshop on Artificial Neural Networks, pp. 195–201, 1995.
    [9] X.-L. Huang, L. Shi, and J. A.K. Suykens, Support vector machine classifier
    with pinball loss, IEEE Transactions on Pattern Analysis and Machine Intelligence,
    vol. 36, pp. 984–997, 2013.
    [10] Y. Lin, A note on margin-based loss functions in classification, Statistics & Probability Letters, vol. 68, pp. 73-82, 2004.
    [11] C.T. Nguyen, B. Saheya, Y.-L. Chang, and J.-S. Chen, Unified smoothing
    functions for absolute value equation associated with second-order cone, Applied
    Numerical Mathematics, vol. 135, pp. 206–227, 2019.
    [12] R.T. Rockafellar, Convex Analysis, Princeton University Press, 1970.
    [13] B. Saheya, C.-H. Yu, and J.-S. Chen, Numerical comparisons based on four
    smoothing functions for absolute value equation, Journal of Applied Mathematics and Computing, vol. 56, pp. 131–149, 2018.
    [14] B. Saheya, C.T. Nguyen, J.-S. Chen, Neural network based on systematically
    generated smoothing functions for absolute value equation, Journal of Applied Mathematics and Computing, vol. 61, pp. 533–558, 2019.
    [15] X. Shen, L.-F. Niu, Z.-Q. Qi, and Y.-J. Tian, Support vector machine classifier
    with truncated pinball loss, Pattern Recognition, vol. 68, pp.199-210, 2017.
    [16] I. Steinwart, Sparseness of Support Vector Machines—some asymptotically sharp bounds, Advances in Neural Information Processing Systems, vol. 16, pp. 1069–1076, 2004.
    [17] I. Steinwart, and A. Christmann, Support Vector Machines, Springer, 1st
    edition, 2008.
    [18] K.B. Stolarsky, Hölder Means, Lehmer Means, and x^−1 log cosh x, Journal of
    mathematical analysis and applications, vol. 202, pp. 810-818, 1996.
    [19] A. Tharwat, Behavioral analysis of support vector machine classifier with Gaussian kernel and imbalanced data, arXiv preprint arXiv:2007.05042, 2020.
    [20] H. Tuy, Convex Analysis and Global Optimization, Springer International Publishing, 2nd, 2016.
    [21] L. Vandenberghe, Smoothing, seas.ucla.edu/~vandenbe/236C/lectures/
    smoothing.pdf, Spring 2013-14.
    [22] V. Vapnik, The Nature of Statical Learning Theory, Springer-Verlag New York,
    2nd, 2000.
    [23] S. Voronin, G. Ozkaya, and D. Yoshida, Convolution based smooth approximations to the absolute value function with application to non-smooth regularization, arXiv preprint arXiv:1408.6795 [math.NA], 2014.
    [24] X.-G. Zhang, Using class-center vectors to build support vector machines, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No. 98TH8468), pp. 3–11, 1999.

    下載圖示
    QR CODE