支持向量機(Support Vector Machine/SVM)是一種監督學習方法，通常用於分類問題。此外，SVM也被用於迴歸問題中，稱為支持向量迴歸（Support Vector Regression/SVR）。通過調整懲罰參數和可容忍的誤差界限，SVR比多數線性迴歸模型更有彈性。但其較常被用於單一結構的迴歸問題。
在無監督學習領域，聚類和混合回歸問題是我們非常感興趣的問題。因此，SVR也被推廣到混合回歸問題。基於模糊理論，延伸出了模糊加權支持向量迴歸(Fuzzy Weighted Support Vector Regression/FWSVR)。通過將隸屬度引入懲罰項，FWSVR可以處理混合迴歸問題，而不是像以前那樣一對一地使用SVR。然而，FWSVR在處理大規模數據時，所需的耗時較久。
在本文中，我們介紹了支持向量迴歸如何解決迴歸問題。並且我們將隸屬度作為不同數據的模糊權重，以構建模糊加權支持向量回歸（FWSVR）模型。然後，我們使用對偶坐標下降法找到模糊加權支持向量迴歸中拉格朗日乘數的更新函數。最後，我們考慮使用alpha cut方法來使模糊加權支持向量迴歸的結果更加有效率且穩定。實驗表明，FWSVR-DCD在處理大規模數據具有良好的性能且減少了所需的計算時間，並且估計結果對於有離群值的數據具有穩定性。

Support Vector Machine is a kind of supervise learning method, which is commonly used for classification problems. In addition, SVM is also used in regression problems called Support Vector Regression (SVR). SVR is a more flexible method than some linear regression models, through tuning the tolerance parameters and the acceptance error margin. However, SVR is most used for the regression problem of a single structure.
In the field of unsupervised learning, clustering and mixture regression problem are issues we are very interested in. Therefore, SVR is generalized to the mixture regression problems. Based on the fuzzy theory, the Fuzzy Weighted Support Vector Regression (FWSVR) is extended. By introducing the membership to the penalty term, FWSVR can handle the mixture regression problem at the same time, instead of using SVR one by one as before. However, FWSVR takes a long time when it processes large-scale data.
In this thesis, we introduce how SVR solve the regression problems. And we consider the membership as the fuzzy weight of different data to construct the Fuzzy Weighted Support Vector Regression models. Then we use the dual coordinate descent method to find the updating equations of the parameters of FWSVR, this method called FWSVR-DCD. Finally, we consider the alpha cut method to make FWSVR more robust. The simulations show the FWSVR has good performance in large-scale cases and reduce the computing time, and the estimated results are robust for data with outliers.

[1] Burges, C. J. (1998). A tutorial on support vector machines for pattern recognition. Data mining and knowledge discovery, 2(2), 121-167.

[2] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.

[3] Ben-Hur, A., Horn, D., Siegelmann, H. T., & Vapnik, V. (2001). Support vector clustering. Journal of machine learning research, 2(Dec), 125-137.

[4] Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and computing, 14(3), 199-222.

[5] Dufrenois, F., & Hamad, D. (2007, August). Fuzzy weighted support vector regression for multiple linear model estimation: application to object tracking in image sequences. In 2007 International Joint Conference on Neural Networks (pp. 1289-1294). IEEE.

[6] Ho, C. H., & Lin, C. J. (2012). Large-scale linear support vector regression. The Journal of Machine Learning Research, 13(1), 3323-3348.

[7] Yang, M. S., Wu, K. L., Hsieh, J. N., & Yu, J. (2008). Alpha-cut implemented fuzzy clustering algorithms and switching regressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38(3), 588-603.

[8] Drucker, H., Burges, C. J., Kaufman, L., Smola, A. J., & Vapnik, V. (1997). Support vector regression machines. In Advances in neural information processing systems (pp. 155-161).

[9] Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines.

[10] Flake, G. W., & Lawrence, S. (2002). Efficient SVM regression training with SMO. Machine Learning, 46(1-3), 271-290.

[11] Hsieh, C. J., Chang, K. W., Lin, C. J., Keerthi, S. S., & Sundararajan, S. (2008, July). A dual coordinate descent method for large-scale linear SVM. In Proceedings of the 25th international conference on Machine learning (pp. 408-415).

[12] Bai, X., Yao, W., & Boyer, J. E. (2012). Robust fitting of mixture regression models. Computational Statistics & Data Analysis, 56(7), 2347-2359.

[13] Cohen, E. A. (1984). Some effects of inharmonic partials on interval perception. Music Perception, 1(3), 323-349.