机器学习 | MATLAB实现SVR(支持向量机回归)fitrsvm参数优化
目录
- 机器学习 | MATLAB实现SVR(支持向量机回归)fitrsvm参数优化
- 目录
- fitrsvm
- 超参数优化
- 参考资料
- 致谢
目录
fitrsvm是MATLAB的Statistics and Machine Learning Toolbox用于实现支持向量机回归的内部函数，以下介绍 fitrsvm 超参数优化。
fitrsvm
- fitrsvm拟合支持向量机回归模型
- 函数用法：
-
Mdl = fitrsvm(Tbl,ResponseVarName) -
Mdl = fitrsvm(Tbl,formula) -
Mdl = fitrsvm(Tbl,Y) -
Mdl = fitrsvm(X,Y) -
Mdl = fitrsvm(___,Name,Value)
- fitrsvm 在低维到中等维预测数据集上训练或交叉验证支持向量机 (SVM) 回归模型。 fitrsvm 支持使用核函数映射预测数据，并通过二次规划支持 SMO、ISDA 或 L1 软边际最小化以实现目标函数最小化。
要在高维数据集（即包含许多预测变量的数据集）上训练线性 SVM 回归模型，请改用 fitrlinear。- 要为二元分类训练 SVM 模型，请参阅 fitcsvm 用于低维到中等维预测数据集，或 fitclinear 用于高维数据集。
超参数优化
load carsmall
%Specify Horsepower and Weight as the predictor variables (X) and MPG as the response variable (Y).
X = [Horsepower Weight];
Y = MPG;
- 通过使用自动超参数优化，找到将五倍交叉验证损失最小化的超参数。
- 为了重现性，设置随机种子并使用“预期改进加”采集功能。
rng default
Mdl = fitrsvm(X,Y,'OptimizeHyperparameters','auto',...
'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName',...
'expected-improvement-plus'))
|====================================================================================================================|
| Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | BoxConstraint| KernelScale | Epsilon |
| | result | log(1+loss) | runtime | (observed) | (estim.) | | | |
|====================================================================================================================|
| 1 | Best | 6.8077 | 10.27 | 6.8077 | 6.8077 | 0.35664 | 0.043031 | 0.30396 |
| 2 | Best | 2.9108 | 0.097504 | 2.9108 | 3.1259 | 70.67 | 710.65 | 1.6369 |
| 3 | Accept | 4.1884 | 0.18596 | 2.9108 | 3.1211 | 14.367 | 0.0059144 | 442.64 |
| 4 | Accept | 4.159 | 0.060684 | 2.9108 | 3.0773 | 0.0030879 | 715.31 | 2.6045 |
| 5 | Best | 2.902 | 0.17193 | 2.902 | 2.9015 | 969.07 | 703.1 | 0.88614 |
| 6 | Accept | 4.1884 | 0.052457 | 2.902 | 2.9017 | 993.93 | 919.26 | 22.16 |
| 7 | Accept | 2.9307 | 0.07753 | 2.902 | 2.9018 | 219.88 | 613.28 | 0.015526 |
| 8 | Accept | 2.9537 | 0.35015 | 2.902 | 2.9017 | 905.17 | 395.74 | 0.021914 |
| 9 | Accept | 2.9073 | 0.096023 | 2.902 | 2.9017 | 24.242 | 647.2 | 0.17855 |
| 10 | Accept | 2.9044 | 0.23756 | 2.902 | 2.9017 | 117.27 | 173.98 | 0.73387 |
| 11 | Accept | 2.9035 | 0.082249 | 2.902 | 2.9016 | 1.3516 | 131.19 | 0.0093404 |
| 12 | Accept | 4.0917 | 0.060391 | 2.902 | 2.902 | 0.012201 | 962.58 | 0.0092777 |
| 13 | Accept | 2.9525 | 0.98384 | 2.902 | 2.902 | 77.38 | 65.508 | 0.0093299 |
| 14 | Accept | 2.9352 | 0.082393 | 2.902 | 2.9019 | 21.591 | 166.43 | 0.035214 |
| 15 | Accept | 2.9341 | 0.10258 | 2.902 | 2.9019 | 45.286 | 207.56 | 0.009379 |
| 16 | Accept | 2.9104 | 0.057282 | 2.902 | 2.9018 | 0.064315 | 23.313 | 0.0093341 |
| 17 | Accept | 2.9056 | 0.12011 | 2.902 | 2.9018 | 0.33909 | 40.311 | 0.053394 |
| 18 | Accept | 2.9335 | 0.11486 | 2.902 | 2.8999 | 0.9904 | 41.169 | 0.0099688 |
| 19 | Accept | 2.9885 | 0.18598 | 2.902 | 2.8994 | 0.0010647 | 32.9 | 0.013178 |
| 20 | Accept | 4.1884 | 0.070586 | 2.902 | 2.9 | 0.0014524 | 1.9514 | 856.49 |
|====================================================================================================================|
| Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | BoxConstraint| KernelScale | Epsilon |
| | result | log(1+loss) | runtime | (observed) | (estim.) | | | |
|====================================================================================================================|
| 21 | Accept | 2.904 | 0.097499 | 2.902 | 2.8835 | 88.487 | 405.92 | 0.44372 |
| 22 | Accept | 2.9107 | 0.080064 | 2.902 | 2.8848 | 344.34 | 992 | 0.28418 |
| 23 | Accept | 2.904 | 0.063158 | 2.902 | 2.8848 | 0.92028 | 70.985 | 0.52233 |
| 24 | Accept | 2.9145 | 0.067186 | 2.902 | 2.8822 | 0.0011412 | 18.159 | 0.16652 |
| 25 | Best | 2.8909 | 0.18742 | 2.8909 | 2.8848 | 0.7614 | 17.3 | 1.9075 |
| 26 | Accept | 2.9188 | 0.10836 | 2.8909 | 2.8816 | 0.31049 | 18.301 | 0.35285 |
| 27 | Accept | 2.9327 | 0.065803 | 2.8909 | 2.8811 | 0.015362 | 20.395 | 1.6596 |
| 28 | Accept | 2.897 | 0.11764 | 2.8909 | 2.8811 | 3.3454 | 46.537 | 2.224 |
| 29 | Accept | 4.1884 | 0.061914 | 2.8909 | 2.881 | 983.2 | 0.52308 | 910.91 |
| 30 | Accept | 2.9167 | 0.055172 | 2.8909 | 2.8821 | 0.0011533 | 10.597 | 0.029731 |
__________________________________________________________
Optimization completed.
MaxObjectiveEvaluations of 30 reached.
Total function evaluations: 30
Total elapsed time: 46.34 seconds
Total objective function evaluation time: 14.3647
Best observed feasible point:
BoxConstraint KernelScale Epsilon
_____________ ___________ _______
0.7614 17.3 1.9075
Observed objective function value = 2.8909
Estimated objective function value = 2.8957
Function evaluation time = 0.18742
Best estimated feasible point (according to models):
BoxConstraint KernelScale Epsilon
_____________ ___________ _______
88.487 405.92 0.44372
Estimated objective function value = 2.8821
Estimated function evaluation time = 0.10031
Mdl =
RegressionSVM
ResponseName: 'Y'
CategoricalPredictors: []
ResponseTransform: 'none'
Alpha: [86x1 double]
Bias: 46.2480
KernelParameters: [1x1 struct]
NumObservations: 93
HyperparameterOptimizationResults: [1x1 BayesianOptimization]
BoxConstraints: [93x1 double]
ConvergenceInfo: [1x1 struct]
IsSupportVector: [93x1 logical]
Solver: 'SMO'
Properties, Methods
- 优化搜索了 BoxConstraint、KernelScale 和 Epsilon。 输出是具有最小估计交叉验证损失的回归。
