机器学习十大算法---1.线性回归

1. 线性回归的模型函数和损失函数

线性回归遇到的问题一般是这样的。我们有m个样本，每个样本对应于n维特征和一个结果输出，如下：

　　　　我们的问题是，对于一个新的, 他所对应的是多少呢？ 如果这个问题里面的y是连续的，则是一个回归问题，否则是一个分类问题。

　　　　对于n维特征的样本数据，如果我们决定使用线性回归，那么对应的模型是这样的：

其中θi(i = 0,1,2... n)为模型参数，xi (i = 0,1,2... n)为每个样本的n个特征值。这个表示可以简化，我们增加一个特征x0=1 ，这样

进一步用矩阵形式表达更加简洁如下：

　　　　其中， 假设函数hθ(x)为mx1的向量,θ为nx1的向量，里面有n个代数法的模型参数。为mxn维的矩阵。m代表样本的个数，n代表样本的特征数。

　　　　得到了模型，我们需要求出需要的损失函数，一般线性回归我们用均方误差作为损失函数。损失函数的代数法表示如下：

　　　　进一步用矩阵形式表达损失函数：

　　　　由于矩阵法表达比较的简洁，后面我们将统一采用矩阵方式表达模型函数和损失函数。

2. 线性回归的算法

梯度下降法，一种是最小二乘法。

　　　　如果采用梯度下降法，则θ的迭代公式是这样的：

　　　　通过若干次迭代后，我们可以得到最终的的结果

　　　　如果采用最小二乘法，则θ的结果公式如下：

当然线性回归，还有其他的常用算法，比如牛顿法和拟牛顿法，这里不详细描述。

3. 线性回归的推广：多项式回归

　　　　回到我们开始的线性模型

如果这里不仅仅是x的一次方，比如增加二次方，那么模型就变成了多项式回归。这里写一个只有两个特征的p次方多项式回归的模型：

　　　　我们令

    我们就得到了下式：

　　　　可以发现，我们又重新回到了线性回归，这是一个五元线性回归，可以用线性回归的方法来完成算法。对于每个二元样本特征（x1,x2）我们得到一个五元样本特征，通过这个改进的五元样本特征，我们重新把不是线性回归的函数变回线性回归。

4. 线性回归的推广：广义线性回归

　　　　在上一节的线性回归的推广中，我们对样本特征端做了推广，这里我们对于特征y做推广。比如我们的输出Y不满足和X的线性关系，但是lnY和X满足线性关系，模型函数如下：

lnY=Xθ

　　　　这样对与每个样本的输入y，我们用 lny去对应， 从而仍然可以用线性回归的算法去处理这个问题。我们把 Iny一般化，假设这个函数是单调可微函数g(.),则一般化的广义线性回归形式是：

　　　　这个函数g(.)我们称为联系函数。

5. 线性回归的正则化

　　　　为了防止模型的过拟合，我们在建立线性模型的时候经常需要加入正则化项。一般有L1正则化和L2正则化。

　　　　线性回归的L1正则化:Lasso回归,L1正则化的项有一个常数系数来调节损失函数的均方差项和正则化项的权重,如下：　　

　　　　其中n为样本个数，α为常数系数，需要进行调优。||θ||1为L1范数。

 　　　　Lasso回归可以使得一些特征的系数变小，甚至还是一些绝对值较小的系数直接变为0。增强模型的泛化能力。Lasso回归的求解办法一般有坐标轴下降法（coordinate descent）和最小角回归法（ Least Angle Regression）

　　　　线性回归的L2正则化:Ridge回归，Ridge回归的正则化项是L2范数，如下：

　　　　其中为常数系数，需要进行调优。||θ||2为L2范数。

　　　　Ridge回归在不抛弃任何一个特征的情况下，缩小了回归系数，使得模型相对而言比较的稳定，但和Lasso回归比，这会使得模型的特征留的特别多，模型解释性差。

 　　　  Ridge回归的求解比较简单，一般用最小二乘法。这里给出用最小二乘法的矩阵推导形式，和普通线性回归类似。

　　　　令J(θ)的导数为0，得到下式：

　　　　整理即可得到最后的θ的结果：

 　　　 其中E为单位矩阵。

练习代码：

#!/usr/bin/python 
# -*- coding: utf-8 -*- 

__author__ = "zhuxiaoxia"
__version__ = "$Revision: 1.0 $" 
__date__ = "$Date: 2017/10/31 21:20:19 $"
__copyright__ = "Copyright (c) 2017 zhuxiaoxia"
__license__ = "Python"

class LinearRegression:  
    def __init__(self):
        self.__LEARNING_RATE = 0.1  #学习率lr
        self.__MAX_FEATURE_CNT = 11   #样本数m
        self.theta =[0,0,0,0,0,0,0,0,0,0,0]  #初始化权重delta
        #训练样本 
        self.samples = [[1,1,0,1,0,0,1,0,0,1,1,4], \
                        [1,0,1,0,0,1,0,0,3,0,0,0], \
                        [1,0,0,2,2,0,1,0,0,0,0,0], \
                        [1,2,0,1,0,1,1,0,0,0,0,2], \
                        [1,0,1,0,0,0,0,0,3,1,0,2], \
                        [1,0,1,0,0,1,1,1,1,0,0,1], \
                        [1,0,1,0,0,1,0,1,0,1,1,4], \
                        [1,2,0,1,1,0,0,0,0,0,1,3], \
                        [1,0,1,0,0,2,0,0,0,2,0,4], \
                        [1,0,0,1,0,1,1,0,0,1,1,3], \
                        [1,0,1,0,0,0,2,0,0,1,1,3], \
                        [1,1,1,1,1,0,0,1,0,0,0,2], \
                        [1,0,0,0,0,0,0,2,1,1,1,5]]
        #测试样本
        self.test_cases = \
            [[1,1,1,1,1,1,1,0,0,1,0,3], \
            [1,1,0,1,0,1,0,1,0,0,1,3], \
            [1,0,1,1,1,0,0,1,0,1,0,3], \
            [1,0,1,0,0,0,1,1,0,2,0,5], \
            [1,0,0,0,1,1,1,0,0,1,0,2], \
            [1,1,1,0,0,0,0,1,0,1,1,5], \
            [1,3,0,1,0,0,1,0,0,0,3,3], \
            [1,0,1,0,0,0,1,1,1,0,0,1], \
            [1,0,0,0,2,0,0,0,1,0,2,2], \
            [1,0,1,0,0,3,1,0,0,0,0,0], \
            [1,0,1,1,2,0,0,0,0,1,0,2]]
    def __hypothesis(self,x):
        h = 0
        for idx in range(0,self.__MAX_FEATURE_CNT):
            h+=x[idx]*self.theta[idx]
        return h
    def __update_theta(self,x,delta):
        for idx in range(0,self.__MAX_FEATURE_CNT):
            self.theta[idx]-=x[idx]*delta
    def __train(self):
        for x in self.samples:
            h = self.__hypothesis(x[0:-1])
            y = x[self.__MAX_FEATURE_CNT]
            delta = (h-y)*self.__LEARNING_RATE
            self.__update_theta(x,delta)
    def __get_loss(self):
        loss_sum = 0
        for x in self.samples:
            h = self.__hypothesis(x[0:-1])
            y = x[self.__MAX_FEATURE_CNT]
            loss_sum +=(h-y)*(h-y)/2
            return loss_sum
    def online_training(self):
        for itr in range(0,100):
            self.__train()  #更新theta
            loss_sum = self.__get_loss()  #获取loss
            for i in range(0,self.__MAX_FEATURE_CNT):
                print "theta[%d] =%1f" %(i,self.theta[i])
            print "第%d迭代的损失为%1f" %(itr,loss_sum)
            if loss_sum<0.00001:
                break
    def test(self):
        for t in self.test_cases:
            h=self.__hypothesis(t[0:-1])
            print "预测值H=%1f,真实值ANS=%d，误差loss=%f" %(h,t[self.__MAX_FEATURE_CNT],self.__get_loss())
    def pt(self):
        for i in range(0,self.__MAX_FEATURE_CNT):
                print "theta[%d]=%1f" %(i,self.theta[i])

if __name__=="__main__":
    lr=LinearRegression()   #实例化线性回归类
    lr.online_training()  #训练
    print "-------train over!!!-----------"
    lr.pt()
    print "------------------------------"
    lr.test()  #测试数据集

输出：

theta[0] =0.932768
theta[1] =0.689504
theta[2] =0.398498
theta[3] =0.188364
theta[4] =-0.238094
theta[5] =0.357043
theta[6] =0.178898
theta[7] =0.599572
theta[8] =0.554495
theta[9] =1.090739
theta[10] =0.844823
第0迭代的损失为0.002805
theta[0] =0.790254
theta[1] =0.752568
theta[2] =0.234935
theta[3] =0.050874
theta[4] =-0.339084
theta[5] =0.049102
theta[6] =-0.055743
theta[7] =0.773089
theta[8] =0.070681
theta[9] =1.433472
theta[10] =0.981377
第1迭代的损失为0.001114
theta[0] =0.739567
theta[1] =0.762793
theta[2] =0.155520
theta[3] =0.022372
theta[4] =-0.340424
theta[5] =-0.076967
theta[6] =-0.169710
theta[7] =0.823717
theta[8] =-0.088236
theta[9] =1.596149
theta[10] =1.012621
第2迭代的损失为0.000655
theta[0] =0.721642
theta[1] =0.764553
theta[2] =0.108136
theta[3] =0.023634
theta[4] =-0.325732
theta[5] =-0.129802
theta[6] =-0.218305
theta[7] =0.834445
theta[8] =-0.142559
theta[9] =1.684289
theta[10] =1.009551
第3迭代的损失为0.000107
theta[0] =0.715244
theta[1] =0.767323
theta[2] =0.075176
theta[3] =0.029650
theta[4] =-0.313209
theta[5] =-0.153439
theta[6] =-0.235075
theta[7] =0.834937
theta[8] =-0.160420
theta[9] =1.735453
theta[10] =0.995822
第4迭代的损失为0.000035
theta[0] =0.712808
theta[1] =0.771849
theta[2] =0.050200
theta[3] =0.033593
theta[4] =-0.304852
theta[5] =-0.164903
theta[6] =-0.237359
theta[7] =0.834542
theta[8] =-0.164998
theta[9] =1.766492
theta[10] =0.979475
第5迭代的损失为0.000361
theta[0] =0.711707
theta[1] =0.777326
theta[2] =0.030370
theta[3] =0.034509
theta[4] =-0.299348
theta[5] =-0.170870
theta[6] =-0.233504
theta[7] =0.835425
theta[8] =-0.164882
theta[9] =1.786138
theta[10] =0.963370
第6迭代的损失为0.000782
theta[0] =0.711062
theta[1] =0.783034
theta[2] =0.014156
theta[3] =0.032983
theta[4] =-0.295309
theta[5] =-0.174087
theta[6] =-0.227391
theta[7] =0.837536
theta[8] =-0.163344
theta[9] =1.799183
theta[10] =0.948549
第7迭代的损失为0.001124
theta[0] =0.710598
theta[1] =0.788554
theta[2] =0.000608
theta[3] =0.029784
theta[4] =-0.291837
theta[5] =-0.175784
theta[6] =-0.220767
theta[7] =0.840347
theta[8] =-0.161583
theta[9] =1.808311
theta[10] =0.935355
第8迭代的损失为0.001343
theta[0] =0.710233
theta[1] =0.793688
theta[2] =-0.010917
theta[3] =0.025522
theta[4] =-0.288453
theta[5] =-0.176569
theta[6] =-0.214378
theta[7] =0.843385
theta[8] =-0.159986
theta[9] =1.815037
theta[10] =0.923836
第9迭代的损失为0.001455
theta[0] =0.709946
theta[1] =0.798371
theta[2] =-0.020870
theta[3] =0.020621
theta[4] =-0.284943
theta[5] =-0.176773
theta[6] =-0.208506
theta[7] =0.846336
theta[8] =-0.158632
theta[9] =1.820227
theta[10] =0.913904
第10迭代的损失为0.001488
theta[0] =0.709730
theta[1] =0.802601
theta[2] =-0.029577
theta[3] =0.015357
theta[4] =-0.281238
theta[5] =-0.176592
theta[6] =-0.203224
theta[7] =0.849029
theta[8] =-0.157500
theta[9] =1.824382
theta[10] =0.905412
第11迭代的损失为0.001472
theta[0] =0.709578
theta[1] =0.806409
theta[2] =-0.037274
theta[3] =0.009912
theta[4] =-0.277338
theta[5] =-0.176150
theta[6] =-0.198514
theta[7] =0.851391
theta[8] =-0.156544
theta[9] =1.827806
theta[10] =0.898192
第12迭代的损失为0.001425
theta[0] =0.709483
theta[1] =0.809838
theta[2] =-0.044140
theta[3] =0.004399
theta[4] =-0.273280
theta[5] =-0.175526
theta[6] =-0.194325
theta[7] =0.853407
theta[8] =-0.155720
theta[9] =1.830684
theta[10] =0.892076
第13迭代的损失为0.001360
theta[0] =0.709437
theta[1] =0.812929
theta[2] =-0.050312
theta[3] =-0.001104
theta[4] =-0.269106
theta[5] =-0.174775
theta[6] =-0.190597
theta[7] =0.855094
theta[8] =-0.154995
theta[9] =1.833141
theta[10] =0.886908
第14迭代的损失为0.001286
theta[0] =0.709433
theta[1] =0.815724
theta[2] =-0.055896
theta[3] =-0.006545
theta[4] =-0.264864
theta[5] =-0.173936
theta[6] =-0.187268
theta[7] =0.856485
theta[8] =-0.154345
theta[9] =1.835261
theta[10] =0.882547
第15迭代的损失为0.001208
theta[0] =0.709463
theta[1] =0.818261
theta[2] =-0.060977
theta[3] =-0.011892
theta[4] =-0.260592
theta[5] =-0.173037
theta[6] =-0.184286
theta[7] =0.857616
theta[8] =-0.153754
theta[9] =1.837104
theta[10] =0.878869
第16迭代的损失为0.001129
theta[0] =0.709521
theta[1] =0.820571
theta[2] =-0.065623
theta[3] =-0.017119
theta[4] =-0.256326
theta[5] =-0.172097
theta[6] =-0.181603
theta[7] =0.858525
theta[8] =-0.153209
theta[9] =1.838719
theta[10] =0.875768
第17迭代的损失为0.001051
theta[0] =0.709602
theta[1] =0.822683
theta[2] =-0.069891
theta[3] =-0.022211
theta[4] =-0.252094
theta[5] =-0.171134
theta[6] =-0.179179
theta[7] =0.859246
theta[8] =-0.152703
theta[9] =1.840140
theta[10] =0.873152
第18迭代的损失为0.000976
theta[0] =0.709700
theta[1] =0.824621
theta[2] =-0.073828
theta[3] =-0.027156
theta[4] =-0.247921
theta[5] =-0.170159
theta[6] =-0.176979
theta[7] =0.859809
theta[8] =-0.152230
theta[9] =1.841399
theta[10] =0.870945
第19迭代的损失为0.000904
theta[0] =0.709811
theta[1] =0.826405
theta[2] =-0.077473
theta[3] =-0.031948
theta[4] =-0.243825
theta[5] =-0.169183
theta[6] =-0.174975
theta[7] =0.860241
theta[8] =-0.151785
theta[9] =1.842519
theta[10] =0.869080
第20迭代的损失为0.000836
theta[0] =0.709933
theta[1] =0.828054
theta[2] =-0.080857
theta[3] =-0.036581
theta[4] =-0.239819
theta[5] =-0.168212
theta[6] =-0.173141
theta[7] =0.860564
theta[8] =-0.151367
theta[9] =1.843520
theta[10] =0.867503
第21迭代的损失为0.000772
theta[0] =0.710061
theta[1] =0.829582
theta[2] =-0.084011
theta[3] =-0.041055
theta[4] =-0.235915
theta[5] =-0.167252
theta[6] =-0.171456
theta[7] =0.860798
theta[8] =-0.150971
theta[9] =1.844418
theta[10] =0.866167
第22迭代的损失为0.000711
theta[0] =0.710194
theta[1] =0.831003
theta[2] =-0.086957
theta[3] =-0.045368
theta[4] =-0.232121
theta[5] =-0.166308
theta[6] =-0.169903
theta[7] =0.860959
theta[8] =-0.150597
theta[9] =1.845228
theta[10] =0.865034
第23迭代的损失为0.000655
theta[0] =0.710330
theta[1] =0.832329
theta[2] =-0.089716
theta[3] =-0.049521
theta[4] =-0.228442
theta[5] =-0.165385
theta[6] =-0.168465
theta[7] =0.861060
theta[8] =-0.150241
theta[9] =1.845962
theta[10] =0.864072
第24迭代的损失为0.000602
theta[0] =0.710468
theta[1] =0.833567
theta[2] =-0.092305
theta[3] =-0.053517
theta[4] =-0.224881
theta[5] =-0.164483
theta[6] =-0.167132
theta[7] =0.861112
theta[8] =-0.149904
theta[9] =1.846630
theta[10] =0.863253
第25迭代的损失为0.000553
theta[0] =0.710605
theta[1] =0.834728
theta[2] =-0.094741
theta[3] =-0.057357
theta[4] =-0.221442
theta[5] =-0.163606
theta[6] =-0.165890
theta[7] =0.861125
theta[8] =-0.149584
theta[9] =1.847239
theta[10] =0.862554
第26迭代的损失为0.000508
theta[0] =0.710742
theta[1] =0.835818
theta[2] =-0.097036
theta[3] =-0.061046
theta[4] =-0.218123
theta[5] =-0.162755
theta[6] =-0.164730
theta[7] =0.861107
theta[8] =-0.149279
theta[9] =1.847798
theta[10] =0.861956
第27迭代的损失为0.000466
theta[0] =0.710877
theta[1] =0.836844
theta[2] =-0.099202
theta[3] =-0.064586
theta[4] =-0.214926
theta[5] =-0.161931
theta[6] =-0.163646
theta[7] =0.861063
theta[8] =-0.148989
theta[9] =1.848313
theta[10] =0.861444
第28迭代的损失为0.000428
theta[0] =0.711009
theta[1] =0.837811
theta[2] =-0.101249
theta[3] =-0.067983
theta[4] =-0.211849
theta[5] =-0.161134
theta[6] =-0.162628
theta[7] =0.861000
theta[8] =-0.148713
theta[9] =1.848788
theta[10] =0.861003
第29迭代的损失为0.000392
theta[0] =0.711139
theta[1] =0.838723
theta[2] =-0.103186
theta[3] =-0.071240
theta[4] =-0.208889
theta[5] =-0.160364
theta[6] =-0.161672
theta[7] =0.860921
theta[8] =-0.148450
theta[9] =1.849228
theta[10] =0.860623
第30迭代的损失为0.000359
theta[0] =0.711265
theta[1] =0.839586
theta[2] =-0.105022
theta[3] =-0.074362
theta[4] =-0.206046
theta[5] =-0.159623
theta[6] =-0.160771
theta[7] =0.860831
theta[8] =-0.148200
theta[9] =1.849637
theta[10] =0.860294
第31迭代的损失为0.000329
theta[0] =0.711388
theta[1] =0.840403
theta[2] =-0.106763
theta[3] =-0.077353
theta[4] =-0.203316
theta[5] =-0.158909
theta[6] =-0.159922
theta[7] =0.860732
theta[8] =-0.147961
theta[9] =1.850019
theta[10] =0.860008
第32迭代的损失为0.000301
theta[0] =0.711507
theta[1] =0.841177
theta[2] =-0.108416
theta[3] =-0.080218
theta[4] =-0.200696
theta[5] =-0.158221
theta[6] =-0.159119
theta[7] =0.860628
theta[8] =-0.147733
theta[9] =1.850375
theta[10] =0.859759
第33迭代的损失为0.000276
theta[0] =0.711622
theta[1] =0.841911
theta[2] =-0.109986
theta[3] =-0.082961
theta[4] =-0.198183
theta[5] =-0.157561
theta[6] =-0.158360
theta[7] =0.860518
theta[8] =-0.147515
theta[9] =1.850709
theta[10] =0.859541
第34迭代的损失为0.000252
theta[0] =0.711734
theta[1] =0.842608
theta[2] =-0.111480
theta[3] =-0.085588
theta[4] =-0.195774
theta[5] =-0.156927
theta[6] =-0.157641
theta[7] =0.860407
theta[8] =-0.147307
theta[9] =1.851022
theta[10] =0.859350
第35迭代的损失为0.000231
theta[0] =0.711842
theta[1] =0.843270
theta[2] =-0.112900
theta[3] =-0.088102
theta[4] =-0.193466
theta[5] =-0.156318
theta[6] =-0.156959
theta[7] =0.860294
theta[8] =-0.147109
theta[9] =1.851317
theta[10] =0.859181
第36迭代的损失为0.000211
theta[0] =0.711945
theta[1] =0.843900
theta[2] =-0.114253
theta[3] =-0.090507
theta[4] =-0.191254
theta[5] =-0.155734
theta[6] =-0.156312
theta[7] =0.860181
theta[8] =-0.146920
theta[9] =1.851595
theta[10] =0.859031
第37迭代的损失为0.000193
theta[0] =0.712045
theta[1] =0.844499
theta[2] =-0.115541
theta[3] =-0.092809
theta[4] =-0.189136
theta[5] =-0.155174
theta[6] =-0.155698
theta[7] =0.860068
theta[8] =-0.146739
theta[9] =1.851857
theta[10] =0.858898
第38迭代的损失为0.000177
theta[0] =0.712141
theta[1] =0.845070
theta[2] =-0.116768
theta[3] =-0.095011
theta[4] =-0.187108
theta[5] =-0.154637
theta[6] =-0.155114
theta[7] =0.859957
theta[8] =-0.146567
theta[9] =1.852105
theta[10] =0.858778
第39迭代的损失为0.000161
theta[0] =0.712233
theta[1] =0.845613
theta[2] =-0.117938
theta[3] =-0.097118
theta[4] =-0.185167
theta[5] =-0.154123
theta[6] =-0.154558
theta[7] =0.859848
theta[8] =-0.146402
theta[9] =1.852340
theta[10] =0.858671
第40迭代的损失为0.000148
theta[0] =0.712322
theta[1] =0.846131
theta[2] =-0.119053
theta[3] =-0.099133
theta[4] =-0.183309
theta[5] =-0.153630
theta[6] =-0.154029
theta[7] =0.859742
theta[8] =-0.146245
theta[9] =1.852562
theta[10] =0.858574
第41迭代的损失为0.000135
theta[0] =0.712407
theta[1] =0.846625
theta[2] =-0.120117
theta[3] =-0.101060
theta[4] =-0.181531
theta[5] =-0.153158
theta[6] =-0.153526
theta[7] =0.859637
theta[8] =-0.146095
theta[9] =1.852773
theta[10] =0.858487
第42迭代的损失为0.000123
theta[0] =0.712489
theta[1] =0.847096
theta[2] =-0.121132
theta[3] =-0.102903
theta[4] =-0.179830
theta[5] =-0.152707
theta[6] =-0.153046
theta[7] =0.859536
theta[8] =-0.145951
theta[9] =1.852973
theta[10] =0.858407
第43迭代的损失为0.000113
theta[0] =0.712567
theta[1] =0.847544
theta[2] =-0.122100
theta[3] =-0.104665
theta[4] =-0.178202
theta[5] =-0.152274
theta[6] =-0.152589
theta[7] =0.859438
theta[8] =-0.145814
theta[9] =1.853163
theta[10] =0.858334
第44迭代的损失为0.000103
theta[0] =0.712642
theta[1] =0.847973
theta[2] =-0.123025
theta[3] =-0.106350
theta[4] =-0.176646
theta[5] =-0.151860
theta[6] =-0.152153
theta[7] =0.859343
theta[8] =-0.145683
theta[9] =1.853344
theta[10] =0.858267
第45迭代的损失为0.000094
theta[0] =0.712714
theta[1] =0.848382
theta[2] =-0.123907
theta[3] =-0.107961
theta[4] =-0.175157
theta[5] =-0.151465
theta[6] =-0.151737
theta[7] =0.859251
theta[8] =-0.145558
theta[9] =1.853516
theta[10] =0.858205
第46迭代的损失为0.000086
theta[0] =0.712782
theta[1] =0.848772
theta[2] =-0.124750
theta[3] =-0.109501
theta[4] =-0.173733
theta[5] =-0.151086
theta[6] =-0.151341
theta[7] =0.859162
theta[8] =-0.145438
theta[9] =1.853679
theta[10] =0.858148
第47迭代的损失为0.000079
theta[0] =0.712848
theta[1] =0.849144
theta[2] =-0.125554
theta[3] =-0.110974
theta[4] =-0.172372
theta[5] =-0.150724
theta[6] =-0.150962
theta[7] =0.859076
theta[8] =-0.145324
theta[9] =1.853835
theta[10] =0.858096
第48迭代的损失为0.000072
theta[0] =0.712911
theta[1] =0.849500
theta[2] =-0.126322
theta[3] =-0.112382
theta[4] =-0.171070
theta[5] =-0.150377
theta[6] =-0.150601
theta[7] =0.858994
theta[8] =-0.145215
theta[9] =1.853984
theta[10] =0.858046
第49迭代的损失为0.000066
theta[0] =0.712972
theta[1] =0.849839
theta[2] =-0.127056
theta[3] =-0.113728
theta[4] =-0.169825
theta[5] =-0.150046
theta[6] =-0.150257
theta[7] =0.858914
theta[8] =-0.145110
theta[9] =1.854126
theta[10] =0.858001
第50迭代的损失为0.000060
theta[0] =0.713029
theta[1] =0.850163
theta[2] =-0.127757
theta[3] =-0.115015
theta[4] =-0.168635
theta[5] =-0.149729
theta[6] =-0.149928
theta[7] =0.858838
theta[8] =-0.145011
theta[9] =1.854261
theta[10] =0.857958
第51迭代的损失为0.000055
theta[0] =0.713085
theta[1] =0.850473
theta[2] =-0.128427
theta[3] =-0.116245
theta[4] =-0.167497
theta[5] =-0.149426
theta[6] =-0.149613
theta[7] =0.858765
theta[8] =-0.144915
theta[9] =1.854390
theta[10] =0.857918
第52迭代的损失为0.000050
theta[0] =0.713137
theta[1] =0.850769
theta[2] =-0.129066
theta[3] =-0.117420
theta[4] =-0.166409
theta[5] =-0.149136
theta[6] =-0.149313
theta[7] =0.858695
theta[8] =-0.144824
theta[9] =1.854512
theta[10] =0.857880
第53迭代的损失为0.000046
theta[0] =0.713188
theta[1] =0.851052
theta[2] =-0.129677
theta[3] =-0.118544
theta[4] =-0.165369
theta[5] =-0.148859
theta[6] =-0.149027
theta[7] =0.858627
theta[8] =-0.144737
theta[9] =1.854630
theta[10] =0.857845
第54迭代的损失为0.000042
theta[0] =0.713236
theta[1] =0.851322
theta[2] =-0.130261
theta[3] =-0.119619
theta[4] =-0.164375
theta[5] =-0.148594
theta[6] =-0.148753
theta[7] =0.858563
theta[8] =-0.144654
theta[9] =1.854742
theta[10] =0.857811
第55迭代的损失为0.000038
theta[0] =0.713283
theta[1] =0.851580
theta[2] =-0.130819
theta[3] =-0.120646
theta[4] =-0.163424
theta[5] =-0.148341
theta[6] =-0.148492
theta[7] =0.858501
theta[8] =-0.144575
theta[9] =1.854848
theta[10] =0.857780
第56迭代的损失为0.000035
theta[0] =0.713327
theta[1] =0.851826
theta[2] =-0.131352
theta[3] =-0.121627
theta[4] =-0.162515
theta[5] =-0.148098
theta[6] =-0.148242
theta[7] =0.858442
theta[8] =-0.144499
theta[9] =1.854950
theta[10] =0.857750
第57迭代的损失为0.000032
theta[0] =0.713369
theta[1] =0.852061
theta[2] =-0.131861
theta[3] =-0.122566
theta[4] =-0.161647
theta[5] =-0.147867
theta[6] =-0.148003
theta[7] =0.858385
theta[8] =-0.144426
theta[9] =1.855048
theta[10] =0.857722
第58迭代的损失为0.000029
theta[0] =0.713410
theta[1] =0.852286
theta[2] =-0.132348
theta[3] =-0.123462
theta[4] =-0.160817
theta[5] =-0.147646
theta[6] =-0.147775
theta[7] =0.858330
theta[8] =-0.144357
theta[9] =1.855141
theta[10] =0.857695
第59迭代的损失为0.000027
theta[0] =0.713448
theta[1] =0.852501
theta[2] =-0.132813
theta[3] =-0.124320
theta[4] =-0.160023
theta[5] =-0.147434
theta[6] =-0.147557
theta[7] =0.858278
theta[8] =-0.144290
theta[9] =1.855230
theta[10] =0.857670
第60迭代的损失为0.000024
theta[0] =0.713485
theta[1] =0.852707
theta[2] =-0.133258
theta[3] =-0.125139
theta[4] =-0.159264
theta[5] =-0.147232
theta[6] =-0.147349
theta[7] =0.858228
theta[8] =-0.144227
theta[9] =1.855315
theta[10] =0.857645
第61迭代的损失为0.000022
theta[0] =0.713521
theta[1] =0.852903
theta[2] =-0.133682
theta[3] =-0.125922
theta[4] =-0.158539
theta[5] =-0.147039
theta[6] =-0.147150
theta[7] =0.858181
theta[8] =-0.144166
theta[9] =1.855396
theta[10] =0.857623
第62迭代的损失为0.000020
theta[0] =0.713554
theta[1] =0.853091
theta[2] =-0.134088
theta[3] =-0.126671
theta[4] =-0.157846
theta[5] =-0.146854
theta[6] =-0.146960
theta[7] =0.858135
theta[8] =-0.144109
theta[9] =1.855473
theta[10] =0.857601
第63迭代的损失为0.000019
theta[0] =0.713587
theta[1] =0.853270
theta[2] =-0.134476
theta[3] =-0.127386
theta[4] =-0.157184
theta[5] =-0.146677
theta[6] =-0.146779
theta[7] =0.858091
theta[8] =-0.144053
theta[9] =1.855547
theta[10] =0.857580
第64迭代的损失为0.000017
theta[0] =0.713618
theta[1] =0.853441
theta[2] =-0.134847
theta[3] =-0.128070
theta[4] =-0.156550
theta[5] =-0.146508
theta[6] =-0.146605
theta[7] =0.858050
theta[8] =-0.144000
theta[9] =1.855618
theta[10] =0.857560
第65迭代的损失为0.000015
theta[0] =0.713647
theta[1] =0.853605
theta[2] =-0.135201
theta[3] =-0.128724
theta[4] =-0.155945
theta[5] =-0.146347
theta[6] =-0.146439
theta[7] =0.858010
theta[8] =-0.143950
theta[9] =1.855685
theta[10] =0.857542
第66迭代的损失为0.000014
theta[0] =0.713675
theta[1] =0.853762
theta[2] =-0.135540
theta[3] =-0.129349
theta[4] =-0.155366
theta[5] =-0.146193
theta[6] =-0.146281
theta[7] =0.857972
theta[8] =-0.143901
theta[9] =1.855750
theta[10] =0.857524
第67迭代的损失为0.000013
theta[0] =0.713702
theta[1] =0.853911
theta[2] =-0.135864
theta[3] =-0.129946
theta[4] =-0.154813
theta[5] =-0.146045
theta[6] =-0.146129
theta[7] =0.857935
theta[8] =-0.143855
theta[9] =1.855812
theta[10] =0.857507
第68迭代的损失为0.000012
theta[0] =0.713728
theta[1] =0.854054
theta[2] =-0.136173
theta[3] =-0.130517
theta[4] =-0.154285
theta[5] =-0.145904
theta[6] =-0.145985
theta[7] =0.857900
theta[8] =-0.143811
theta[9] =1.855870
theta[10] =0.857490
第69迭代的损失为0.000011
theta[0] =0.713753
theta[1] =0.854191
theta[2] =-0.136468
theta[3] =-0.131062
theta[4] =-0.153780
theta[5] =-0.145770
theta[6] =-0.145846
theta[7] =0.857867
theta[8] =-0.143769
theta[9] =1.855927
theta[10] =0.857475
第70迭代的损失为0.000010
theta[0] =0.713776
theta[1] =0.854321
theta[2] =-0.136751
theta[3] =-0.131584
theta[4] =-0.153297
theta[5] =-0.145641
theta[6] =-0.145714
theta[7] =0.857835
theta[8] =-0.143729
theta[9] =1.855981
theta[10] =0.857460
第71迭代的损失为0.000009
theta[0] =0.713799
theta[1] =0.854446
theta[2] =-0.137021
theta[3] =-0.132082
theta[4] =-0.152835
theta[5] =-0.145518
theta[6] =-0.145588
theta[7] =0.857804
theta[8] =-0.143690
theta[9] =1.856032
theta[10] =0.857446
第72迭代的损失为0.000008
theta[0] =0.713820
theta[1] =0.854565
theta[2] =-0.137279
theta[3] =-0.132559
theta[4] =-0.152394
theta[5] =-0.145400
theta[6] =-0.145467
theta[7] =0.857775
theta[8] =-0.143653
theta[9] =1.856081
theta[10] =0.857432
第73迭代的损失为0.000008
theta[0] =0.713841
theta[1] =0.854679
theta[2] =-0.137526
theta[3] =-0.133014
theta[4] =-0.151973
theta[5] =-0.145288
theta[6] =-0.145352
theta[7] =0.857747
theta[8] =-0.143618
theta[9] =1.856128
theta[10] =0.857419
第74迭代的损失为0.000007
theta[0] =0.713861
theta[1] =0.854788
theta[2] =-0.137762
theta[3] =-0.133449
theta[4] =-0.151570
theta[5] =-0.145180
theta[6] =-0.145241
theta[7] =0.857720
theta[8] =-0.143584
theta[9] =1.856173
theta[10] =0.857407
第75迭代的损失为0.000006
theta[0] =0.713879
theta[1] =0.854892
theta[2] =-0.137987
theta[3] =-0.133865
theta[4] =-0.151184
theta[5] =-0.145078
theta[6] =-0.145136
theta[7] =0.857695
theta[8] =-0.143552
theta[9] =1.856216
theta[10] =0.857395
第76迭代的损失为0.000006
theta[0] =0.713897
theta[1] =0.854992
theta[2] =-0.138202
theta[3] =-0.134263
theta[4] =-0.150816
theta[5] =-0.144980
theta[6] =-0.145035
theta[7] =0.857670
theta[8] =-0.143522
theta[9] =1.856257
theta[10] =0.857384
第77迭代的损失为0.000005
theta[0] =0.713914
theta[1] =0.855087
theta[2] =-0.138408
theta[3] =-0.134643
theta[4] =-0.150464
theta[5] =-0.144886
theta[6] =-0.144939
theta[7] =0.857647
theta[8] =-0.143492
theta[9] =1.856296
theta[10] =0.857374
第78迭代的损失为0.000005
theta[0] =0.713931
theta[1] =0.855178
theta[2] =-0.138605
theta[3] =-0.135006
theta[4] =-0.150128
theta[5] =-0.144796
theta[6] =-0.144847
theta[7] =0.857625
theta[8] =-0.143464
theta[9] =1.856334
theta[10] =0.857363
第79迭代的损失为0.000004
theta[0] =0.713947
theta[1] =0.855265
theta[2] =-0.138793
theta[3] =-0.135353
theta[4] =-0.149807
theta[5] =-0.144710
theta[6] =-0.144759
theta[7] =0.857604
theta[8] =-0.143437
theta[9] =1.856369
theta[10] =0.857354
第80迭代的损失为0.000004
theta[0] =0.713962
theta[1] =0.855348
theta[2] =-0.138972
theta[3] =-0.135685
theta[4] =-0.149499
theta[5] =-0.144628
theta[6] =-0.144675
theta[7] =0.857583
theta[8] =-0.143412
theta[9] =1.856404
theta[10] =0.857344
第81迭代的损失为0.000004
theta[0] =0.713976
theta[1] =0.855427
theta[2] =-0.139144
theta[3] =-0.136002
theta[4] =-0.149206
theta[5] =-0.144550
theta[6] =-0.144594
theta[7] =0.857564
theta[8] =-0.143387
theta[9] =1.856436
theta[10] =0.857335
第82迭代的损失为0.000003
theta[0] =0.713990
theta[1] =0.855503
theta[2] =-0.139308
theta[3] =-0.136305
theta[4] =-0.148925
theta[5] =-0.144475
theta[6] =-0.144517
theta[7] =0.857545
theta[8] =-0.143364
theta[9] =1.856467
theta[10] =0.857327
第83迭代的损失为0.000003
theta[0] =0.714003
theta[1] =0.855576
theta[2] =-0.139465
theta[3] =-0.136595
theta[4] =-0.148657
theta[5] =-0.144404
theta[6] =-0.144444
theta[7] =0.857527
theta[8] =-0.143341
theta[9] =1.856497
theta[10] =0.857319
第84迭代的损失为0.000003
theta[0] =0.714015
theta[1] =0.855645
theta[2] =-0.139615
theta[3] =-0.136872
theta[4] =-0.148400
theta[5] =-0.144335
theta[6] =-0.144374
theta[7] =0.857510
theta[8] =-0.143320
theta[9] =1.856526
theta[10] =0.857311
第85迭代的损失为0.000003
theta[0] =0.714027
theta[1] =0.855711
theta[2] =-0.139759
theta[3] =-0.137136
theta[4] =-0.148155
theta[5] =-0.144270
theta[6] =-0.144307
theta[7] =0.857494
theta[8] =-0.143299
theta[9] =1.856553
theta[10] =0.857303
第86迭代的损失为0.000002
theta[0] =0.714039
theta[1] =0.855774
theta[2] =-0.139896
theta[3] =-0.137389
theta[4] =-0.147921
theta[5] =-0.144207
theta[6] =-0.144243
theta[7] =0.857479
theta[8] =-0.143280
theta[9] =1.856579
theta[10] =0.857296
第87迭代的损失为0.000002
theta[0] =0.714050
theta[1] =0.855835
theta[2] =-0.140027
theta[3] =-0.137631
theta[4] =-0.147697
theta[5] =-0.144148
theta[6] =-0.144181
theta[7] =0.857464
theta[8] =-0.143261
theta[9] =1.856604
theta[10] =0.857289
第88迭代的损失为0.000002
theta[0] =0.714060
theta[1] =0.855893
theta[2] =-0.140152
theta[3] =-0.137862
theta[4] =-0.147483
theta[5] =-0.144091
theta[6] =-0.144123
theta[7] =0.857450
theta[8] =-0.143243
theta[9] =1.856628
theta[10] =0.857283
第89迭代的损失为0.000002
theta[0] =0.714070
theta[1] =0.855948
theta[2] =-0.140271
theta[3] =-0.138083
theta[4] =-0.147278
theta[5] =-0.144036
theta[6] =-0.144067
theta[7] =0.857436
theta[8] =-0.143226
theta[9] =1.856651
theta[10] =0.857277
第90迭代的损失为0.000002
theta[0] =0.714079
theta[1] =0.856001
theta[2] =-0.140386
theta[3] =-0.138294
theta[4] =-0.147083
theta[5] =-0.143984
theta[6] =-0.144013
theta[7] =0.857423
theta[8] =-0.143210
theta[9] =1.856673
theta[10] =0.857271
第91迭代的损失为0.000001
theta[0] =0.714089
theta[1] =0.856051
theta[2] =-0.140495
theta[3] =-0.138496
theta[4] =-0.146896
theta[5] =-0.143934
theta[6] =-0.143962
theta[7] =0.857411
theta[8] =-0.143194
theta[9] =1.856693
theta[10] =0.857265
第92迭代的损失为0.000001
theta[0] =0.714097
theta[1] =0.856100
theta[2] =-0.140599
theta[3] =-0.138689
theta[4] =-0.146718
theta[5] =-0.143887
theta[6] =-0.143913
theta[7] =0.857399
theta[8] =-0.143179
theta[9] =1.856713
theta[10] =0.857260
第93迭代的损失为0.000001
theta[0] =0.714106
theta[1] =0.856146
theta[2] =-0.140699
theta[3] =-0.138873
theta[4] =-0.146547
theta[5] =-0.143841
theta[6] =-0.143867
theta[7] =0.857388
theta[8] =-0.143165
theta[9] =1.856732
theta[10] =0.857255
第94迭代的损失为0.000001
theta[0] =0.714114
theta[1] =0.856190
theta[2] =-0.140795
theta[3] =-0.139049
theta[4] =-0.146384
theta[5] =-0.143798
theta[6] =-0.143822
theta[7] =0.857377
theta[8] =-0.143152
theta[9] =1.856750
theta[10] =0.857250
第95迭代的损失为0.000001
theta[0] =0.714121
theta[1] =0.856232
theta[2] =-0.140886
theta[3] =-0.139217
theta[4] =-0.146228
theta[5] =-0.143756
theta[6] =-0.143779
theta[7] =0.857366
theta[8] =-0.143139
theta[9] =1.856768
theta[10] =0.857245
第96迭代的损失为0.000001
theta[0] =0.714128
theta[1] =0.856272
theta[2] =-0.140973
theta[3] =-0.139378
theta[4] =-0.146079
theta[5] =-0.143716
theta[6] =-0.143739
theta[7] =0.857356
theta[8] =-0.143126
theta[9] =1.856784
theta[10] =0.857240
第97迭代的损失为0.000001
theta[0] =0.714135
theta[1] =0.856311
theta[2] =-0.141056
theta[3] =-0.139532
theta[4] =-0.145936
theta[5] =-0.143678
theta[6] =-0.143700
theta[7] =0.857347
theta[8] =-0.143114
theta[9] =1.856800
theta[10] =0.857236
第98迭代的损失为0.000001
theta[0] =0.714142
theta[1] =0.856348
theta[2] =-0.141136
theta[3] =-0.139679
theta[4] =-0.145800
theta[5] =-0.143642
theta[6] =-0.143662
theta[7] =0.857338
theta[8] =-0.143103
theta[9] =1.856815
theta[10] =0.857232
第99迭代的损失为0.000001
-------train over!!!-----------
theta[0]=0.714142
theta[1]=0.856348
theta[2]=-0.141136
theta[3]=-0.139679
theta[4]=-0.145800
theta[5]=-0.143642
theta[6]=-0.143662
theta[7]=0.857338
theta[8]=-0.143103
theta[9]=1.856815
theta[10]=0.857232
------------------------------
预测值H=2.713385,真实值ANS=3，误差loss=0.000001
预测值H=3.001739,真实值ANS=3，误差loss=0.000001
预测值H=3.001680,真实值ANS=3，误差loss=0.000001
预测值H=5.000312,真实值ANS=5，误差loss=0.000001
预测值H=2.137853,真实值ANS=2，误差loss=0.000001
预测值H=5.000739,真实值ANS=5，误差loss=0.000001
预测值H=5.571539,真实值ANS=3，误差loss=0.000001
预测值H=1.143579,真实值ANS=1，误差loss=0.000001
预测值H=1.993903,真实值ANS=2，误差loss=0.000001
预测值H=-0.001582,真实值ANS=0，误差loss=0.000001
预测值H=1.998542,真实值ANS=2，误差loss=0.000001
In [ ]: