具有两个特征的梯度下降计算特征空间

Question

我尝试为形状的线性回归计算最佳权重 W[0] 和 W[1]：

prize=W0*dummy+W1*size

地点：

dummy=[1,1,1,1,1,1,1,1,1,1]
size=[500,550,620,630,665,700,770,880,920,1000]

目标特征 (rental_prize) 具有以下值：

rental_price=[320,380,400,390,385,410,480,600,570,620]

使用以下代码，我尝试计算 W0 和 W1，以使线性回归最适合数据。

# descriptive features
size=[500,550,620,630,665,700,770,880,920,1000]
dummy=[1,1,1,1,1,1,1,1,1,1]

# Vector which contains the descriptive features
features=[dummy,size]

# target feature
rental_price=[320,380,400,390,385,410,480,600,570,620]

# Set the learning rate alpha
alpha=0.002 

# Feature weight vector --> model=[W0,W1]
# Set initial values for W0 and W1
model=[0,0]

for i in range(len(model)):
    for j in range(100):
        errordelta=np.sum([(rental_price[x]-(model[0]*dummy[x]+model[1]*size[x]))*features[i][x] for x in range(len(size))])
        model[i]=model[i]+alpha*errordelta

print(model[0])
print(model[1])

395.09179229

南

该模型实际上应该返回大约 6.47 的 W0 和 0.62 的 W1。如果我更改 alpha 以及初始权重和迭代 (j) 的值，模型仍然没有接近所需值...

显然代码中一定有故障....

谁能帮帮我。

Answer 1

您的算法存在三个错误：

• 我不知道你为什么将绝对误差乘以特征值；这使误差函数成为二次函数，但您没有使用任何 sqrt 进行补偿。
• 同样，您在调整体重之前未能平均误差量；这有效地将变化放大了一个等于训练行数的因子。
• 您的循环顺序颠倒了：您想依次训练特征，每个迭代一次。您首先训练截距dummy，然后尝试将该值作为绝对值，然后再训练斜率size。您需要交替使用它们。

更新后（以及一些文本改进）：

for j in range(100):
    for i in range(len(model)):
        errordelta  =  np.sum([(rental_price[x] -
                                 (model[0]*dummy[x] + model[1]*sqft[x])) 
                             for x in range(len(sqft))]) / len(sqft)
        print(model, errordelta)
        model[i]  =  model[i] + alpha*errordelta

print(" sqft coeff", model[0])
print("dummy coeff", model[1])

输出：

[0, 0] 455.5
[0.91100000000000003, 0] 454.589
[0.91100000000000003, 0.90917800000000015] -203.201283
[0.5045974339999999, 0.90917800000000015] -202.794880434
[0.5045974339999999, 0.50358823913200013] 90.649311554
[0.68589605710799573, 0.50358823913200013] 90.4680129309
...
[0.62996765105739105, 0.62870771575527662] -1.70530256582e-14
[0.62996765105739105, 0.62870771575527662] -1.70530256582e-14
 sqft coeff 0.629967651057
dummy coeff 0.628707715755

Answer 2

#descriptive features
size=[500,550,620,630,665,700,770,880,920,1000]
dummy=[1,1,1,1,1,1,1,1,1,1]

#Vector which contains the descriptive features
features=[dummy,size]


#target feature
rental_prize=[320,380,400,390,385,410,480,600,570,620]


#########Gradient decent Algorithm#############



#Set the learning rate alpha
alpha=0.00000002

#Feature weight vector --> model=[W0,W1]
#Set initail values for W0 and W1
model=[-0.146,0.185]


#Sum Squared Error
scatterSSE=[]



for j in range(100):

    #Squared Error
    SSE=np.sum([(rental_prize[x]-(model[0]*features[0][x]+model[1]*features[1][x]))**2 for x in range(len(rental_prize))])
    scatterSSE.append(SSE)

    for i in range(len(model)):       

        #Updating the weight factors w[i]
        errorDelta=np.sum([(rental_prize[x]-(model[0]*features[0][x]+model[1]*features[1][x]))*features[i][x] for x in range(len(rental_prize))])
        model[i]=model[i]+alpha*errorDelta





#Linear Equation after 100 itarations        
print("Linear Equation after 100 iterations:","HOUSE PRIZE={0}+{1}*SIZE".format(model[0],model[1]),sep="\n")




#########Plot the results######        

fig= plt.figure(figsize=(20,10))
ax = fig.add_subplot(131)
ax1 = fig.add_subplot(132)

#Plot the SSE 
y=list(range(1,101,1))
ax.scatter(y,scatterSSE)
ax.set_title("SSE")
ax.set_ylabel("SSE")
ax.set_xlabel("Iterations")


#Plot the linear regression
ax1.scatter(size,rental_prize)

X=list(np.linspace(min(size),max(size),100))
y=[model[0]+model[1]*x for x in X]
ax1.plot(X,y,"red")
ax1.set_title("Linear Regression")
ax1.set_ylabel("Prize")
ax1.set_xlabel("Size")




plt.show()

100次迭代后的线性方程：

房屋奖=-0.145+0.629*大小