使用 C# 的线性回归梯度下降

Question

我现在正在参加 Coursera 机器学习课程，但无法最小化梯度下降线性回归函数。我使用：一个因变量、一个截距以及 x 和 y 的四个值，因此方程相当简单。 Gradient Decent 方程的最终值根据 alpha 和 beta 的初始值变化很大，我不知道为什么。 我只写了大约两个星期的代码，所以我的知识至少可以说是有限的，如果你花时间提供帮助，请记住这一点。

using System;
namespace LinearRegression
{
class Program
{
    static void Main(string[] args)
    {
        Random rnd = new Random();
        const int N = 4;

        //We randomize the inital values of alpha and beta
        double theta1 = rnd.Next(0, 100);
        double theta2 = rnd.Next(0, 100);

        //Values of x, i.e the independent variable
        double[] x = new double[N] { 1, 2, 3, 4 };
        //VAlues of y, i.e the dependent variable
        double[] y = new double[N] { 5, 7, 9, 12 };
        double sumOfSquares1;
        double sumOfSquares2;
        double temp1;
        double temp2;
        double sum;
        double learningRate = 0.001;
        int count = 0;

        do
        {
            //We reset the Generalized cost function, called sum of squares 
            //since I originally used SS to 
            //determine if the function was minimized
            sumOfSquares1 = 0;
            sumOfSquares2 = 0;
            //Adding 1 to counter for each iteration to keep track of how 
            //many iterations are completed thus far
            count += 1;

            //First we calculate the Generalized cost function, which is
            //to be minimized
            sum = 0;
            for (int i = 0; i < (N - 1); i++)
            {
                sum += Math.Pow((theta1 + theta2 * x[i] - y[i]), 2);
            }
            //Since we have 4 values of x and y we have 1/(2*N) = 1 /8 = 0.125
            sumOfSquares1 = 0.125 * sum;

            //Then we calcualte the new alpha value, using the derivative of 
            //the cost function. 
            sum = 0;
            for (int i = 0; i < (N - 1); i++)
            {
                sum += theta1 + theta2 * x[i] - y[i];
            }
            //Since we have 4 values of x and y we have 1/(N) = 1 /4 = 0.25
            temp1 = theta1 - learningRate * 0.25 * sum;

            //Same for the beta value, it has a different derivative
            sum = 0;
            for (int i = 0; i < (N - 1); i++)
            {
                sum += (theta1 + theta2 * x[i]) * x[i] - y[i];
            }
            temp2 = theta2 - learningRate * 0.25 * sum;

            //WE change the values of alpha an beta at the same time, otherwise the 
            //function wont work                
            theta1 = temp1;
            theta2 = temp2;

            //We then calculate the cost function again, with new alpha and beta values 
            sum = 0;
            for (int i = 0; i < (N - 1); i++)
            {
                sum += Math.Pow((theta1 + theta2 * x[i] - y[i]), 2);
            }
            sumOfSquares2 = 0.125 * sum;

            Console.WriteLine("Alpha: {0:N}", theta1);
            Console.WriteLine("Beta: {0:N}", theta2);
            Console.WriteLine("GCF Before: {0:N}", sumOfSquares1);
            Console.WriteLine("GCF After: {0:N}", sumOfSquares2);
            Console.WriteLine("Iterations: {0}", count);
            Console.WriteLine(" ");

        } while (sumOfSquares2 <= sumOfSquares1 && count < 5000);
        //we end the iteration cycle once the generalized cost function
        //cannot be reduced any further or after 5000 iterations            
        Console.ReadLine();
    }
}
}

Answer 1

代码中有两个错误。

• 首先，我假设您想要遍历数组中的所有元素。所以像这样重做 for 循环：for (int i = 0; i < N; i++)
• 其次，更新theta2 值时，总和计算得不好。根据更新功能应该是这样的：sum += (theta1 + theta2 * x[i] - y[i]) * x[i];

为什么最终值取决于初始值？

因为梯度下降更新步骤是根据这些值计算的。如果初始值（起点）太大或太小，则与最终值（Final Value）。您可以通过以下方式解决此问题：

1. 增加迭代步数（例如 5000 到 50000）：梯度下降算法有更多时间收敛。
2. 降低学习率（例如 0.001 到 0.01）：梯度下降更新步长更大，因此收敛更快。 注意：如果学习率太小，则有可能步入全局最小值。

对于给定数据，斜率 (theta2) 约为 2.5，截距 (theta1) 约为 2.3。我创建了一个 github 项目来修复您的代码，并且还使用 LINQ 添加了一个更短的解决方案。它是 5 行代码。如果您好奇，请查看here。