使用 Scipy 的 fmin_cg 进行逻辑回归答案

【问题标题】：Logistic Regression using Scipy's fmin_cg使用 Scipy 的 fmin_cg 进行逻辑回归
【发布时间】：2015-08-05 04:58:21
【问题描述】：

我正在尝试使用 python 实现逻辑分类器。目标是训练算法使用 mnist 手写数字数据集识别数字 0-9。但是， fmin_cg 似乎正在改变我输入参数的尺寸。我尝试过重塑 cost() 和 gradient() 内部的争论，但没有成功；只是更多的错误。

from scipy.io import loadmat
from numpy import shape, zeros, ones, dot, hstack, vstack, log, transpose, kron
from scipy.special import expit as sigmoid
import scipy.optimize


def cost(theta, X, y):
    h = sigmoid( X.dot(theta) )
    pos_class = y.T.dot( log(h) )
    neg_class = (1.0-y).T.dot( log(1.0-h) )
    cost = ((-1.0/m)*(pos_class+neg_class)) 
    return cost


def gradient(theta, X, y):
    h = sigmoid( X.dot(theta) )
    grad = (1.0/m)*(X.T.dot((h-y)))
    return grad

def one_vs_all(X, y, theta):
    # add x1 feature,x1 = 1, to each example set 
    X = hstack( (ones((m,1)),X) )
    #  train the classifier for digit 9.0
    temp_y = (y == 9.0)+0 
    result          = scipy.optimize.fmin_cg( cost, fprime=gradient, x0=theta, \
                                              args=(X, temp_y), maxiter=50, disp=False, full_output=True )
    print result[1]

# Load data from Matlab file
data = loadmat('data.mat')
X,y = data['X'],data['y']

m,n = shape(X)
theta = zeros((n+1, 1))

one_vs_all(X, y, theta)

我收到的错误：

Traceback (most recent call last):
  File "/Users/jkarimi91/Documents/Digit Recognizer/Digit_Recognizer.py", line 36, in <module>
    one_vs_all(X, y, theta)
  File "/Users/jkarimi91/Documents/Digit Recognizer/Digit_Recognizer.py", line 26, in one_vs_all
    args=(X, temp_y), maxiter=50, disp=False, full_output=True )
  File "/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 1092, in fmin_cg
    res = _minimize_cg(f, x0, args, fprime, callback=callback, **opts)
  File "/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 1156, in _minimize_cg
    deltak = numpy.dot(gfk, gfk)
ValueError: shapes (401,5000) and (401,5000) not aligned: 5000 (dim 1) != 401 (dim 0)
[Finished in 1.0s with exit code 1]

【问题讨论】：

标签： python-2.7 machine-learning scipy logistic-regression

【解决方案1】：

使用当前代码，成本和梯度函数都返回一个二维数组。要使 fmin_cg 正常工作，这些函数必须各自返回一个一维数组（如 documentation 所述）。

【讨论】：

【解决方案2】：

我知道这可能有点晚了，但这应该有效 .在你的梯度函数中我遇到了几个内存错误，所以我稍微改变了代码并添加了正则化，检查一下

def gradients (theta,X,y,Lambda):
    m,n = shape(X)
    theta = reshape(theta,(n,1))
    h = sigmoid(X.dot(theta))
    h = h-y
    theta[0,0] = 0
    grad = ((X.T.dot(h)) / m) + (Lambda / m *  theta)
    return grad.ravel()

【讨论】：