如何将深度学习梯度下降方程转换为python

Question

我一直在关注深度学习的在线教程。它有一个关于梯度下降和成本计算的实际问题，一旦将其转换为 python 代码，我一直在努力获得给定的答案。希望您能帮我得到正确的答案

有关使用的方程式，请参阅以下链接 Click here to see the equations used for the calculations

以下是计算梯度下降、成本等的函数。需要在不使用 for 循环但使用矩阵操作操作的情况下找到值

import numpy as np

def propagate(w, b, X, Y):
"""
Arguments:
w -- weights, a numpy array of size (num_px * num_px * 3, 1)
b -- bias, a scalar
X -- data of size (num_px * num_px * 3, number of examples)
Y -- true "label" vector (containing 0 if non-cat, 1 if cat) of size
  (1, number of examples)

Return:
cost -- negative log-likelihood cost for logistic regression
dw -- gradient of the loss with respect to w, thus same shape as w
db -- gradient of the loss with respect to b, thus same shape as b

Tips:
- Write your code step by step for the propagation. np.log(), np.dot()
"""

m = X.shape[1]


# FORWARD PROPAGATION (FROM X TO COST)
### START CODE HERE ### (≈ 2 lines of code)
A =                                      # compute activation
cost =                                   # compute cost
### END CODE HERE ###


# BACKWARD PROPAGATION (TO FIND GRAD)
### START CODE HERE ### (≈ 2 lines of code)
dw = 
db = 
### END CODE HERE ###


assert(dw.shape == w.shape)
assert(db.dtype == float)
cost = np.squeeze(cost)
assert(cost.shape == ())

grads = {"dw": dw,
         "db": db}

return grads, cost

以下是测试上述功能的数据

w, b, X, Y = np.array([[1],[2]]), 2, np.array([[1,2],[3,4]]), 
np.array([[1,0]])
grads, cost = propagate(w, b, X, Y)
print ("dw = " + str(grads["dw"]))
print ("db = " + str(grads["db"]))
print ("cost = " + str(cost))

以下是上述的预期输出

Expected Output:
dw  [[ 0.99993216] [ 1.99980262]]
db  0.499935230625
cost    6.000064773192205

对于上面的传播函数，我使用了下面的替换，但输出不是预期的。请帮助如何获得预期的输出

A = sigmoid(X)
cost = -1*((np.sum(np.dot(Y,np.log(A))+np.dot((1-Y),(np.log(1-A))),axis=0))/m)
dw = (np.dot(X,((A-Y).T)))/m
db = np.sum((A-Y),axis=0)/m

以下是用于计算 Activation 的 sigmoid 函数：

def sigmoid(z):
  """
  Compute the sigmoid of z

  Arguments:
  z -- A scalar or numpy array of any size.

  Return:
  s -- sigmoid(z)
  """

  ### START CODE HERE ### (≈ 1 line of code)
  s = 1 / (1+np.exp(-z))
  ### END CODE HERE ###

return s

希望有人可以帮助我理解如何解决这个问题，因为如果不理解这一点，我就无法继续学习其余的教程。非常感谢

Answer 1

您可以按如下方式计算 A,cost,dw,db：

A = sigmoid(np.dot(w.T,X) + b)     
cost = -1 / m * np.sum(Y*np.log(A)+(1-Y)*np.log(1-A)) 

dw = 1/m * np.dot(X,(A-Y).T)
db = 1/m * np.sum(A-Y)

sigmoid 在哪里：

def sigmoid(z):
    s = 1 / (1 + np.exp(-z))    
    return s

Answer 2

看了几遍代码和注释，终于找出了错误。

首先它需要计算 Z，然后将其传递给 sigmoid 函数，而不是 X

Z = w(T)X+b 的公式。所以在python中计算如下

Z=np.dot(w.T,X)+b

然后通过将 z 传递给 sigmoid 函数来计算 A

A = sigmoid(Z)

那么dw可以计算如下

dw=np.dot(X,(A-Y).T)/m

其他变量的计算； b的成本和导数如下

cost = -1*((np.sum((Y*np.log(A))+((1-Y)*(np.log(1-A))),axis=1))/m) 
db = np.sum((A-Y),axis=1)/m

Answer 3

def sigmoid(x):
      #You have it right
      return 1/(1 + np.exp(-x))

def derivSigmoid(x):
      return sigmoid(x) * (1 - sigmoid(x))

error = targetSample - output

#Make sure to keep the sigmoided value around.  For instance, an output that has already been sigmoided can be used to get the sigmoid derivative faster (output = sigmoid(x)):
dOutput = output * (1 - output)

看起来您已经在处理反向传播了。只是想我会帮你简化一些前向道具。