神经网络的前向传播

Question

我正在尝试在 Python 3.8.2 中创建一个前向传播函数。输入如下所示：

Test_Training_Input = [(1,2,3,4),(1.45,16,5,4),(3,7,19,67)]
Test_Training_Output = [1,1,0]

我没有使用偏差（不确定它们是否那么重要并且它使我的代码非常复杂）但我正在使用权重。权重存储在一个列表中，Layer1W，我不确定要完成多长时间，但我认为，len(Test_Training_Input)+len(Test_Training_Output) 应该可以工作。

到目前为止，我的函数如下所示：

def forwardprop():
    global Layer1O
    Layer1O = []
    for init in range(0,len(Layer1W)):
        total = sum(Test_Training_Input[1][1])*Layer1W[init]
        Layer1O.append(relu(total))
    return Layer1O

我认为这是非常错误的...... 有什么建议吗？

Answer 1

这是对一个相当复杂的主题的非常粗略的解释，我强烈推荐这个e-book

import numpy as np
import numpy.random as rnd

#inputs
x = np.array([[0, 1, 0],
             [1, 0, 1]])

# x is of shape 2 x 3

#Here is a 3 -> 2 network, layer 1 (L1) will have 3 neurons, layer 2 (L2) will have 2
#   n00 is the first neuron of layer 1
#   n01 is the second neuron of layer 2
#   n02 is the third neuron of layer 3

#   n10 is the first neuron of layer 2
#   n11 is the second neuron of layer 2

#so there will be 6 weights, (let us assume their values also)
#   W(n00->n10) = 0.1, W(n00->n11) = 0.2
#   W(n01->n10) = 0.3, W(n01->n11) = 0.4
#   W(n02->n10) = 0.5, W(n02->n11) = 0.6

# we will store the corresponding weights in this fashion
# you might wonder why make it 2 x 3 matrix, and not 3 x 2
# this is to make use of matrix multiplication

#                 n00   n01  n02
w =    np.array([[0.1, 0.3, 0.5],  #n10
                 [0.2, 0.4, 0.6]]) #n11

# so the weight between n02 and n11 = w[1, 2] = 0.6

# note the input should be a vector,
# for example if the input is like this x = [1,1,0], shape = 3
# we will feed it as a vector, i.e
# [[1], <- x0
#  [1], <- x1
#  [0]] <- x2
# shape =  3 x 1

# so there will be 2 outputs from the feedforward, given by
#o1 = w00*x0 + w01*x1 + w02*x2
#o2 = w10*x0 + w11*x1 + w12*x2

# this is just the matrix multiplication between w and x,
#
#   [[0.1, 0.3, 0.5],   X  [[1],
#    [0.2, 0.4, 0.6]]      [1],     =    [[o1],
#                          [0]]           [o2]]
#

def feedforward(w, x):
    z = np.matmul(w, x)
    return z

# we will make x a vector, i.e change the shape as 2 x (3 x 1)
x = x[:, :, np.newaxis]
print(x.shape)
print(feedforward(w, x[0]))


# for your example
x = np.array([[1,2,3,4],[1.45,16,5,4],[3,7,19,67]])
# make it to 3 x 4 x 1
x = x[:, :, np.newaxis]

#lets try a 4 -> 2 -> 1 network

w1 = rnd.normal(size=(2, 4))
w2 = rnd.normal(size=(1, 2))

wgt = [w1, w2]

def feedforward2(wght, x):
    # best practice would be to put this in a class and store the outputs of each layer
    for w in wght:
        x = np.matmul(w, x)
    return x 

for i in x:
    print(feedforward2(wgt, i))

Answer 2

所以通常在该函数中初始化任何变量都不是一个好主意。初始化所有变量，可能在函数之前的构造函数中。我还建议在编写之前手动创建或映射神经网络的结构。它可能有助于将您需要做的事情形象化。

如果您想尝试进行前向传播，您需要分解这些步骤，它的作用是什么？首先，您确定您在哪一层，如果您在输入上，然后将输入矩阵乘以下一个权重，等等。这应该可以更容易地编写您的网络并减少您可能犯的错误数量。

Answer 3

研究使用 numpy，一个使用矩阵的库。像前向传播这样的东西可以很容易地实现，比如：

import numpy as np

for layer in layers:
  inputs = np.dot(inputs, layer)

# this returns the outputs after propogating

这只是一个简单的例子，我排除了一堆东西，比如在传播期间缓存每一层的输入。