计算图片大小_计算图

计算图片大小

计算图 (Computational Graphs)

什么是计算图 (What is Computational Graph)

A computational graph is defined as a directed graph where the nodes correspond to mathematical operations. Computational graphs are a way of expressing and evaluating a mathematical expression.

计算图被定义为有向图，其中节点对应于数学运算。计算图是表达和评估数学表达式的一种方式。

For example, here is a simple mathematical equation −

例如，这是一个简单的数学方程式-

$$p = x+y$$

$$ p = x + y $$

We can draw a computational graph of the above equation as follows.

我们可以如下绘制上述方程的计算图。

The above computational graph has an addition node (node with "+" sign) with two input variables x and y and one output q.

上面的计算图包含一个带有两个输入变量x和y和一个输出q的加法节点(带有“ +”号的节点)。

Let us take another example, slightly more complex. We have the following equation.

让我们再举一个例子，稍微复杂一点。我们有以下等式。

$$g = \left (x+y \right ) \ast z $$

$$ g = \ left(x + y \ right)\ ast z $$

The above equation is represented by the following computational graph.

上式由以下计算图表示。

计算图和反向传播 (Computational Graphs and Backpropagation)

Computational graphs and backpropagation, both are important core concepts in deep learning for training neural networks.

计算图和反向传播都是深度学习中训练神经网络的重要核心概念。

前进通行证 (Forward Pass)

Forward pass is the procedure for evaluating the value of the mathematical expression represented by computational graphs. Doing forward pass means we are passing the value from variables in forward direction from the left (input) to the right where the output is.

前向传递是评估由计算图表示的数学表达式的值的过程。进行正向传递意味着我们将变量的值沿正向从左(输入)传递到输出所处的右侧。

Let us consider an example by giving some value to all of the inputs. Suppose, the following values are given to all of the inputs.

让我们考虑一个示例，它为所有输入提供一些价值。假设以下值被赋予所有输入。

$$x=1, y=3, z=−3$$

$$ x = 1，y = 3，z = -3 $$

By giving these values to the inputs, we can perform forward pass and get the following values for the outputs on each node.

通过将这些值提供给输入，我们可以执行前向传递，并在每个节点上获得输出的以下值。

First, we use the value of x = 1 and y = 3, to get p = 4.

首先，我们使用x = 1和y = 3的值得到p = 4。

Then we use p = 4 and z = -3 to get g = -12. We go from left to right, forwards.

然后我们使用p = 4和z = -3得到g = -12。我们从左到右前进。

后退通行证的目的 (Objectives of Backward Pass)

In the backward pass, our intention is to compute the gradients for each input with respect to the final output. These gradients are essential for training the neural network using gradient descent.

在后向传递中，我们的目的是计算每个输入相对于最终输出的梯度。这些梯度对于使用梯度下降训练神经网络至关重要。

For example, we desire the following gradients.

例如，我们需要以下渐变。

所需的渐变 (Desired gradients)

$$\frac{\partial x}{\partial f}, \frac{\partial y}{\partial f}, \frac{\partial z}{\partial f}$$

$$ \ frac {\ partial x} {\ partial f}，\ frac {\ partial y} {\ partial f}，\ frac {\ partial z} {\ partial f} $$

向后传播(反向传播) (Backward pass (backpropagation))

We start the backward pass by finding the derivative of the final output with respect to the final output (itself!). Thus, it will result in the identity derivation and the value is equal to one.

我们通过找到最终输出相对于最终输出(本身！)的导数来开始向后传递。因此，它将导致同一性推导，并且该值等于1。

$$\frac{\partial g}{\partial g} = 1$$

$$ \ frac {\ partial g} {\ partial g} = 1 $$

Our computational graph now looks as shown below −

我们的计算图现在看起来如下所示-

Next, we will do the backward pass through the "*" operation. We will calculate the gradients at p and z. Since g = p*z, we know that −

接下来，我们将通过“ *”操作进行反向传递。我们将计算p和z处的梯度。由于g = p * z，我们知道-

$$\frac{\partial g}{\partial z} = p$$

$$ \ frac {\ partial g} {\ partial z} = p $$

$$\frac{\partial g}{\partial p} = z$$

$$ \ frac {\ partial g} {\ partial p} = z $$

We already know the values of z and p from the forward pass. Hence, we get −

我们已经从向前传递中知道了z和p的值。因此，我们得到-

$$\frac{\partial g}{\partial z} = p = 4$$

$$ \ frac {\ partial g} {\ partial z} = p = 4 $$

and

和

$$\frac{\partial g}{\partial p} = z = -3$$

$$ \ frac {\ partial g} {\ partial p} = z = -3 $$

We want to calculate the gradients at x and y −

我们要计算x和y处的梯度-

$$\frac{\partial g}{\partial x}, \frac{\partial g}{\partial y}$$

$$ \ frac {\ partial g} {\ partial x}，\ frac {\ partial g} {\ partial y} $$

However, we want to do this efficiently (although x and g are only two hops away in this graph, imagine them being really far from each other). To calculate these values efficiently, we will use the chain rule of differentiation. From chain rule, we have −

但是，我们希望有效地做到这一点(尽管x和g在此图中仅相距两跳，想象它们之间确实距离很远)。为了有效地计算这些值，我们将使用微分的链式规则。根据链式规则，我们有-

$$\frac{\partial g}{\partial x}=\frac{\partial g}{\partial p}\ast \frac{\partial p}{\partial x}$$

$$ \ frac {\ partial g} {\ partial x} = \ frac {\ partial g} {\ partial p} \ ast \ frac {\ partial p} {\ partial x} $$

$$\frac{\partial g}{\partial y}=\frac{\partial g}{\partial p}\ast \frac{\partial p}{\partial y}$$

$$ \ frac {\ partial g} {\ partial y} = \ frac {\ partial g} {\ partial p} \ ast \ frac {\ partial p} {\ partial y} $$

But we already know the dg/dp = -3, dp/dx and dp/dy are easy since p directly depends on x and y. We have −

但是我们已经知道dg / dp = -3，dp / dx和dp / dy很容易，因为p直接取决于x和y。我们有-

$$p=x+y\Rightarrow \frac{\partial x}{\partial p} = 1, \frac{\partial y}{\partial p} = 1$$

$$ p = x + y \ Rightarrow \ frac {\ partial x} {\ partial p} = 1，\ frac {\ partial y} {\ partial p} = 1 $$

Hence, we get −

因此，我们得到-

$$\frac{\partial g} {\partial f} = \frac{\partial g} {\partial p}\ast \frac{\partial p} {\partial x} = \left ( -3 \right ).1 = -3$$

$$ \ frac {\ partial g} {\ partial f} = \ frac {\ partial g} {\ partial p} \ ast \ frac {\ partial p} {\ partial x} = \ left(-3 \ right) .1 = -3 $$

In addition, for the input y −

另外，对于输入y-

$$\frac{\partial g} {\partial y} = \frac{\partial g} {\partial p}\ast \frac{\partial p} {\partial y} = \left ( -3 \right ).1 = -3$$

$$ \ frac {\ partial g} {\ partial y} = \ frac {\ partial g} {\ partial p} \ ast \ frac {\ partial p} {\ partial y} = \ left(-3 \ right) .1 = -3 $$

The main reason for doing this backwards is that when we had to calculate the gradient at x, we only used already computed values, and dq/dx (derivative of node output with respect to the same node's input). We used local information to compute a global value.

向后进行此操作的主要原因是，当我们必须计算x处的梯度时，我们仅使用已经计算出的值和dq / dx(节点输出相对于同一节点输入的导数)。我们使用本地信息来计算全球价值。

训练神经网络的步骤 (Steps for training a neural network)

Follow these steps to train a neural network −

请按照以下步骤训练神经网络-

For data point x in dataset,we do forward pass with x as input, and calculate the cost c as output.

对于数据集中的数据点x，我们将x作为输入进行正向传递，并计算成本c作为输出。
We do backward pass starting at c, and calculate gradients for all nodes in the graph. This includes nodes that represent the neural network weights.

我们从c开始进行反向传递，并计算图中所有节点的梯度。这包括代表神经网络权重的节点。
We then update the weights by doing W = W - learning rate * gradients.

然后，我们通过执行W = W-学习率*梯度来更新权重。
We repeat this process until stop criteria is met.

我们重复此过程，直到满足停止条件。

Previous Page Print Page

上一页打印页面