Connection Mode of Neural Network 神经网络的连接方式
- According to the connection range between nodes, the neural network can be divided into fully connected neural network and partially connected neural network.
根据单元之间的连接范围，可以把神经网络分为全连接神经网络和部分连接神经网络。 -
Fully connected neural network: Each node is connected to all the nodes of the adjacent layer.
在全连接神经网络中，每个单元和相邻层的所有单元都相连。 -
Partially connected neural network: Each node is only connected to partial nodes on the adjacent layer.
在部分连接神经网络中，每个单元只与相邻层上的部分单元相连。
- 根据层次之间的连接方式，可以把神经网络分为前馈型神经网络和反馈型神经网络。
- 在前馈型神经网络中，连接是单向的，上层神经元的输出是下层神经元的输入。BP神经网络是一种前馈型神经网络。
- 在反馈型神经网络中，除了单向连接之外，最后一层神经元的输出返回去作为第一层神经元的输入。 如：Hopfield神经网络
Structures of Neural Networks 神经网络的结构
Feedforward neural network 前馈神经网络
information moves in only one direction, forward, from input nodes, through hidden nodes and to the output nodes.
信息从输入结点仅仅以一个方向，即前进方向，穿过隐藏层并抵达输出节点。
Recurrent neural network 循环神经网络
- connections form a directed cycle. 连接形成有向循环。
- Creating an internal state of the network which allows it to exhibit dynamic temporal behavior.
建立网络的内部状态，使之展现动态的时间特性。
Multilayer Feedforward Neural Network （多层前馈神经网络）
- Multilayer feedforward neural network is also called multilayer feedforward fully-connected network. 多层前馈神经网络又称为多层前馈全连接网。
- Multiple layers means that there are one or more hidden layers besides the input and output layers. 多层是指：除了输入层和输出层以外，还存在一个或者多个隐含层。
-
Feedforward means that there is no reverse propagation of the signal from the input layer to the output layer via the hidden layer(s).
前馈是指：外界信号从输入层，经由隐含层到达输出层，不存在信号的逆向传播。 - Fully connected means that each layer of neurons and the next layer of neurons are all connected to each other, there is no connection among the neurons in the same layer, and there is no cross-layer connection.
全连接是指：每层神经元与下一层神经元全部互相连接，同层神经元之间不存在连接，也不存在跨层连接。 - The expression ability of multilayer feedforward neural network is much better than that of single layer sensor.
多层前馈神经网络的表达能力比单层感知机要强得多。 - In order to learn multilayer feedforward neural networks, the learning rules of single layer perceptron are far from enough, and more powerful learning algorithms are needed. In 1974 Werbos proposed BP algorithm.
要学习多层前馈神经网络，单层感知机的学习规则是远远不够的，需要更强大的学习算法。1974年Werbos提出了BP算法. - The algorithm of Backward propagation of errors （BP ）is one of the most outstanding algorithms. It is the most successful neural network learning algorithm so far.
反向误差传播（简称BP）算法是其中最杰出的代表，它是迄今最成功的神经网络学习算法。