吴恩达机器学习第5周Neural Networks（Cost Function and Backpropagation）

5.1 Cost Function

假设训练样本为：{(x¹⁾,y⁽¹⁾),(x⁽²⁾,y⁽²⁾),...(x^(m),y^(m))}

L  = total no.of layers in network

s_L= no,of units(not counting bias unit) in layer L

K = number of output units/classes

如图所示的神经网络，L = 4，s1 = 3，s2 = 5，s3 = 5, s4 = 4

逻辑回归的代价函数：

            

神经网络的代价函数：

　　　

 

 5.2 反向传播算法 Backpropagation

 关于反向传播算法的一篇通俗的解释http://blog.csdn.net/shijing_0214/article/details/51923547

 

 

 5.3 Training a neural network

 

隐藏层的单元数一般一样，隐藏层一般越多越好，但计算量会较大。

Training a Neural Network

1. Randomly initialize the weights
2. Implement forward propagation to get )
3. Implement the cost function
4. Implement backpropagation to compute partial derivatives
5. Use gradient checking to confirm that your backpropagation works. Then disable gradient checking.
6. Use gradient descent or a built-in optimization function to minimize the cost function with the weights in theta.