李宏毅ML lecture-16 unsupervised Learning Auto-encoder
Auto-encoder
两个神经网络，一个降维，压缩（compact）即Encoder, 把图片转换为code。一个根据code重建（reconstruct）原始的图片。
这两个神经网络单独无法训练，必须同时训练。
Recap: PCA
训练一个三层神经网络实现PCA，使输入和输出接近，同时存在一个Bottleneck（瓶颈） later layer。这个layer的参数小于input layer的参数，这样就实现了encode和decode。
Deep Auto-encoder
增加一些隐藏层，就实现了Deep Auto-encoder.
Reference: Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. “Reducing the dimensionality of data with neural networks.” Science 313.5786 (2006): 504-507
Deep Auto-encoder 结果非常好
提高auto-encoder的能力，可以选择增加噪声
Ref: Rifai, Salah, et al. "Contractive auto-encoders: Explicit invariance during feature extraction.“ Proceedings of the 28th International Conference on Machine Learning (ICML-11). 2011.
Vincent, Pascal, et al. “Extracting and composing robust features with denoising autoencoders.” ICML, 2008.
Vector Space Model
搜索图片
直接搜索效果并不好
Reference: Krizhevsky, Alex, and Geoffrey E. Hinton. “Using very deep autoencoders for content-based image retrieval.” ESANN. 2011.
建立一个auto-encoder
在code上搜寻
pre-training
监督学习使用的不多了。
可以用于半监督学习，非标注数据大量的时候。
Learning More - Restricted Boltzmann Machine
- Neural networks [5.1] : Restricted Boltzmann machine – definition https://www.youtube.com/watch?v=p4Vh_zMw-HQ&index=36&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH
- Neural networks [5.2] : Restricted Boltzmann machine – inference https://www.youtube.com/watch?v=lekCh_i32iE&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&index=37
- Neural networks [5.3] : Restricted Boltzmann machine - free energy https://www.youtube.com/watch?v=e0Ts_7Y6hZU&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&index=38
Learning More - Deep Belief Network
- Neural networks [7.7] : Deep learning - deep belief network https://www.youtube.com/watch?v=vkb6AWYXZ5I&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&index=57
- Neural networks [7.8] : Deep learning - variational bound https://www.youtube.com/watch?v=pStDscJh2Wo&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&index=58
- Neural networks [7.9] : Deep learning - DBN pre-training https://www.youtube.com/watch?v=35MUlYCColk&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH&index=59
auto-encoding 在CNN上
自动编码机
反池化
反卷积