Lecture 8 | Deep Learning Software
Deep Learning Software
GPUs are really good for parallel things where you need to do many things at the same time.
the dot product of two vectors
CUDA can process deep learning in a efficiencies way. (cuDNN libraries)
GPU vs CPU in practice
Deeplearning framework
The point of deep learning frameworks
Numpy cs PyTorch
Numpy bad:
- can't run on GPU
- Have to compute our own gradients
Tensorflow
Problem: copying weights between CPU / GPU each step
Problem: loss not going down! Assign calls not actually being executed!
TensorFlow: Loss
TensorFlow: Layers
Keras: High-Level Wrapper
TF package
Keras (https://keras.io/)
tf.keras (https://www.tensorflow.org/api_docs/python/tf/keras)
tf.layers (https://www.tensorflow.org/api_docs/python/tf/layers)
tf.estimator (https://www.tensorflow.org/api_docs/python/tf/estimator)
tf.contrib.estimator (https://www.tensorflow.org/api_docs/python/tf/contrib/estimator)
tf.contrib.layers (https://www.tensorflow.org/api_docs/python/tf/contrib/layers)
tf.contrib.slim (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/slim)
tf.contrib.learn (https://www.tensorflow.org/api_docs/python/tf/contrib/learn)
TFLearn (http://tflearn.org/)
TensorLayer (http://tensorlayer.readthedocs.io/en/latest/)
PyTorch
PyTorch: Tensors
PyTorch: Autograd
PyTorch: New Autograd Functions
PyTorch: nn
PyTorch: opti
PyTorch: nn Define new Modules
PyTorch: DataLoaders
PyTorch: Pretrained Models
PyTorch: Visdom
PyTorch: Dynamic Computation Graphs
Static vs Dynamic
Static vs Dynamic: Graphs
Static vs Dynamic: Conditional
Static vs Dynamic: Loops
Dynamic Graph Applications
PyTorch vs TensorFlow, Static vs Dynamic
Caffe