Building Neural Networks with Keras
Artificial Neural Network
Pre-Trained Networks
Introducing Keras
- Xception, VGG16, VGG19, ResNet50, InceptionV3 (Images)
- MusicTaggerCRNN (Audio)
- Runs on top of TensorFlow or Theano
- Allows for fast prototyping
- Runs on CPU or GPU
Image from https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html
Photo credit: http://kotsanas.com/gb/exh.php?exhibit=2103008
Figures from: https://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
Resources
Guiding Principles
Data Manipulation
Hyperparameters are fun!
- Modularity
- Minimalism
- Easy extensibility
- Python!
- CIFAR10 small image classification
- IMDB Movie reviews sentiment classification
- MNIST database of handwritten digits
- Reuters newswire topics classification
- Sequence, Text and Image Preprocessing
Tutorial from: http://machinelearningmastery.com/tutorial-first-neural-network-python-keras/
Keras Documentation: https://keras.io/
This book is awesome: http://neuralnetworksanddeeplearning.com/
- Network Structure
- Model initialization
- Activation Functions
- Loss Function
- Learning "optimizer"
- Number of iterations
- Mini-batch size
Let's See One!
Cool Stuff They've Done
Lots of Types of Layers
- Google DeepMind (AlphaGo)
- IBM Watson (Oncology)
- Even Kristen Stewart! (Neural Style Transfer)
- So many core!
- Convolutional
- Recurrence
- Write your own!
Code for neural network diagram:
https://gist.github.com/craffel/2d727968c3aaebd10359
https://arxiv.org/pdf/1701.04928v1.pdf