Prerequisites for Deep Learning
1.Knowledge of calculus and linear algebra, Math 53/54 or equivalent.
2.Probability and Statistics, CS70 or Stat 134. CS70 is bare minimum preparation, a stat course is better.
3.Machine Learning, CS189.
4.Programming, CS61B or equivalent. Assignments will mostly use Python.
Deep Learning
A machine learning subfield of learning representations of data. Exceptional effective at learning patterns. Deep learning algorithms attempt to learn (multiple levels of) representation by using a hierarchy of multiple layers. If you provide the system tons of information, it begins to understand it and respond in useful ways.
Before going into deep learning, We will take a look at Types of Learning
1.Supervised: Learning with a labeled training set
Example: email classification with already labeled emails
2.Unsupervised: Discover patterns in unlabeled data
Example: cluster similar documents based on text
3.Reinforcement learning: learn to act based on feedback/reward
Example: learn to play Go, reward: win or lose
Anomaly Detection
Sequence labeling
Machine Learning Verses Deep Learning
Most machine learning methods work well because of human-designed representations and input features, ML becomes just optimizing weights to best make a final prediction
Why Deep Learning is useful?
1.Manually designed features are often over-specified, incomplete and take a long time to design and validate
2.Learned Features are easy to adapt, fast to learn
3.Deep learning provides a very flexible, (almost?) universal, learnable framework for representing world, visual and linguistic information.
4.Can learn both unsupervised and supervised
5.Effective end-to-end joint system learning
6.Utilize large amounts of training data
In ~2010 DL started outperforming other ML techniques first in speech and vision, then NLP
State of Art in NLP
Several big improvements in recent years in NLP (Natural Language Processing)
* Machine Translation
* Sentiment Analysis
* Dialogue Agents
* Question Answering
* Text Classification
Leverage different levels of representation
* words & characters
* syntax & semantics
Neural Network Intro
Application Example: Relation Extraction from text
Useful for:
- knowledge base completion
- social media analysis
- question answering
References
- http://web.stanford.edu/class/cs224n
- https://www.coursera.org/specializations/deep-learning
- https://chrisalbon.com/#Deep-Learning
- http://www.asimovinstitute.org/neural-network-zoo
- https://www.google.co.in