Summary
Deep Learning summary
- Course
- Institution
Deep Learning summary of lecture slides and notes/extra information from lectures. The summary includes a lot of pictures to make content more clear.
[Show more]
1 review
By: lx_4820 • 1 month ago
Seller
Follow
liekebuuron
Reviews received
Content preview
Lecture 1Training set – for training the algorithm:
Training set: train the algorithm
Validation set: optimize hyperparameters
Test set: evaluate performance on the test set
Introduction:
What is deep learning:
AI:
-
Hard-code knowledge about the world in formal languages.
-
People struggle to devise formal rules with enough complexity to accurately describe
world knowledge.
Machine learning:
- Acquire their own knowledge by extracting patterns from raw data.
- Performance of simple ML algorithms depends heavily on the representation of the
data (features).
- AI is larger than machine learning because you might also hard code, but in machine
learning, the machine itself learns this kind of information.
- Feature extraction is a crucial step. What might lead to the price of a house. Square
meter, number of rooms, garden, etc. These all give an idea about the price of the
house. These are the features. Feature extraction is very important because it has
impact on how good the target variable is predicted. If we fail to extract reasonable
features, we fail to predict accurately.
,Representation learning:
- Use ML not only to discover mappings from representation to output, but to learn
the representation itself.
Deep learning:
- Solves problem representation learning: introduces representations that are
expressed in terms of other, simpler representations.
- Enables computer to build complex concepts out of simple ones.
- Depth enables the computer to learn a multistep computer program.
History of deep learning: apparently ‘new’ technology with a ‘history’:
The 3 waves of development:
- 1940 – 1960: Cybernetics
o McCulloch-Pitts Neuron: early model of brain function. A neuron itself does
not learn, but when they come together, they can do a lot.
o Perceptron: digital version of a neuron.
o Adaptive linear element (ADLINE): predicting continuous variables
- 1980 – 1990: Connectionism
o Distributed representation
o Backpropagation: essence of learning in the computers. This is where the
algorithm updates its parameters.
- 2006 - …: Deep learning: it has many layers
Artificial neural networks (ANNs):
- Engineered systems inspired by the brain
- One of the names DL has gone by
Motivation:
1. The proof by example that intelligent behavior is possible –> reverse engineer the
computational principles behind it and duplicate its functionality.
2. ML models that help us understand the principles that underlie human intelligence
→ shed light on basic scientific questions.
Deep learning (DL) vs ANNs:
- Appeals to more general principle of learning multiple levels of composition (i.e.,
multiple layers that create ‘depth’).
,Layered structure – primate brain:
The things we see with our eyes goes to the v1 which is the edges of the first hidden layer of
the DL network. Then it goes through more layers and eventually it categorizes. PFC is the
region where we do the cognitive decision. When we have the decision, it is sent to the
motor cortex and then that sends it to the muscles. So, the deep learning layers is inspired
by this structure. Each layer becomes more complex and in the end a decision is made.
We expect a complex network, with millions of parameters to be optimized. It first extracts
complex features and then it makes a decision. It needs to optimize the features. So, it
needs big data.
, CPU vs GPU:
DL frameworks (libraries that you can use):
The perceptron:
What is a perceptron:
- Most basic single-layer NN → typically used for binary classification problems (1 or 0,
yes or no)
- Data needs to be linearly separable (if the decision boundary is non-linear, the
perceptron can’t be used)
- Goal: find a line that splits the data/observations
Bias term: describes the threshold. If the weighted sum of the input is higher or equal to the
threshold, it fires, it is 1. The line that you fit should always go through the original (0,0)
location when not using a bias term). If you do use a bias term, you can fit your line at
different line intersections.
Activation function (step activation function):
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller liekebuuron. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $4.89. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
75323 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now