ML Fundamentals
AI vs. Machine Learning vs. Deep Learning
How artificial intelligence, machine learning and deep learning relate: three nested rings rather than three separate fields, each one a specific way of doing the one outside it.
beginner · 12 min read
Lesson content
News articles use "AI", "machine learning" and "deep learning" as though they were interchangeable. They are not.
They are three rings nested inside each other. Each inner one is a specific way of doing the one outside it, which means naming the inner one always implies the outer ones as well.
The distinction is worth ten minutes, because it is the difference between reading a headline and understanding it.
Definition
Artificial intelligence (AI)
The broad goal of building computer systems that perform tasks we normally think require human intelligence: understanding language, recognizing images, playing games, making decisions.
Formal definition
A field of computer science concerned with creating systems capable of tasks that would require intelligence if performed by a human, using any technique that achieves it.
Definition
Machine learning (ML)
One way to build AI. Instead of a person writing down the rules, the system learns a pattern from examples.
Formal definition
A subfield of AI in which a system improves its performance on a task by learning statistical patterns from data, rather than executing rules a programmer specified in advance.
Definition
Deep learning (DL)
One way to do machine learning. It uses a particular kind of model called a neural network, stacked in many layers, so it can learn very complex patterns directly from raw data.
Formal definition
A subfield of machine learning that uses artificial neural networks with many layers to learn hierarchical representations directly from raw or lightly processed data.
Vehicles, cars and electric cars
Every electric car is a car, and every car is a vehicle. The reverse fails in both directions: a bicycle is a vehicle but not a car, and most cars are not electric.
AI, machine learning and deep learning nest exactly the same way. Every deep learning system is machine learning, and every machine learning system is AI.
But plenty of AI uses no learning at all, and plenty of machine learning uses no neural networks.
If AI is the broadest category, why do people say "AI" when they mean machine learning?
Because machine learning, and especially deep learning, is currently the most visible and successful way of building AI. The broad word gets used for the narrow thing out of habit.
It is still worth keeping the distinction straight. A rule-based chess engine and a modern image-recognizing neural network are both AI, and they have almost nothing in common mechanically.
When a product says it uses AI, the useful follow-up question is: does it learn from data, and if so, from whose?
Why does deep learning need so much more data and computing power?
Because it is doing an extra job that simpler machine learning never attempts.
A plain machine learning model is handed features a person already chose: size, age, word count. It only has to work out how to weigh them, which is a comparatively small task.
A deep learning model starts from raw pixel values or raw text. Before it can find the pattern it has to invent the features, discovering for itself that edges matter, then that combinations of edges make shapes, then that combinations of shapes make faces.
That discovery is where the millions of examples and the hardware bill go.
Example
AI without any learning: a chess engine's opening book
Early chess programs played strong opening moves by looking them up in a table a grandmaster had typed in by hand. For the rest of the game they searched forward through possible move sequences and scored the resulting positions.
No examples. No training. No learning of any kind.
It is still AI, because the goal is a task that normally needs human intelligence. There is no machine learning anywhere in it.
Example
Machine learning: a spam filter that learns from labeled email
A spam filter is shown thousands of past emails, each already marked spam or not spam, along with simple features such as how many links or exclamation marks each one contains.
It learns which combinations of those features predict spam. No neural network is required, just a pattern extracted from examples.
Notice what a person still did here: somebody decided that link count and exclamation marks were worth counting in the first place.
Example
Deep learning: a neural network recognizing photos
A photo app that labels pictures as "beach", "dog" or "birthday party" is usually powered by a deep neural network trained on millions of labeled photos.
Unlike the spam filter, nobody hand-picked which pixel patterns matter. The network builds its own layered representations straight from the raw pixels: edges in the early layers, shapes in the middle ones, whole objects near the end.
That progression from edges to objects is not something anyone programmed. It is what the layers converge on when you train them this way, and it is the single most striking thing about deep learning.
| Artificial intelligence | Machine learning | Deep learning | |
|---|---|---|---|
| Scope | Broadest: any technique for human-like task performance | A specific technique: learn patterns from data | A specific kind of ML: many-layered neural networks |
| Can use hand-written rules? | Yes, often | No, it learns the pattern instead | No, it learns the pattern instead |
| Who chooses the features? | Not applicable | A person, before training | The model, during training |
| Typical data needs | Varies, often none | Hundreds to thousands of examples | Thousands to millions of examples |
| Example from this lesson | A chess engine's hard-coded opening book | A spam filter learning from labeled emails | A network recognizing objects in photos |
Scope
- Artificial intelligence
- Broadest: any technique for human-like task performance
- Machine learning
- A specific technique: learn patterns from data
- Deep learning
- A specific kind of ML: many-layered neural networks
Can use hand-written rules?
- Artificial intelligence
- Yes, often
- Machine learning
- No, it learns the pattern instead
- Deep learning
- No, it learns the pattern instead
Who chooses the features?
- Artificial intelligence
- Not applicable
- Machine learning
- A person, before training
- Deep learning
- The model, during training
Typical data needs
- Artificial intelligence
- Varies, often none
- Machine learning
- Hundreds to thousands of examples
- Deep learning
- Thousands to millions of examples
Example from this lesson
- Artificial intelligence
- A chess engine's hard-coded opening book
- Machine learning
- A spam filter learning from labeled emails
- Deep learning
- A network recognizing objects in photos
| Approach | What the model receives | Who chose the features? |
|---|---|---|
| Machine learning, e.g. logistic regression | Link count: 4, exclamation marks: 6, sender known: no | A person, in advance |
| Deep learning, e.g. a network on raw text | The raw sequence of words in the email | Nobody. The network learns which patterns matter |
This is the practical difference between plain machine learning and deep learning: how much of the feature-finding work the model does for you, and therefore how much data it needs before it can do it.
from sklearn.linear_model import LogisticRegression# Each row: [link_count, exclamation_marks, sender_is_known]# A person chose these three features by hand. That choice is# what makes this machine learning rather than deep learning.X = [[4, 6, 0], [0, 0, 1], [8, 3, 0], [1, 0, 1], [6, 5, 0]]y = [1, 0, 1, 0, 1] # 1 = spam, 0 = not spammodel = LogisticRegression()model.fit(X, y)new_email = [[5, 4, 0]] # 5 links, 4 exclamation marks, unknown senderprint(model.predict(new_email)) # [1] — spamfrom sklearn.linear_model import LogisticRegressionLogisticRegression is a simple, well-understood machine learning model. There are no neural network layers anywhere in it.
# Each row: [link_count, exclamation_marks, sender_is_known]
# A person chose these three features by hand. That choice is
# what makes this machine learning rather than deep learning.The three numbers per email are features a person picked in advance. A deep learning approach would delete this step and read the raw email text instead.
X = [[4, 6, 0], [0, 0, 1], [8, 3, 0], [1, 0, 1], [6, 5, 0]]X holds those hand-chosen features for five past emails, one row each.
y = [1, 0, 1, 0, 1] # 1 = spam, 0 = not spamy holds the known answer for each email, in the same order as X.
model.fit(X, y).fit(X, y) is the learning step. The model works out which combination of the three features best separates spam from the rest.
print(model.predict(new_email)) # [1] — spamMany links, many exclamation marks and an unknown sender all point the same way, so the prediction is 1. Five training rows is far too few to trust in practice, and plenty to see the mechanism.
"AI-powered" on a product page tells you almost nothing
The word covers everything from a neural network with a billion parameters to an if statement somebody felt good about.
Because AI is the outermost ring, any claim about it is the weakest claim available. It is true of a chess opening book from 1985.
When it matters, ask the narrower questions instead. Does it learn from data? What data? Can it be retrained when the world changes, and what happens on a case unlike anything it was trained on?
- Neural network
A model built from layers of simple connected units, each doing a small calculation.
It is the specific kind of model that makes deep learning deep. The depth is the number of stacked layers.
- Feature
One measurable input a model is allowed to look at, such as an email's link count or a house's size.
Plain machine learning relies on features a person chose. Deep learning is notable for finding useful ones on its own.
- Representation
The form the data takes inside a model, after it has been transformed from the raw input.
Deep learning is sometimes called representation learning, because learning good internal representations is the thing it is actually doing.
- Subset
A smaller group fully contained inside a bigger one. Every member of the smaller group also belongs to the bigger one.
It is the exact relationship in this lesson's title: deep learning inside machine learning inside AI.
When to use
- Reach for deep learning when you have large amounts of raw data (images, audio, long text) and no good hand-crafted features
- Reach for plain machine learning when your data is already structured into a modest number of meaningful columns
- Reach for rule-based AI with no learning at all when the logic is simple, fixed and fully known in advance
When to avoid
- Do not reach for deep learning on a small tabular dataset. A simpler model usually does as well with far less data and compute
- Do not assume every AI system involves learning from data
- Do not treat "deep learning" and "neural network" as different things. A deep learning model is a neural network with many layers
- Do not pick the approach before looking at the data, because the data is what decides which of the three is even available to you
Treating the three terms as unrelated buzzwords.
They nest. Deep learning is a kind of machine learning and machine learning is a kind of AI, so naming one implies everything outside it.
Assuming deep learning is always the more advanced, better choice.
It needs far more data and compute, and on small structured datasets gradient boosting routinely beats it. Advanced is not the same as appropriate.
Believing "AI" always means a system that learns from data.
A hard-coded chess opening book or a decision tree a person wrote out by hand is AI with no learning in it whatsoever.
Thinking the rings describe how new or advanced something is.
They describe containment, not chronology. Rule-based AI is still the right answer for many problems being solved today.
Key takeaway
- AI is the broadest goal: systems doing tasks that normally need human intelligence, by any technique at all.
- Machine learning is one technique for AI: learning a pattern from examples instead of hand-writing rules.
- Deep learning is one technique for machine learning: many-layered neural networks that learn their own features from raw data.
- Every deep learning system is machine learning and every machine learning system is AI. The reverse fails at both steps.
- The practical dividing line between ML and DL is who chooses the features, and that is what drives the difference in data appetite.
- Because AI is the outermost ring, calling something AI is the weakest true claim you can make about it.
Finished this lesson?
Guest progress is saved on this device.