ML Fundamentals
What Is Machine Learning?
A plain-language introduction to machine learning: what it is, why it exists and how it differs from the step-by-step programming you may already know.
beginner · 13 min read
Lesson content
Machine learning is a way of getting a computer to make decisions by learning from examples, rather than by following instructions you wrote out by hand.
You show it many examples of a problem together with their answers. It works out a pattern. Then it applies that pattern to new cases nobody has answered yet.
That is the whole idea. Everything else in this path is detail about how to do it well.
Definition
Machine learning
Teaching a computer to spot patterns in examples so it can make predictions about new cases it has not seen before.
Formal definition
A field of computer science in which systems improve their performance on a task by learning statistical patterns from data, rather than by executing rules a programmer specified in advance.
Why do we need machine learning at all?
Because some problems cannot be written down as rules, however long you spend trying.
Take recognizing a face. You could describe two eyes above a nose above a mouth, and your rule would fail on a face in profile, in shadow, wearing sunglasses or turned slightly away. Every fix you add creates two new exceptions.
The same is true of translating a sentence, spotting spam or reading handwriting. In each case the pattern is real and people apply it effortlessly, yet nobody can state it.
Machine learning sidesteps the problem. Instead of demanding that someone articulate the rule, it extracts one from thousands of examples where the answer is already known.
Teaching a child to recognize cats
Imagine teaching a toddler what a cat is. You do not hand them a checklist, "four legs, pointy ears, whiskers, meows", because that list also matches a fox, a small dog and a toy.
Instead you point at cats over and over: "cat," "cat," "cat." Nothing is ever defined. After enough examples the child spots a cat they have never seen, including a hairless one that matches almost none of the checklist.
A model learns the same way, from many labeled examples rather than from a rule somebody typed out. And like the toddler, it will be confidently wrong about the first few odd cases it meets.
| Traditional programming | Machine learning | |
|---|---|---|
| You provide | Rules and data | Data and answers |
| The computer produces | Answers | Rules |
| Good fit when | The logic is known and can be written down | The logic is too complex or fuzzy to write down by hand |
| When it breaks | A case the rules never anticipated | A case unlike anything in the training data |
| How you fix it | Edit the rules | Gather better data and retrain |
You provide
- Traditional programming
- Rules and data
- Machine learning
- Data and answers
The computer produces
- Traditional programming
- Answers
- Machine learning
- Rules
Good fit when
- Traditional programming
- The logic is known and can be written down
- Machine learning
- The logic is too complex or fuzzy to write down by hand
When it breaks
- Traditional programming
- A case the rules never anticipated
- Machine learning
- A case unlike anything in the training data
How you fix it
- Traditional programming
- Edit the rules
- Machine learning
- Gather better data and retrain
That table and that figure carry the single most useful idea in this lesson.
In traditional programming you supply the rules, something like if income > 50000 and credit_score > 700: approve, along with the data. The computer applies your rules and hands back answers.
Machine learning flips the arrangement. You supply the data and the answers you already know. The computer works out the rules for itself.
Once it has them it can apply them to brand-new data where nobody knows the answer yet. The rules are the product, not the input.
Example
Approving a loan, two ways
A traditional program says: if income is above $50,000 and credit score is above 700, approve the loan. Some person chose those two numbers, probably in a meeting.
A machine learning model instead reads ten thousand past loans, each one marked "repaid" or "defaulted". It works out for itself which combination of income, credit score, existing debt and employment history actually predicted repayment.
Nobody typed a threshold. The model found whatever thresholds the data supported, including ones no person would have guessed.
This cuts both ways. If the past decisions in that data were biased, the model learns the bias just as faithfully as it learns everything else.
How a model learns, at a high level
Collect examples
Gather many past cases along with their known outcomes, such as emails already marked spam or not spam.
Show the model the examples
The model sees the inputs and their answers together, usually a batch at a time rather than all at once.
Let it find a pattern
It makes a guess, measures how wrong that guess was, and adjusts itself slightly to be less wrong. Then it does that again, thousands of times.
Test on new cases
It is shown examples it has never seen, with the answers hidden, to check whether the pattern it found actually generalizes or was just memorization.
Use it
Once it performs well on unseen cases, it is ready to make predictions on real data. Then you watch it, because the world keeps moving.
| Size (m²) | Bedrooms | Age (yrs) | Price ($1000s) |
|---|---|---|---|
| 70 | 2 | 10 | 200 |
| 96 | 3 | 4 | 294 |
| 52 | 1 | 20 | 120 |
| 88 | 3 | 12 | 262 |
| 120 | 4 | 2 | 377 |
Each row is one past sale. Price is the answer the model is shown while learning. Afterwards, given a new house's size, bedrooms and age with the price hidden, it predicts one.
from sklearn.linear_model import LinearRegression# X holds the inputs for each house: size, bedrooms, ageX = [[70, 2, 10], [96, 3, 4], [52, 1, 20], [88, 3, 12], [120, 4, 2]]# y holds the known answer (price, in $1000s) for each of those housesy = [200, 294, 120, 262, 377]model = LinearRegression()model.fit(X, y) # this is the "learning" stepprint(model.coef_) # [ 2.5 20. -1.5 ]# A new house: 100 m², 3 bedrooms, 10 years oldprint(model.predict([[100, 3, 10]])) # [295.]from sklearn.linear_model import LinearRegressionLinearRegression is one specific, simple kind of model. It is a good first one to meet and very far from the only kind.
X = [[70, 2, 10], [96, 3, 4], [52, 1, 20], [88, 3, 12], [120, 4, 2]]X is the list of inputs, also called features, for each past house. These are the only things the model is allowed to look at.
y = [200, 294, 120, 262, 377]y is the list of known answers, in the same order as X. Row 1 of X goes with row 1 of y, and so on.
model.fit(X, y) # this is the "learning" step.fit(X, y) is where the learning actually happens. The model compares its guesses against y and adjusts itself until they line up.
print(model.coef_) # [ 2.5 20. -1.5 ]The three numbers it recovered: each extra square metre is worth 2.5, each bedroom 20, and each year of age costs 1.5. Nobody wrote those down. They came out of five rows of arithmetic.
print(model.predict([[100, 3, 10]])) # [295.].predict(...) applies what it learned to a house it never saw. 2.5 × 100 + 20 × 3 − 1.5 × 10 = 295.
That example fits too perfectly, on purpose
The five houses above were built to sit exactly on one formula, so the model recovers it precisely and the arithmetic is checkable by hand.
Real data never behaves like that. Two identical houses sell for different prices because one seller was in a hurry. A column is missing for a third of your rows. The relationship bends instead of running straight.
Everything difficult about machine learning lives in that gap, which is why this path continues for seven more lessons after this one.
- Model
The thing that has learned a pattern and can make predictions.
Almost everything in machine learning is about building, training and evaluating a model, so you will see the word constantly.
- Training
The process of showing a model examples so it can learn the pattern.
"Training a model" is the standard phrase for the learning step. It is what
.fit()did in the code above.- Prediction
The model's guess about a new case it has not been told the answer to.
Predictions on unseen cases are the entire point. A model that only works on data it has already seen is useless.
- Data
The examples a model learns from: the inputs, plus their known answers while it is learning.
A model is only ever as good as the data it was shown. This becomes the central concern of almost every later lesson.
When to use
- The rule is too complex, fuzzy, or full of exceptions to write down by hand, such as recognizing handwriting
- You have plenty of real examples with known answers to learn from
- The pattern shifts over time, so a model can be retrained where a program would have to be rewritten
- Being right most of the time is genuinely useful, even though it will never be right always
When to avoid
- A simple fixed rule already solves the problem correctly, such as "charge 8% sales tax"
- You do not have enough real examples to learn a trustworthy pattern from
- You need a decision that is fully explainable and guaranteed, such as a legal eligibility rule
- The cost of a confident wrong answer is higher than the value of many right ones
Thinking machine learning means the computer is thinking or reasoning like a person.
A model finds statistical patterns in examples. It has no understanding of what a cat or a loan is, only of what the patterns in the data looked like.
Assuming you need thousands of examples before machine learning is possible at all.
Some models work with a few hundred. How much you need depends on the problem and how many features it has. The tiny table above is illustrative rather than a minimum.
Believing machine learning always beats a well-written traditional program.
If a simple rule already solves the problem, it is faster, cheaper and far easier to trust. Machine learning earns its place only where rules are not good enough.
Expecting a model to be right every time.
It learned a pattern from a sample, so it is a very good guess rather than a guarantee. Systems built on models need a plan for the cases they get wrong.
Key takeaway
- Traditional programming: you supply rules and data, the computer produces answers.
- Machine learning: you supply data and answers, the computer produces the rules.
- A model learns a pattern from labeled examples during training, then applies it to new data.
- It is worth reaching for when a rule is too fuzzy or complex to write by hand, and not otherwise.
- A model inherits whatever is in its data, including mistakes and bias in past decisions.
- Predictions are good guesses rather than guarantees, so plan for the cases it gets wrong.
Finished this lesson?
Guest progress is saved on this device.