Supervised Learning
Supervised Learning
Supervised learning is the branch of machine learning where the model learns from examples whose correct answer is already known, using features to predict a label.
beginner · 12 min read
Lesson content
Supervised learning is the most common branch of machine learning and the one most people picture when they hear "machine learning" at all. It covers everything from predicting house prices to filtering spam. The core idea is simple: the model learns from examples where the correct answer is already known.
Definition
Supervised learning
Showing a model many past examples, each paired with the correct answer, so it can learn to predict the answer for new examples where the answer is hidden.
Formal definition
A machine learning approach in which a model is trained on a labeled dataset, pairs of inputs and known correct outputs, to learn a mapping it can apply to new, unlabeled inputs.
Studying with an answer key
Imagine practicing for a test with a stack of flashcards: the question on the front, the correct answer on the back.
You make a guess, flip the card, compare, and adjust your understanding a little. Do this with enough cards and you start answering questions you have never seen, because you learned the pattern behind the answers rather than the cards themselves.
That is supervised learning. The answer on the back of each card is the label, and the whole method depends on having it.
The analogy also warns about the classic failure. Drill the same twenty cards long enough and you can recite them perfectly while understanding nothing, which is what overfitting looks like from the inside.
Why is it called "supervised"?
Because the correct answers act like a supervisor watching over training. For every guess the model makes on a training example, there is a known right answer to check it against and correct it toward.
Without that check there would be nothing to supervise the learning, and that absence is exactly what defines the unsupervised methods in the next chapter.
Why do features and labels need to be separated at all?
The model needs to know exactly what it is allowed to look at, the features, and exactly what it is trying to predict, the label.
Suppose the label leaks into the features by accident. Perhaps the actual sale price sits among the inputs used to predict sale price, or a column that was computed from it. The model finds that shortcut immediately, scores almost perfectly in testing, and fails completely on new data, where the label does not exist yet.
The practical test for every feature is simple: will I actually have this value at the moment I need a prediction? If not, it is not a feature.
Example
Predicting whether a customer will cancel their subscription
A streaming service has a year of past customer records: how many hours each person watched, how many support tickets they filed, which plan they were on, and whether they ultimately canceled.
The "canceled or not" column is the label, the known answer for each past customer. Watch hours, ticket count and plan type are features, the inputs the model is allowed to use.
Trained on last year, the model is pointed at this month's customers, none of whom has canceled yet, and asked which ones are about to. That is the payoff: the business gets a list to call before they leave rather than after.
| Watch hours/month | Support tickets | Plan type | Canceled? |
|---|---|---|---|
| 42 | 0 | Premium | No |
| 3 | 4 | Basic | Yes |
| 28 | 1 | Premium | No |
| 1 | 2 | Basic | Yes |
| 55 | 0 | Basic | No |
The first three columns are features, inputs the model is allowed to see. "Canceled?" is the label, the known answer during training and what the model predicts for new customers.
Features and labels, from raw data to a trained model
Split the data into features and a label
Decide which columns the model is allowed to look at (features) and which single column it is trying to predict (the label).
Show the model both together, during training
For every example, the model sees the features and the true label side by side and adjusts itself to reduce the gap between its guess and the real label.
Hide the label at prediction time
For a new example, the model only receives the features, the label is exactly what it is being asked to produce.
| Regression | Classification | |
|---|---|---|
| Predicts | A number | A category |
| Example label | House price: $310,000 | Email: spam or not spam |
| "Correct" means | The prediction is close to the true number | The prediction matches the true category exactly |
Predicts
- Regression
- A number
- Classification
- A category
Example label
- Regression
- House price: $310,000
- Classification
- Email: spam or not spam
"Correct" means
- Regression
- The prediction is close to the true number
- Classification
- The prediction matches the true category exactly
That split, regression versus classification, is the first fork every supervised learning problem takes.
When the label is a number, such as a price or a temperature, the problem is regression. When the label is a category, such as spam or not spam, the problem is classification.
Everything else about supervised learning — features, labels, training, testing — works the same either way. The fork decides which models you can use and, more importantly, how you measure whether they are any good.
from sklearn.tree import DecisionTreeClassifier# Features (X): watch hours, support tickets, plan type (0 = Basic, 1 = Premium)X = [[42, 0, 1], [3, 4, 0], [28, 1, 1], [1, 2, 0], [55, 0, 0]]# Label (y): the known answer for each past customery = [0, 1, 0, 1, 0] # 1 = canceled, 0 = stayedmodel = DecisionTreeClassifier()model.fit(X, y)# A new customer: 10 watch hours, 3 tickets, Basic plannew_customer = [[10, 3, 0]]print(model.predict(new_customer))from sklearn.tree import DecisionTreeClassifierDecisionTreeClassifier predicts a category (canceled or not), so this is a classification model, not a regression one.
# Features (X): watch hours, support tickets, plan type (0 = Basic, 1 = Premium)
X = [[42, 0, 1], [3, 4, 0], [28, 1, 1], [1, 2, 0], [55, 0, 0]]X holds only the features, the inputs the model is allowed to see. No column here reveals the answer.
# Label (y): the known answer for each past customer
y = [0, 1, 0, 1, 0] # 1 = canceled, 0 = stayedy holds the label, the known correct answer for each customer, in the same order as X.
model.fit(X, y).fit(X, y) is the supervised part: the model checks its guesses against the real labels in y and adjusts.
print(model.predict(new_customer))The new customer has features only, no label. Predicting that missing label is the entire point of training the model.
- Feature
One input column a model is allowed to use, such as watch hours or plan type.
Choosing good features is often more important to a model's accuracy than which algorithm you pick.
- Label
The known correct answer for a training example, what the model is trying to predict.
Labels are what make supervised learning supervised. Without them there is nothing to check a guess against, and they are usually the most expensive part of the dataset to obtain.
- Regression
A supervised learning task where the label is a number.
It is one of the two fundamental supervised learning task types you will see throughout this site.
- Classification
A supervised learning task where the label is a category.
It is the other fundamental task type, recognizing which one applies is the first step in any new problem.
When to use
- You have past examples where the true answer is already known (a label for every row)
- The thing you want to predict is a number (regression) or a category (classification)
- You have enough labeled examples to learn a trustworthy pattern from
When to avoid
- You do not have labels for your data at all, that calls for unsupervised learning instead
- The "correct answer" for past examples is itself unreliable or inconsistently recorded
- You are trying to discover unknown groupings in data rather than predict a specific known quantity
Accidentally including the label as one of the features.
Double-check that every input column is something you would actually have available at prediction time, the label itself never belongs among the features.
Treating a category that happens to be written as a number (like a zip code) as a regression target.
Ask whether the values have a meaningful order and scale. A zip code is a category rather than a quantity, so predicting it is classification, not regression.
Assuming supervised learning requires the label to be perfectly accurate.
Real-world labels are often a little noisy. A supervised model can still learn a useful pattern as long as most labels are correct.
Key takeaway
- Supervised learning trains a model on examples that already have a known correct answer.
- Features are the inputs the model is allowed to see. The label is the answer it is trying to predict.
- If the label is a number, it is regression. If the label is a category, it is classification.
- At prediction time, the model only receives features, producing the label is the whole point.
Finished this lesson?
Guest progress is saved on this device.