Supervised, Unsupervised, and Reinforcement Learning: When to Use Each

Machine learning is a family of approaches for different kinds of decisions. In a food-delivery app, supervised learning predicts known outcomes, unsupervised learning discovers structure such as customer segments, and reinforcement learning optimizes sequences of decisions such as courier positioning. Every number below comes from running the code shown.

Supervised learning learns a mapping from inputs to known outputs; unsupervised learning finds structure without ground-truth labels; reinforcement learning learns from state–action–reward interactions to maximize cumulative reward. Each needs a different kind of evidence that it is working, and each section below shows that check.

1. Supervised learning

Problem statement

A food-delivery company wants to estimate delivery time for a new order before assigning a courier, to set customer expectations, prioritize restaurants at peak hours, and flag orders likely to breach the promised window.

Conceptual explanation

Supervised learning uses historical examples where both the inputs and the correct answer are known. Each past order contributes features available when the order was placed — distance, expected preparation time, weather, peak hour, courier availability — and the eventual delivery duration as the target. Two disciplines matter as much as the model: compare against simple baselines, because a flexible model is not automatically better, and turn the prediction into a promise with calibrated uncertainty, because customers experience the late tail, not the average.

Runnable Python code

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(42)
n = 1_000
df = pd.DataFrame({  # features known when the order is placed
    "distance_km": rng.uniform(0.5, 12, n),
    "restaurant_prep_minutes": rng.integers(5, 45, n),
    "couriers_available": rng.integers(1, 20, n),
    "rain": rng.integers(0, 2, n),
    "peak_hour": rng.integers(0, 2, n),
})
df["delivery_time_minutes"] = (
    8 + 3.2 * df["distance_km"] + 0.8 * df["restaurant_prep_minutes"]
    + 6 * df["rain"] + 5 * df["peak_hour"] - 0.7 * df["couriers_available"]
    + rng.normal(0, 4, n)).clip(lower=10)

X = df.drop(columns="delivery_time_minutes")
y = df["delivery_time_minutes"]
# Random split only because these rows are independent synthetic draws;
# real order history needs a chronological train / calibration / test.
X_train, X_rest, y_train, y_rest = train_test_split(
    X, y, test_size=0.4, random_state=42)
X_cal, X_test, y_cal, y_test = train_test_split(
    X_rest, y_rest, test_size=0.5, random_state=42)

candidates = {
    "always predict the mean": None,
    "linear regression": LinearRegression(),
    "gradient boosting": HistGradientBoostingRegressor(random_state=42),
}
for name, model in candidates.items():
    pred = (np.full(len(y_test), y_train.mean()) if model is None
            else model.fit(X_train, y_train).predict(X_test))
    abs_err = np.abs(y_test - pred)
    print(f"{name:>24}: MAE {mean_absolute_error(y_test, pred):5.2f}  "
          f"90th-pct error {np.quantile(abs_err, 0.9):5.2f} min")

# Customer promise: add the 90th percentile of residuals measured on a
# held-out calibration set (split-conformal). Train residuals would be
# too optimistic.
model = candidates["linear regression"]
margin = np.quantile(y_cal - model.predict(X_cal), 0.9)
promise_test = model.predict(X_test) + margin
print(f"P90 promise met for {(y_test <= promise_test).mean():.0%} of "
      f"test orders (target 90%); margin +{margin:.1f} min")

new_order = pd.DataFrame([{"distance_km": 4.2, "restaurant_prep_minutes": 18,
                           "couriers_available": 6, "rain": 1,
                           "peak_hour": 1}])
typical = model.predict(new_order)[0]
print(f"Expected {typical:.0f} min; show customer "
      f"'by {typical + margin:.0f} min'")

The simple linear model beat gradient boosting here, because the simulated relationship is linear; the earlier version reported only a random forest (MAE 4.10), which linear regression also beat (3.32) on that version’s split. The customer promise adds a 5.2-minute margin learned from a held-out calibration set, and it held for 88% of test orders against a 90% target, so the new order is expected at 42 minutes but promised by 48. Margins taken from training residuals would be too optimistic, and a quantile gradient-boosting model tried on this small dataset covered only 78%.

Use case and common mistakes

A platform can show “arriving by 48 minutes” at checkout and, when the promise exceeds the window, warn the customer, suggest faster restaurants, or prioritize the order in dispatch. Avoid features unavailable at prediction time (the actual route or final timestamp), random splits on real order history (use chronological train, calibration, and test periods, since outcomes arrive later), judging only average error, and presenting estimates as guarantees. In production, log inputs, prediction, actual outcome, and model version; monitor error and late-delivery rate by city, zone, weather, and hour; check that promise coverage stays near target; set fallbacks for missing or stale inputs; alert on drift in distance, prep time, courier supply, or weather; and retrain on a schedule and after major menu, courier, or city changes.

2. Unsupervised learning

Problem statement

The company has millions of orders but no customer categories. Marketing wants natural groups — late-night snack buyers, family orderers, price-sensitive regulars, premium loyal customers — to tailor campaigns without labelling customers by hand.

Conceptual explanation

Unsupervised learning finds structure in unlabelled data; clustering groups customers with similar behaviour. The catch is that k-means always returns k groups, whether or not the data contains any. The earlier version of this example generated every behaviour column independently, so there were no segments to find: silhouette scores were about 0.16 for every k from 2 to 7, and refitting on bootstrap samples reproduced the clusters poorly (adjusted Rand index 0.29–0.76). Its four “personas” were arbitrary slices of noise. Before naming segments, check that they exist (silhouette across k) and that they are stable (bootstrap agreement).

Runnable Python code

The data below has four hidden segments the algorithm does not know about, and the same checks are run on structureless data for contrast.

import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score, silhouette_score
from sklearn.preprocessing import StandardScaler

rng = np.random.default_rng(7)
FEATURES = ["orders_per_month", "avg_order_value", "discount_usage_rate",
            "late_night_order_rate", "repeat_customer_rate"]
# Hidden segments (unknown to the algorithm): premium loyal,
# price-sensitive, late-night, occasional family buyers
centres = np.array([[9, 950, 0.10, 0.10, 0.85],
                    [7, 420, 0.65, 0.15, 0.60],
                    [6, 380, 0.25, 0.70, 0.55],
                    [2, 1100, 0.20, 0.05, 0.30]])
spread = np.array([1.5, 120, 0.08, 0.08, 0.10])
segment = rng.integers(0, 4, 600)
customers = pd.DataFrame(centres[segment] + rng.normal(0, 1, (600, 5))
                         * spread, columns=FEATURES).clip(lower=0)

def choose_k(X, ks=range(2, 9), seed=42):
    """Silhouette for each k: around 0.5 or more suggests real groups."""
    return {k: silhouette_score(X, KMeans(k, n_init=10, random_state=seed)
                                .fit_predict(X)) for k in ks}

def bootstrap_stability(X, k, n_boot=20, seed=0):
    """Mean adjusted Rand index between the full-data clustering and
    clusterings refit on bootstrap samples (1.0 = perfectly stable)."""
    rng = np.random.default_rng(seed)
    base = KMeans(k, n_init=10, random_state=seed).fit(X).predict(X)
    scores = []
    for _ in range(n_boot):
        idx = rng.choice(len(X), len(X), replace=True)
        refit = KMeans(k, n_init=10, random_state=seed).fit(X[idx])
        scores.append(adjusted_rand_score(base, refit.predict(X)))
    return float(np.mean(scores))

X = StandardScaler().fit_transform(customers)
silhouettes = choose_k(X)
best_k = max(silhouettes, key=silhouettes.get)
print("silhouette by k:", {k: round(s, 3) for k, s in silhouettes.items()})
stability = bootstrap_stability(X, best_k)
print(f"chosen k={best_k}  bootstrap stability (ARI)={stability:.2f}")

customers["cluster"] = KMeans(best_k, n_init=10,
                              random_state=42).fit_predict(X)
profile = customers.groupby("cluster")[FEATURES].mean().round(2)
profile["customers"] = customers["cluster"].value_counts().sort_index()
print(profile.to_string())

# Sanity check: independent random columns have no segments to find
noise = StandardScaler().fit_transform(rng.normal(size=(600, 5)))
print(f"structureless data: best silhouette "
      f"{max(choose_k(noise).values()):.3f}, "
      f"stability {bootstrap_stability(noise, 4):.2f}")

Silhouette peaked clearly at k = 4 (0.63), bootstrap stability was 1.00, and the clusters matched the hidden segments almost exactly. The structureless data scored a best silhouette of only 0.15 and stability 0.43 — the signature of segments that are not really there. Names are assigned by a person reading the cluster means; the algorithm only supplies the groups.

Use case and common mistakes

Marketing can then give premium loyal customers early access rather than discounts, price-sensitive customers bundle offers, late-night customers late-night recommendations and faster courier coverage, and occasional family buyers weekend meal promotions. Avoid choosing k arbitrarily, clustering unscaled features (large-valued columns dominate), assuming segments stay fixed through seasons and promotions, treating a statistical cluster as a real persona without evidence, and using sensitive attributes that could lead to unfair treatment. Build features from a consistent window such as the last 90 days, handle missing values explicitly, give each segment an owner, track size, revenue, retention, and redemption over time, refresh segments periodically, and keep marketing use within privacy and fairness policy.

3. Reinforcement learning

Problem statement

During the dinner rush, the platform must keep deciding where to send idle couriers. Sending one to the nearest order can leave another zone uncovered ten minutes later, so the company wants a policy that maximizes long-run service, not just the next delivery.

Conceptual explanation

Reinforcement learning trains an agent through interaction: it observes a state (orders, courier positions, prep times, forecasts), takes an action, receives a reward, and moves to a new state, learning a policy that maximizes cumulative reward. It fits problems where actions change future states. When an action does not affect what happens next — choosing which offer to show a customer, say — a contextual bandit or uplift model is simpler and usually enough. Research has explored RL for dispatching and repositioning, but results depend on the simulator, constraints, offline evaluation, and safe rollout as much as on the algorithm.

Runnable Python code

This is a deliberately tiny teaching environment, not a dispatch simulator: one idle courier per step goes to zone A or B, demand persists and grows, and a shift lasts ten steps. Compared with the earlier version, it follows the Gymnasium-style reset/step signature, uses a seeded random generator so runs are reproducible, treats the end of the shift as a time limit rather than a terminal state, and — most importantly — evaluates the learned policy against simple baselines on identical scenarios.

import random
from collections import defaultdict

class CourierAllocationEnv:
    """Toy two-zone courier allocation with a Gymnasium-style API:
    reset(seed) -> (obs, info); step(a) -> (obs, reward, terminated,
    truncated, info). Not a realistic dispatch simulator."""

    actions = ("A", "B")

    def __init__(self, horizon=10, max_demand=2):
        self.horizon, self.max_demand = horizon, max_demand
        self.rng = random.Random()

    def _obs(self):
        return (self.demand["A"], self.demand["B"])

    def reset(self, seed=None):
        if seed is not None:
            self.rng.seed(seed)
        self.t = 0
        self.demand = {z: self.rng.choice([0, 1, 2]) for z in self.actions}
        return self._obs(), {}

    def step(self, action):
        if action not in self.actions:
            raise ValueError(f"unknown action: {action}")
        if self.demand[action] > 0:
            reward = 10
            self.demand[action] -= 1
        else:
            reward = -3                      # courier sent to no demand
        reward -= 2 * sum(self.demand.values())  # orders left waiting
        for zone in self.actions:            # new orders arrive
            arrivals = self.rng.choice([0, 0, 1])
            self.demand[zone] = min(self.max_demand,
                                    self.demand[zone] + arrivals)
        self.t += 1
        # The shift ending is a time limit, not a natural end state
        return self._obs(), reward, False, self.t >= self.horizon, {}

def q_learning(env, episodes=20_000, alpha=0.1, gamma=0.9, epsilon=0.1,
               seed=0):
    rng = random.Random(seed)
    q = defaultdict(lambda: dict.fromkeys(env.actions, 0.0))
    for episode in range(episodes):
        state, _ = env.reset(seed=seed * 1_000_003 + episode)
        while True:
            if rng.random() < epsilon:
                action = rng.choice(env.actions)
            else:
                action = max(q[state], key=q[state].get)
            nxt, reward, terminated, truncated, _ = env.step(action)
            # Bootstrap through truncation: the state carries no clock, so
            # treating the time limit as terminal would corrupt values.
            future = 0.0 if terminated else max(q[nxt].values())
            q[state][action] += alpha * (reward + gamma * future
                                         - q[state][action])
            state = nxt
            if terminated or truncated:
                break
    return q

def evaluate(env, policy, episodes=5_000, seed=10**9):
    """Average episode return; same seeds for every policy."""
    total = 0.0
    for episode in range(episodes):
        state, _ = env.reset(seed=seed + episode)
        done = False
        while not done:
            state, reward, terminated, truncated, _ = env.step(
                policy(state))
            total += reward
            done = terminated or truncated
    return total / episodes

env = CourierAllocationEnv()
q = q_learning(env)
policies = {
    "random": lambda s, r=random.Random(1): r.choice(env.actions),
    "always zone A": lambda s: "A",
    "greedy: busier zone": lambda s: "A" if s[0] >= s[1] else "B",
    "Q-learning": lambda s: max(q[s], key=q[s].get),
}
for name, policy in policies.items():
    print(f"{name:>20}: average return {evaluate(env, policy):7.2f}")
for state in sorted(q):
    print(state, {a: round(v, 1) for a, v in q[state].items()})

Q-learning learned a good policy, but it matched a one-line rule — “send the courier to the busier zone” — rather than beating it. In this toy, the immediate reward already points to the right action, so there is little long-term planning to discover. That is the most useful result in the section: without the baseline comparison, the Q-table would look like a success. The time-limit detail matters too. Because the state carries no clock, treating the last step as terminal (as the earlier version did) gave the same average return here, but distorted the learned values badly: state (0, 0) was valued at 14.6 and 8.3 instead of 36.9 and 41.9, and it preferred the other zone. Values like these feed monitoring and safety thresholds, so they need to be right.

Use case and common mistakes

At 7:30 PM, with heavy demand in a business district and light demand in a residential area, a learned policy might hold some couriers near the district or move idle riders toward a forecast hotspot, with a reward that combines delivery time, unassigned orders, courier idle time, and satisfaction. Common failures are rewards that can be gamed (rewarding only speed encourages dropping long orders), simulators missing traffic, restaurant delays, courier behaviour, or cancellations, exploring on real customers, ignoring delayed consequences, and having no fallback. Build a simulator from historical orders and courier movements, encode business, courier, and fairness constraints, compare against the current dispatch rules with offline policy evaluation, launch in shadow mode or a small geographic experiment, monitor delivery time, cancellations, utilization, and fairness across zones, constrain actions with operational rules such as maximum distance and mandatory breaks, and keep a deterministic fallback policy.

How to choose

Use supervised learning when historical labels exist and the task is prediction: delivery time, cancellation risk, churn, or fraud. Use unsupervised learning when labels do not exist and the goal is discovery — and confirm the structure is real before acting on it. Use reinforcement learning when decisions are sequential and today’s action changes tomorrow’s situation, such as courier repositioning, inventory replenishment, or some forms of dynamic pricing; if actions do not affect future states, prefer a contextual bandit. In every case, beat a simple baseline first.

Takeaway: A practical platform often combines all three: supervised models forecast demand and delivery times, unsupervised models describe customer and zone segments, and reinforcement learning — benchmarked against the rule it replaces — uses those outputs to make better real-time dispatch decisions.

Back to all blogs