Feature Engineering for Tabular ML: 12 Techniques That Improve Real Models

Feature engineering is often the difference between a model that looks good offline and one that improves business outcomes. In e-commerce and retail, the strongest features encode customer behaviour, product context, time, and transaction history — not just raw database columns. The running scenario: an online retailer wants to predict whether a customer will purchase in the next 7 days. The same techniques apply to churn, recommendation, demand forecasting, fraud, and marketing response.

Two principles shape every snippet. First, point-in-time correctness: every feature is computed from events strictly before the prediction moment (as_of), by one function used for both training and live scoring. Second, anything learned from data — encoders, bin edges, frequencies, imputation values — is fitted inside a scikit-learn pipeline on training rows only. All results quoted come from running the code.

Setup: shared example dataset

The setup generates customers, sessions, and orders. In a purely random generator no feature can help (a version of this example built that way scored ROC-AUC 0.57 ± 0.05 on 57 positives — noise), so here a hidden propensity drives sessions, cart adds, and orders, and cart sessions convert within days, giving recency real signal. A 300-value pincode column provides a genuine high-cardinality category. Save it as retail_data.py; sections 1–11, saved together in order, form features.py.

# TargetEncoder needs scikit-learn 1.3+; categorical_features="from_dtype" 1.4+
pip install "scikit-learn>=1.4" pandas numpy
# Save as retail_data.py. Synthetic events with a REAL signal: a hidden
# propensity drives sessions, cart adds, and orders, so features can work.
import numpy as np
import pandas as pd

rng = np.random.default_rng(42)
START, END = pd.Timestamp("2026-01-01"), pd.Timestamp("2026-08-01")
n = 2_000

customers = pd.DataFrame({
    "customer_id": [f"C{i:05d}" for i in range(n)],
    "signup_date": pd.Timestamp("2025-01-01")
    + pd.to_timedelta(rng.integers(0, 540, n), unit="D"),
    "acquisition_channel": rng.choice(
        ["organic", "paid_search", "email", "social", "referral"], n,
        p=[0.35, 0.25, 0.15, 0.15, 0.10]),
    "loyalty_tier": rng.choice(["bronze", "silver", "gold"], n,
                               p=[0.6, 0.3, 0.1]),
    "city": rng.choice(["Mumbai", "Delhi", "Bengaluru", "Pune", "Chennai"],
                       n),
    "pincode": rng.choice([f"P{i:03d}" for i in range(300)], n),
    "age": rng.integers(18, 70, n),
})
tier = customers["loyalty_tier"].map({"bronze": 0.8, "silver": 1.1,
                                      "gold": 1.6})
channel = customers["acquisition_channel"].map(
    {"email": 1.3, "referral": 1.2}).fillna(1.0)
pin_effect = np.exp(rng.normal(0, 0.4, 300))[
    customers["pincode"].str[1:].astype(int)]
propensity = rng.gamma(1.0, 1.0, n) * tier * channel * pin_effect

# Sessions: rate scales with propensity, only after signup
active_from = customers["signup_date"].clip(lower=START)
active_days = (END - active_from).dt.days.clip(lower=0)
counts = rng.poisson(propensity * active_days * 0.10)
idx = np.repeat(np.arange(n), counts)
sessions = pd.DataFrame({
    "customer_id": customers["customer_id"].to_numpy()[idx],
    "session_date": active_from.to_numpy()[idx] + pd.to_timedelta(
        (rng.random(len(idx)) * active_days.to_numpy()[idx]).astype(int),
        unit="D"),
    "device": rng.choice(["mobile", "desktop", "tablet"], len(idx),
                         p=[0.65, 0.30, 0.05]),
    "pages_viewed": rng.poisson(8, len(idx)) + 1,
    "time_on_site_seconds": rng.gamma(2.0, 90, len(idx)).astype(int),
    "added_to_cart": rng.binomial(
        1, np.clip(0.12 + 0.08 * propensity[idx], 0, 0.6)),
    "discount_code_used": rng.binomial(1, 0.18, len(idx)),
})

# Orders: cart sessions convert within a few days (recency matters)
carts = sessions[sessions["added_to_cart"].eq(1)]
converted = carts[rng.random(len(carts)) < 0.35]
orders = pd.DataFrame({
    "customer_id": converted["customer_id"].to_numpy(),
    "order_date": converted["session_date"].to_numpy()
    + pd.to_timedelta(rng.integers(0, 4, len(converted)), unit="D"),
})
orders["product_category"] = rng.choice(
    ["electronics", "apparel", "home", "beauty", "grocery"], len(orders))
orders["order_value"] = np.round(rng.gamma(2.0, 1200, len(orders)), 2)
orders = orders[orders["order_date"] < END].reset_index(drop=True)

1. Missing-value indicators

Concept

Missingness is often informative. A missing last_order_date means the customer has never purchased; a missing session average means they have never visited. The important distinction is between “happened zero times”, where 0 is the true value, and “never happened”, where an average or recency has no true value and should stay missing with an indicator.

Code

import numpy as np
import pandas as pd

def safe_divide(numerator, denominator):
    """Ratio with NaN (not 0 or inf) where the denominator is 0 or NaN."""
    denominator = pd.Series(denominator).replace(0, np.nan)
    return pd.Series(numerator) / denominator

def add_missing_indicators(df):
    """Separate 'never happened' from 'happened zero times'."""
    out = df.copy()
    # One indicator per cause: avg_order_value and prior_revenue are
    # missing for exactly the same customers, so one flag covers them.
    out["never_ordered"] = out["last_order_date"].isna().astype(int)
    out["never_visited"] = out["total_sessions"].isna().astype(int)
    # Counts and sums really are zero when nothing happened...
    zero_if_none = ["total_sessions", "prior_orders", "prior_revenue"]
    out[zero_if_none] = out[zero_if_none].fillna(0)
    # ...but averages and recencies stay NaN: there is no true value.
    # HistGradientBoosting handles NaN natively; linear models impute
    # inside the pipeline (section 12), never on the full dataset.
    return out

New customers are fundamentally different from repeat buyers, and never_ordered lets the model learn that directly. The earlier version created three indicators that were identical (average order value and revenue are missing for exactly the same customers as last order date), filled averages with 0 — implying a ₹0 average order — and did it on the full dataset under a comment promising “training-derived” values. Never drop rows with missing values either: that silently removes new customers.

2. Date and time features

Concept

Raw timestamps are hard for models to use. Recency and age relative to as_of, plus calendar components, expose patterns such as weekend shopping, month-end promotions, and seasonality.

Code

def add_date_features(df, as_of):
    out = df.copy()
    out["days_since_last_order"] = (as_of - out["last_order_date"]).dt.days
    out["account_age_days"] = (as_of - out["signup_date"]).dt.days
    out["signup_month"] = out["signup_date"].dt.month
    out["signup_day_of_week"] = out["signup_date"].dt.dayofweek
    out["signup_is_weekend"] = (out["signup_day_of_week"] >= 5).astype(int)
    return out

Recency is usually among the strongest purchase and churn features. Customers who have never ordered keep a missing recency — the earlier version replaced it with “maximum + 1” computed over the whole dataset, a value that changes with the data and leaks the evaluation set’s range. Compute recency from as_of, never from today or from a date inside the label window, and handle time zones when sessions span countries.

3. Cyclical encoding

Concept

Hour, weekday, and month are cyclical: December is next to January and Sunday next to Monday. Sine and cosine together place each value on a circle; one alone maps two different times to the same number. For 1-based values such as months, subtract 1 first so the twelve months are evenly spaced.

Code

import numpy as np

def encode_cyclical(df, column, period, offset=0):
    """Map a cyclical integer to the unit circle (offset=1 for 1-based)."""
    out = df.copy()
    angle = 2 * np.pi * (out[column] - offset) / period
    out[f"{column}_sin"] = np.sin(angle)
    out[f"{column}_cos"] = np.cos(angle)
    return out

# encode_cyclical(df, "signup_month", period=12, offset=1)
# encode_cyclical(df, "signup_day_of_week", period=7)  # already 0-based

Hour and weekday matter for demand forecasting; do not encode non-cyclical features such as age this way. Tree models can usually use the raw integer, and in this example the signup-calendar features added nothing (section 12) — a reminder to keep a feature only if it earns its place.

4. Ratio and interaction features

Concept

Ratios express relationships — discount reliance, browsing speed, conversion of carts to orders — that neither raw column shows alone. Interactions let a model learn effects that depend on another variable.

Code

def add_ratio_features(df):
    out = df.copy()
    # Not revenue_per_order: prior_revenue / prior_orders IS avg_order_value
    out["discount_reliance"] = safe_divide(out["discount_sessions"],
                                           out["cart_sessions"])
    out["pages_per_minute"] = safe_divide(out["avg_pages_viewed"],
                                          out["avg_time_on_site"] / 60)
    out["order_conversion"] = safe_divide(out["prior_orders"],
                                          out["cart_sessions"])
    return out

The earlier version’s headline ratio, revenue per order, was prior_revenue / prior_orders — exactly the existing avg_order_value column, so it added a duplicate rather than information. It also built a high_engagement flag from medians of the full dataset, which leaks the evaluation distribution. Use NaN for undefined ratios rather than adding +0.01 to the denominator, and check every interaction against validation: here the three ratios slightly hurt the model.

5. Aggregation features

Concept

Aggregations turn event-level data into customer-level features: counts, sums, means, maxima, and shares — each computed only from events before as_of.

Code

def lifetime_aggregates(sessions, orders, as_of):
    """Customer-level summaries of everything strictly before as_of."""
    s = sessions[sessions["session_date"] < as_of]
    o = orders[orders["order_date"] < as_of]
    session_agg = s.groupby("customer_id").agg(
        total_sessions=("session_date", "count"),
        avg_pages_viewed=("pages_viewed", "mean"),
        max_pages_viewed=("pages_viewed", "max"),
        avg_time_on_site=("time_on_site_seconds", "mean"),
        cart_sessions=("added_to_cart", "sum"),
        cart_add_rate=("added_to_cart", "mean"),
        discount_sessions=("discount_code_used", "sum"),
        mobile_share=("device", lambda d: d.eq("mobile").mean()),
    )
    order_agg = o.groupby("customer_id").agg(
        prior_orders=("order_date", "count"),
        prior_revenue=("order_value", "sum"),
        avg_order_value=("order_value", "mean"),
        last_order_date=("order_date", "max"),
    )
    return session_agg.join(order_agg, how="outer")

Use counts and maxima as well as averages, because they carry different information, and distrust statistics from very small groups, where one outlier dominates.

6. Rolling-window features

Concept

Recent behaviour usually predicts better than lifetime behaviour. Several trailing windows (7, 14, 30 days) plus a trend ratio separate a customer who is heating up from one who is cooling down.

Code

import pandas as pd

def window_features(sessions, orders, as_of, windows=(7, 14, 30)):
    """Activity in several trailing windows ending the day before as_of."""
    feats = {}
    for days in windows:
        start = as_of - pd.Timedelta(days=days)
        s = sessions[sessions["session_date"].between(start, as_of,
                                                      inclusive="left")]
        o = orders[orders["order_date"].between(start, as_of,
                                                inclusive="left")]
        g = s.groupby("customer_id")
        feats[f"sessions_{days}d"] = g.size()
        feats[f"cart_adds_{days}d"] = g["added_to_cart"].sum()
        feats[f"orders_{days}d"] = o.groupby("customer_id").size()
    out = pd.DataFrame(feats).fillna(0)  # no activity in window = 0
    # Trend: share of the last 30 days' sessions that fell in the last 7
    out["session_trend_7_30"] = safe_divide(out["sessions_7d"],
                                            out["sessions_30d"])
    return out

In the ablation, removing this family cost the most: ROC-AUC fell from 0.863 to 0.854 and PR-AUC from 0.402 to 0.380. The earlier version computed a single 30-day cart-add count identical to the one already built in the aggregation section, then merged an rfm table that was not defined until the next section, so the script stopped with a NameError.

7. Recency, frequency, monetary (RFM)

Concept

RFM summarizes customer value by how recently, how often, and how much a customer buys. Lifetime R, F, and M already exist from sections 2 and 5 (days_since_last_order, prior_orders, prior_revenue), so recomputing them would duplicate columns; the useful addition is a recent-value view.

Code

import pandas as pd

def rfm_features(orders, as_of, days=90):
    """Recent-value RFM. Lifetime R, F and M already exist as
    days_since_last_order, prior_orders and prior_revenue, so this adds a
    90-day view instead of duplicating them."""
    start = as_of - pd.Timedelta(days=days)
    o = orders[orders["order_date"].between(start, as_of, inclusive="left")]
    return o.groupby("customer_id").agg(
        frequency_90d=("order_date", "count"),
        monetary_90d=("order_value", "sum"),
    )

The earlier version filled missing recency for never-purchased customers with 1 day, telling the model they bought yesterday — the opposite of the truth. Validate any RFM score cut-offs for the business rather than treating them as universal, and net out returns, cancellations, and refunds when computing monetary value.

Assembling point-in-time features

One function builds every customer’s features as of a date and attaches the label from the following seven days; stacking weekly snapshots gives many prediction moments instead of one. The same build_features is what a scoring job calls with today’s date, so training and serving cannot drift apart. It also contains the customer-level category mix used in section 10.

import pandas as pd

from retail_data import customers, orders, sessions

def category_mix(orders, as_of):
    """Share of each customer's prior orders in each category."""
    o = orders[orders["order_date"] < as_of]
    counts = pd.crosstab(o["customer_id"], o["product_category"])
    return counts.div(counts.sum(axis=1), axis=0).add_prefix("share_")

def build_features(as_of, horizon_days=7):
    """One row per customer known at as_of; the ONLY feature code path,
    used identically for training snapshots and live scoring."""
    as_of = pd.Timestamp(as_of)
    base = customers[customers["signup_date"] < as_of].set_index(
        "customer_id")
    df = (base.join(lifetime_aggregates(sessions, orders, as_of))
              .join(window_features(sessions, orders, as_of))
              .join(rfm_features(orders, as_of))
              .join(category_mix(orders, as_of)))
    window_cols = [c for c in df if c.endswith(("_7d", "_14d", "_30d"))]
    share_cols = [c for c in df if c.startswith("share_")]
    df[window_cols] = df[window_cols].fillna(0)
    df[share_cols] = df[share_cols].fillna(0)
    df[["frequency_90d", "monetary_90d"]] = df[
        ["frequency_90d", "monetary_90d"]].fillna(0)
    df = add_missing_indicators(df)
    df = add_date_features(df, as_of)
    df = encode_cyclical(df, "signup_month", period=12, offset=1)
    df = encode_cyclical(df, "signup_day_of_week", period=7)
    df = add_ratio_features(df)

    # Label: any order in [as_of, as_of + horizon). Never used as a feature.
    future = orders[orders["order_date"].between(
        as_of, as_of + pd.Timedelta(days=horizon_days), inclusive="left")]
    df["target_purchase_next_7d"] = df.index.isin(
        future["customer_id"]).astype(int)
    df["as_of"] = as_of
    return df.drop(columns=["signup_date", "last_order_date"])

def build_training_set(as_of_dates):
    """Stack weekly snapshots: many prediction moments, not just one."""
    return pd.concat([build_features(d) for d in as_of_dates])

In testing, changing every event on or after as_of left all features unchanged, and later sign-ups were excluded.

8. Categorical encoding: one-hot and ordinal

Concept

Models need numeric inputs. One-hot encoding suits nominal categories with few values; ordinal encoding is only appropriate when the categories have a real order. Both are fitted in the pipeline, so unseen values are handled by policy rather than by crashing.

Code

from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder

# Nominal, low cardinality: one-hot (unseen cities -> all zeros)
one_hot = OneHotEncoder(handle_unknown="ignore", sparse_output=False)

# Genuinely ordered: ordinal, with an explicit order and unknown policy
tier_encoder = OrdinalEncoder(categories=[["bronze", "silver", "gold"]],
                              handle_unknown="use_encoded_value",
                              unknown_value=-1)
# Both are fitted inside the pipeline in section 12, on training rows only.

loyalty_tier has a real order, so ordinal encoding is reasonable; city and acquisition channel do not, so they are one-hot encoded. Do not ordinal-encode unordered categories, do not one-hot encode high-cardinality fields such as SKU or pincode, and always define what happens to unseen categories in production.

9. Target encoding for high-cardinality categories

Concept

Target encoding replaces a category with a smoothed estimate of the target rate for that category. scikit-learn’s TargetEncoder (version 1.3+) cross-fits inside fit_transform, so each training row is encoded by a model that never saw its own label, then uses the full-training encoding in transform for new rows.

Code

from sklearn.preprocessing import TargetEncoder

# pincode has 300 values: too many for one-hot, ideal for target encoding.
# Inside a pipeline, fit_transform cross-fits on the training rows (each
# row encoded by a model that never saw its own label) and transform
# applies the full-data encoding to validation/test rows.
pincode_target = TargetEncoder(target_type="binary", smooth="auto", cv=5,
                               random_state=42)

A smoothed rate shrinks rare pincodes (or SKUs) toward the overall average. The earlier version used TargetEncoder without importing it, fitted it on the full dataset before any split, applied it to a five-value city column that one-hot handles fine, and never used the result. Placed in the pipeline (section 12), cross-fitting happens on training rows only and the test set is only transformed.

10. Frequency and count encoding

Concept

How often a category appears can itself be predictive — a common pincode may indicate a dense delivery market. Frequencies must come from training data; unseen categories get 0. At the customer level, each customer’s share of past orders by category (category_mix in the assembly code) captures preferences.

Code

import pandas as pd
from sklearn.base import BaseEstimator, TransformerMixin

class FrequencyEncoder(TransformerMixin, BaseEstimator):
    """Replace each category with its share of TRAINING rows."""

    def fit(self, X, y=None):
        X = pd.DataFrame(X)
        self.maps_ = [X[c].value_counts(normalize=True) for c in X]
        return self

    def transform(self, X):
        X = pd.DataFrame(X)
        return pd.concat(
            [X[c].map(m).astype(float).fillna(0.0)  # unseen category -> 0
             for c, m in zip(X, self.maps_)], axis=1).to_numpy()

    def get_feature_names_out(self, input_features=None):
        return [f"{c}_freq" for c in input_features]

Do not compute frequencies on the test set, do not confuse frequency (how common) with target rate (how predictive), and normalize raw counts by tenure or opportunity — which is why category mix uses shares rather than counts. In this example category mix added nothing measurable.

11. Binning continuous variables

Concept

Binning converts a continuous value into ordered bands, such as low to very high spend. It helps linear models capture non-linear effects and makes rules easier to explain; tree models rarely benefit.

Code

import numpy as np
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder

class QuantileBands(TransformerMixin, BaseEstimator):
    """Quantile bands learned from non-missing TRAINING values; missing
    gets its own band. Median-imputing first would pile the many
    never-ordered customers into one value and collapse the bins."""

    def __init__(self, n_bins=4):
        self.n_bins = n_bins

    def fit(self, X, y=None):
        X = np.asarray(X, dtype=float)
        qs = np.linspace(0, 1, self.n_bins + 1)[1:-1]
        self.edges_ = [np.unique(np.nanquantile(col, qs)) for col in X.T]
        return self

    def transform(self, X):
        X = np.asarray(X, dtype=float)
        bands = [np.where(np.isnan(col), -1, np.digitize(col, edges))
                 for col, edges in zip(X.T, self.edges_)]
        return np.column_stack(bands)  # -1 = missing, 0..n_bins-1 = band

# For linear models, e.g. on avg_order_value. Trees split the raw value
# themselves and gain little from fixed bands.
spend_bands = make_pipeline(
    QuantileBands(n_bins=4),
    OneHotEncoder(handle_unknown="ignore", sparse_output=False))

Edges must be learned from training data — the earlier version used pd.qcut on the full dataset. Imputing before binning is a trap: with 58% of customers never having ordered, median imputation stacked them on one value and collapsed four quantile bins into two, which a test caught. The transformer above learns edges from customers who have ordered and gives “never ordered” its own band.

12. Feature selection and validation

Concept

Redundant, noisy, or leaky features hurt generalization and complicate monitoring. Validate on a time-based split (random splits put a customer’s neighbouring weeks on both sides), compare against a simple baseline, then drop one feature family at a time to see what earns its place.

Code

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score, roc_auc_score
from sklearn.pipeline import make_pipeline

from features import (FrequencyEncoder, build_training_set, one_hot,
                      pincode_target, tier_encoder)

dates = pd.date_range("2026-03-02", "2026-07-20", freq="7D")
data = build_training_set(dates)
target = "target_purchase_next_7d"
categorical = ["acquisition_channel", "loyalty_tier", "city", "pincode"]

# Time-based split: train on earlier snapshots, test on later ones
train = data[data["as_of"] < "2026-06-01"]
test = data[data["as_of"] >= "2026-06-01"]

raw = ["age", "total_sessions", "avg_pages_viewed", "avg_time_on_site"]
numeric = [c for c in data if c not in {target, "as_of", *categorical}]

def make_model(num_cols, use_categorical=True):
    parts = [("num", "passthrough", num_cols)]
    if use_categorical:
        parts += [("onehot", one_hot, ["acquisition_channel", "city"]),
                  ("tier", tier_encoder, ["loyalty_tier"]),
                  ("pin_te", pincode_target, ["pincode"]),
                  ("pin_freq", FrequencyEncoder(), ["pincode"])]
    return make_pipeline(  # no scaler or imputer: trees handle scale/NaN
        ColumnTransformer(parts),
        HistGradientBoostingClassifier(random_state=42))

def evaluate(model):
    scores = model.fit(train, train[target]).predict_proba(test)[:, 1]
    return (roc_auc_score(test[target], scores),
            average_precision_score(test[target], scores))

print(f"train {len(train)} rows, test {len(test)} rows, "
      f"positive rate {data[target].mean():.3f}")
for name, model in [("raw columns", make_model(raw, False)),
                    ("all engineered", make_model(numeric))]:
    print(f"{name:>16}: ROC-AUC %.3f  PR-AUC %.3f" % evaluate(model))

# Ablation: drop one feature family at a time; keep what earns its place
families = {
    "trailing windows": lambda c: c.endswith(("_7d", "_14d", "_30d",
                                              "_7_30")),
    "90-day RFM": lambda c: c.endswith("_90d"),
    "ratios": lambda c: c in {"discount_reliance", "pages_per_minute",
                              "order_conversion"},
    "signup calendar": lambda c: c.startswith("signup_"),
    "category mix": lambda c: c.startswith("share_"),
}
for family, in_family in families.items():
    kept = [c for c in numeric if not in_family(c)]
    auc, ap = evaluate(make_model(kept))
    print(f"without {family:<17} ROC-AUC {auc:.3f}  PR-AUC {ap:.3f}")

Results

Engineered features lifted PR-AUC from 0.359 to 0.402 on 15,795 later test rows; trailing windows and 90-day RFM carried the gain, while ratios and signup-calendar features are candidates to drop. The earlier version scaled and median-imputed inputs to gradient boosting (which needs neither and loses the missingness signal) and compared feature sets on a random target. Confirm any lift on repeated time splits and business metrics, and never select features on the test set.

Takeaway: Compute every feature as of the prediction moment through one function shared by training and serving, fit anything learned on training rows only, and keep only features that improve a time-based validation.

Back to all blogs