Build a Clean Scikit-Learn Pipeline Preprocessing, Modeling, and Evaluation

Introduction

A scikit-learn Pipeline turns a fragile chain of notebook cells into one object that preprocesses, trains, evaluates and predicts the same way every time. That single change removes the most common silent bug in applied machine learning: data leakage between training and test data.

Most tutorials show preprocessing and modeling as separate steps. You impute missing values, scale numbers, encode categories, and then fit a model. That works in a notebook, but it breaks down in three predictable ways:

Leakage. Statistics such as the mean or the scaling range get computed on the full dataset, so the test set quietly influences training.

Drift between training and serving. The code that cleans data at prediction time is rewritten by hand and slowly diverges from the training code.

Untunable preprocessing. Choices like the imputation strategy or the number of polynomial features cannot be searched alongside model hyperparameters.

In this article you will build a complete, production-shaped workflow on a real dataset with mixed numeric and categorical columns and missing values. By the end you will have:

A preprocessing stage built with ColumnTransformer that handles each column type correctly.

A full Pipeline that chains preprocessing and a classifier into one estimator.

Honest evaluation with stratified cross-validation and multiple metrics.

Hyperparameter tuning that searches preprocessing and model settings together.

A custom transformer for feature engineering.

A saved artifact you can load in an API and call with raw input.

The code targets scikit-learn 1.5 or later, pandas 2.x and Python 3.10+. Every snippet runs end to end if you execute them in order.

The problem with ad-hoc preprocessing

The leak usually hides in two innocent lines: fitting a scaler or imputer on all the data before splitting. Here is the pattern most people write first.

import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# Pure noise: 200 rows, 5,000 random features, random labels
rng = np.random.default_rng(42)
X = rng.normal(size=(200, 5000))
y = rng.integers(0, 2, size=200)

# LEAKY: scale and select features using ALL rows, then cross-validate
X_scaled = StandardScaler().fit_transform(X)
X_selected = SelectKBest(f_classif, k=20).fit_transform(X_scaled, y)

scores = cross_val_score(LogisticRegression(max_iter=1000), X_selected, y, cv=5)
print(f"Leaky accuracy: {scores.mean():.2f}")   # typically 0.75-0.85

The labels are random coin flips, so no model can genuinely beat 50% accuracy. Yet the leaky version reports something close to 80%. The reason is SelectKBest: it looked at every row's label, including rows that later serve as validation folds, and picked the 20 features that happen to correlate with those labels by chance.

Now put the same steps inside a pipeline:

from sklearn.pipeline import make_pipeline

clean = make_pipeline(
    StandardScaler(),
    SelectKBest(f_classif, k=20),
    LogisticRegression(max_iter=1000),
)
scores = cross_val_score(clean, X, y, cv=5)
print(f"Pipeline accuracy: {scores.mean():.2f}")  # ~0.50, the truth

Because the pipeline is passed to cross_val_score as a single estimator, every fold refits the scaler and the feature selector on that fold's training rows only. The validation rows are never seen during fitting, and the score drops to chance, which is the honest answer.

Scaling alone leaks far less than feature selection, but the principle is identical. Any step that learns from data, whether a mean, a vocabulary, a category list or a selection of features, must be fitted inside the cross-validation loop. A pipeline guarantees that by construction rather than by discipline.

Core building blocks

Everything in this article rests on four ideas: transformers, estimators, Pipeline and ColumnTransformer. Scikit-learn's consistent API is what lets them snap together.

A Pipeline behaves like whatever its final step is. If the last step is a classifier, the pipeline has predict; if it is a transformer, the pipeline has transform. When you call pipeline.fit(X, y), scikit-learn calls fit_transform on each intermediate step in order, feeding each output to the next, and finally calls fit on the last step. When you call pipeline.predict(X), it calls only transform on the intermediate steps, reusing what they learned during fitting, and then predict on the model.

That asymmetry is the whole point. Fitting happens once, on training data. Prediction reuses fitted state and never relearns from new data.

Pipeline vs make_pipeline

from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Explicit names: you choose them
pipe = Pipeline(steps=[
    ("scaler", StandardScaler()),
    ("model", LogisticRegression()),
])

# Auto names: lower-cased class names ("standardscaler", "logisticregression")
pipe = make_pipeline(StandardScaler(), LogisticRegression())

Prefer explicit names in real projects. You will refer to steps by name when tuning hyperparameters (model__C) and when inspecting fitted components (pipe.named_steps["model"]), and short stable names make that code readable.

The double-underscore convention

Nested parameters are addressed with step__parameter. For a pipeline whose preprocessing step contains a numeric sub-pipeline with an imputer, the imputation strategy is preprocess__num__imputer__strategy. You can list every addressable parameter with pipe.get_params().keys(). This convention is what makes preprocessing searchable later in the article.

Setting up the dataset

We will use the Titanic passenger dataset from OpenML because it has exactly the mess real projects have: numeric and categorical columns, missing values, and a few columns that leak the answer. The task is binary classification: predict whether a passenger survived.

import pandas as pd
from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split

df = fetch_openml("titanic", version=1, as_frame=True).frame
print(df.shape)          # (1309, 14)
print(df.isna().mean().sort_values(ascending=False).head(6))

The missing-value profile tells you a lot. body and cabin are mostly empty, boat is missing for about 63% of rows, and age is missing for roughly 20%.

Drop leaking and identifier columns first

Before any modeling, decide which columns are allowed to exist at prediction time. Two columns here are post-event information:

boat: the lifeboat number. Having one almost guarantees survival, and it was only known after the disaster.

body: the body identification number, only assigned to passengers who died.

A model trained with these will score near-perfectly in evaluation and be useless in reality. This is target leakage, a different and more dangerous cousin of the train-test leakage in the previous section. No pipeline can catch it for you; it requires domain judgment.

We also drop ticket and home.dest, which are high-cardinality free text, and keep name for later feature engineering. Finally, OpenML stores pclass as a float (1.0, 2.0, 3.0). Casting it to int keeps the encoded column names clean (pclass_1 rather than pclass_1.0) and matches the integer the API will receive later.

TARGET = "survived"
DROP = ["boat", "body", "ticket", "home.dest"]

X = df.drop(columns=DROP + [TARGET])
y = df[TARGET].astype(int)
X["pclass"] = X["pclass"].astype(int)  # OpenML stores it as 1.0, 2.0, 3.0

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)
print(X_train.shape, X_test.shape)   # (1047, 9) (262, 9)
print(y_train.mean().round(3))       # ~0.382 survival rate

Two details matter here. First, stratify=y keeps the survival rate identical in both splits, which matters whenever classes are imbalanced. Second, the test set is now locked away. We will not touch it again until the final evaluation; all model selection happens on X_train through cross-validation.

Define column groups explicitly

numeric_features = ["age", "fare", "sibsp", "parch"]
categorical_features = ["sex", "embarked", "pclass"]

pclass is stored as a number (1, 2, 3) but represents a category, and treating it as categorical lets the model learn a separate effect per class. Being explicit about column groups is safer than selecting by dtype, because a numeric code column can silently be scaled as if it were a quantity. If your schema is stable and well-typed, make_column_selector(dtype_include=...) is a reasonable alternative.

Preprocessing with ColumnTransformer

Numeric and categorical columns need different treatment, so we build one small pipeline per column type and let ColumnTransformer route each column to the right one.

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_pipeline = Pipeline(steps=[
    ("imputer", SimpleImputer(strategy="median", add_indicator=True)),
    ("scaler", StandardScaler()),
])

categorical_pipeline = Pipeline(steps=[
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
])

preprocessor = ColumnTransformer(
    transformers=[
        ("num", numeric_pipeline, numeric_features),
        ("cat", categorical_pipeline, categorical_features),
    ],
    remainder="drop",
    verbose_feature_names_out=False,
)

Each choice here is deliberate:

Median imputation is robust to the heavy right skew of fare, where a few very expensive tickets would drag the mean upward.

add_indicator=True appends a binary column flagging which rows were imputed. On Titanic, a missing age is itself informative because it correlates with passenger class and record-keeping, so the model gets to use that signal instead of losing it.

Scaling matters for linear models and distance-based models. Tree ensembles ignore it, but it does no harm and keeps the preprocessor model-agnostic.

handle_unknown="ignore" means a category never seen during training, such as a new port code in production, becomes an all-zero vector instead of crashing the prediction.
remainder="drop" discards any column not listed, here name and cabin. The alternative, "passthrough", forwards unlisted columns untouched, which is convenient but risky: a new column in production would silently enter the model.

Inspect what the preprocessor produces

Always look at the transformed output once before modeling. The set_output API returns a DataFrame with readable column names instead of an anonymous NumPy array.

preprocessor.set_output(transform="pandas")
Xt = preprocessor.fit_transform(X_train)

print(Xt.shape)          # (1047, 14)
print(Xt.columns.tolist())
# ['age', 'fare', 'sibsp', 'parch', 'missingindicator_age',
#  'missingindicator_fare', 'sex_female', 'sex_male',
#  'embarked_C', 'embarked_Q', 'embarked_S',
#  'pclass_1', 'pclass_2', 'pclass_3']
print(Xt.isna().sum().sum())   # 0: no missing values remain

The exact indicator columns depend on which features had missing values in the training split. Fitting the preprocessor on its own like this is purely for inspection. In the real workflow it is never fitted separately; it is fitted inside the full pipeline, and inside each cross-validation fold.

Optional: ordinal and high-cardinality categories

One-hot encoding is the right default for a handful of unordered categories. Two other encoders are worth knowing:

Adding the model

With preprocessing defined, the full pipeline is two lines: the preprocessor, then a classifier. The result is a single estimator that accepts the raw DataFrame.

from sklearn.linear_model import LogisticRegression

clf = Pipeline(steps=[
    ("preprocess", preprocessor),
    ("model", LogisticRegression(max_iter=1000)),
])

clf.fit(X_train, y_train)

# Raw rows in, predictions out: no manual cleaning
print(clf.predict(X_test.head(3)))
print(clf.predict_proba(X_test.head(3)).round(3))

Notice what you did not have to do. You never imputed, scaled or encoded the test rows by hand. The pipeline applied the medians, means, standard deviations and category lists it learned from X_train to the new rows automatically.

In a Jupyter notebook, simply evaluating clf in a cell renders an interactive diagram of the pipeline, with expandable boxes for every step and its parameters. It is the fastest way to review a colleague's pipeline.

Start with a baseline

Before judging any model, establish what "no skill" looks like. DummyClassifier predicts using only the class distribution and needs no features.

from sklearn.dummy import DummyClassifier

baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print(f"Baseline accuracy: {baseline.score(X_test, y_test):.3f}")  # ~0.618

Predicting "did not survive" for everyone already scores about 62% accuracy. Any model must clear that bar convincingly, and this is the first hint that accuracy alone is a weak metric on imbalanced data.

Swapping models is a one-line change

Because preprocessing is decoupled from modeling, comparing algorithms is trivial. You can even replace a step on a cloned pipeline with set_params:

from sklearn.base import clone
from sklearn.ensemble import RandomForestClassifier, HistGradientBoostingClassifier

candidates = {
    "logreg": LogisticRegression(max_iter=1000),
    "random_forest": RandomForestClassifier(n_estimators=300, random_state=42),
    "hist_gb": HistGradientBoostingClassifier(random_state=42),
}

pipelines = {
    name: clone(clf).set_params(model=model)
    for name, model in candidates.items()
}

clone creates an unfitted copy with identical settings, so each candidate starts fresh. In the next section we evaluate all three fairly.

Evaluation done right

Honest evaluation means two things: estimate performance with cross-validation on the training set, and touch the test set exactly once at the end. Passing the whole pipeline to the cross-validation function is what keeps every fold leak-free.

Cross-validation with multiple metrics

from sklearn.model_selection import StratifiedKFold, cross_validate

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scoring = ["accuracy", "balanced_accuracy", "f1", "roc_auc"]

results = []
for name, pipe in pipelines.items():
    scores = cross_validate(pipe, X_train, y_train, cv=cv,
                            scoring=scoring, n_jobs=-1)
    row = {"model": name}
    for metric in scoring:
        s = scores[f"test_{metric}"]
        row[metric] = f"{s.mean():.3f} ± {s.std():.3f}"
    results.append(row)

print(pd.DataFrame(results).to_string(index=False))

StratifiedKFold preserves the survival rate in every fold, and fixing random_state makes the comparison reproducible. Reporting the standard deviation alongside the mean matters: if two models differ by 0.005 but each fold varies by 0.03, the difference is noise, not a ranking.

On this dataset, all three models typically land around 0.77 to 0.80 accuracy and 0.83 to 0.85 ROC AUC. The two tree ensembles usually edge out logistic regression, but the gaps are within one standard deviation, so treat them as roughly tied. Your exact numbers will vary with library versions; what matters is the method.

Choosing the right metric

The metric should reflect the cost of mistakes in your problem, not habit.

Final evaluation on the test set

Once you have chosen a model using cross-validation, fit it on the full training set and evaluate on the held-out test set once.

from sklearn.metrics import classification_report, ConfusionMatrixDisplay, roc_auc_score

best = pipelines["hist_gb"].fit(X_train, y_train)

y_pred = best.predict(X_test)
y_proba = best.predict_proba(X_test)[:, 1]

print(classification_report(y_test, y_pred, target_names=["died", "survived"]))
print(f"Test ROC AUC: {roc_auc_score(y_test, y_proba):.3f}")

ConfusionMatrixDisplay.from_estimator(best, X_test, y_test,
                                      display_labels=["died", "survived"])

The classification report shows precision, recall and F1 for each class separately, which reveals asymmetries a single accuracy number hides. Survivors are usually recalled less well than non-survivors, because the model sees fewer of them.

The confusion matrix makes those errors concrete: how many survivors were predicted to die (false negatives) and the reverse. If the test score falls far below the cross-validation estimate, suspect a leak or a distribution difference between splits, and investigate before shipping.

Adjusting the decision threshold

By default, predict labels a row positive when its predicted probability exceeds 0.5. That threshold is a business decision, not a law. Since scikit-learn 1.5, TunedThresholdClassifierCV wraps your pipeline and picks the threshold that maximizes a chosen metric using cross-validation:

from sklearn.model_selection import TunedThresholdClassifierCV

tuned = TunedThresholdClassifierCV(best, scoring="balanced_accuracy", cv=cv)
tuned.fit(X_train, y_train)
print(f"Chosen threshold: {tuned.best_threshold_:.2f}")

Hyperparameter tuning across the whole pipeline

The biggest payoff of a pipeline is that preprocessing choices become hyperparameters. You can search the imputation strategy, the encoder settings and even the model itself in one cross-validated search, with no leakage.

Grid search over preprocessing and model together

from sklearn.model_selection import GridSearchCV

param_grid = {
    "preprocess__num__imputer__strategy": ["mean", "median"],
    "preprocess__cat__encoder__min_frequency": [None, 10],
    "model__C": [0.01, 0.1, 1.0, 10.0],
}

search = GridSearchCV(
    clf,                      # the logistic regression pipeline
    param_grid=param_grid,
    cv=cv,
    scoring="roc_auc",
    n_jobs=-1,
    refit=True,
)
search.fit(X_train, y_train)

print(search.best_params_)
print(f"Best CV ROC AUC: {search.best_score_:.3f}")

That grid has 2 × 2 × 4 = 16 combinations, each fitted on 5 folds, for 80 fits. For every one of them, the imputer and encoder are refitted on the fold's training rows only. With refit=True, the best configuration is then retrained on all of X_train, and search itself becomes a ready-to-use model: search.predict(X_test) works directly, and search.best_estimator_ gives you the fitted pipeline. One honest caveat: every category in sex, embarked and pclass appears well over 10 times, so min_frequency=10 changes nothing on this dataset. It is in the grid to show the syntax; it pays off on columns with many rare values.

Searching over different models

The parameter grid can be a list of dictionaries, and a step can itself be a parameter. This lets one search compare entirely different algorithms, each with its own settings:

param_grid = [
    {
        "model": [LogisticRegression(max_iter=1000)],
        "model__C": [0.1, 1.0, 10.0],
    },
    {
        "model": [RandomForestClassifier(random_state=42)],
        "model__n_estimators": [200, 500],
        "model__max_depth": [None, 6, 10],
    },
    {
        "model": [HistGradientBoostingClassifier(random_state=42)],
        "model__learning_rate": [0.05, 0.1],
        "model__max_leaf_nodes": [15, 31],
    },
]

model_search = GridSearchCV(clf, param_grid, cv=cv, scoring="roc_auc", n_jobs=-1)
model_search.fit(X_train, y_train)
print(model_search.best_params_["model"])

When the grid gets large: randomized search

Grid search grows multiplicatively. Five parameters with five values each is 3,125 combinations. RandomizedSearchCV samples a fixed number of configurations from distributions instead, and usually finds a near-optimal setting in a fraction of the time.

from scipy.stats import randint
from sklearn.model_selection import RandomizedSearchCV

rf_pipe = clone(clf).set_params(model=RandomForestClassifier(random_state=42))

param_distributions = {
    "model__n_estimators": randint(100, 800),
    "model__max_depth": [None, 4, 6, 8, 12],
    "model__min_samples_leaf": randint(1, 20),
    "model__max_features": ["sqrt", "log2", 0.5],
}

rand_search = RandomizedSearchCV(
    rf_pipe, param_distributions, n_iter=40,
    cv=cv, scoring="roc_auc", n_jobs=-1, random_state=42,
)
rand_search.fit(X_train, y_train)

For even larger spaces, HalvingRandomSearchCV (enabled via from sklearn.experimental import enable_halving_search_cv) or a dedicated library such as Optuna can prune poor configurations early.

Inspecting the search results

cv_results_ holds every configuration's scores. Loading it into a DataFrame shows whether the winner is clearly better or just lucky:

res = pd.DataFrame(search.cv_results_)
cols = ["params", "mean_test_score", "std_test_score", "rank_test_score"]
print(res[cols].sort_values("rank_test_score").head(5).to_string(index=False))

If the top five configurations are within one standard deviation of each other, prefer the simplest one, for example the one with stronger regularization (smaller C). Simpler models tend to generalize more reliably.

A note on nested cross-validation

search.best_score_ is slightly optimistic, because the same folds were used both to choose the configuration and to score it. For an unbiased estimate, wrap the search in an outer loop: cross_val_score(search, X_train, y_train, cv=outer_cv). It is expensive, but it is the correct number to report when a model's performance claim matters, such as in a paper or a go/no-go decision.

Custom transformers and feature engineering

Feature engineering belongs inside the pipeline too, otherwise it becomes another piece of code that must be duplicated at serving time. Scikit-learn gives you two ways to add your own logic: FunctionTransformer for stateless functions, and a custom class for anything that learns from data.

Stateless features with FunctionTransformer

The Titanic name column contains a title such as Mr, Mrs, Miss or Master, which encodes sex, age group and social status in one token. Family size is the sum of siblings, spouses, parents and children aboard. Neither needs to learn anything from the training data, so a plain function is enough. Write it in a module file, features.py, next to your notebook rather than in a notebook cell; the caveat at the end of this section explains why.

# features.py  (an importable module, not a notebook cell)
import numpy as np
import pandas as pd

def add_features(df: pd.DataFrame) -> pd.DataFrame:
    df = df.copy()
    df["title"] = df["name"].str.extract(r",\s*([^\.]+)\.", expand=False)
    df["family_size"] = df["sibsp"] + df["parch"] + 1
    df["is_alone"] = (df["family_size"] == 1).astype(int)
    df["has_cabin"] = df["cabin"].notna().astype(int)
    # to_numeric turns a missing fare sent as None into NaN for the imputer
    df["log_fare"] = np.log1p(pd.to_numeric(df["fare"], errors="coerce"))
    return df

Always copy the input inside the function. Mutating the caller's DataFrame causes confusing bugs when the same data is reused across folds. The pd.to_numeric call matters at serving time: a request with no fare arrives as None, which np.log1p cannot handle, so it is coerced to NaN and the imputer fills it like any other missing value.

Stateful features with a custom class

Titles are messy: there are a handful of common ones and many rare ones such as Countess, Major or Jonkheer. Grouping rare titles into "Rare" requires knowing which titles are rare, and that knowledge must be learned from the training data only. That makes it stateful, so it needs a class with fit and transform.

# features.py  (continued)
from sklearn.base import BaseEstimator, TransformerMixin

class RareCategoryGrouper(BaseEstimator, TransformerMixin):
    """Replace categories seen fewer than min_count times in training with 'Rare'."""

    def __init__(self, column: str, min_count: int = 10):
        self.column = column
        self.min_count = min_count

    def fit(self, X, y=None):
        counts = X[self.column].value_counts()
        self.frequent_ = set(counts[counts >= self.min_count].index)
        return self

    def transform(self, X):
        X = X.copy()
        X[self.column] = X[self.column].where(
            X[self.column].isin(self.frequent_), "Rare"
        )
        return X

Three conventions make this class a good scikit-learn citizen:

__init__ only stores arguments, unchanged and under the same names. get_params, set_params and clone rely on this, and it is what makes min_count searchable in a grid.

Learned state ends with an underscore (frequent_) and is created only in fit. Scikit-learn uses that convention to check whether an estimator is fitted.

fit returns self, which allows chaining such as RareCategoryGrouper("title").fit(X).transform(X).

Plugging it all together

The engineered columns are added before the ColumnTransformer, so the column lists simply include the new names.

from sklearn.preprocessing import FunctionTransformer
from features import add_features, RareCategoryGrouper  # import, never redefine

# Receives a DataFrame, returns a DataFrame with the new columns added
feature_engineer = FunctionTransformer(add_features)

numeric_features = ["age", "log_fare", "family_size"]
categorical_features = ["sex", "embarked", "pclass", "title"]
passthrough_features = ["is_alone", "has_cabin"]

preprocessor_v2 = ColumnTransformer(
    transformers=[
        ("num", numeric_pipeline, numeric_features),
        ("cat", categorical_pipeline, categorical_features),
        ("flags", "passthrough", passthrough_features),
    ],
    remainder="drop",
    verbose_feature_names_out=False,
)

full_pipeline = Pipeline(steps=[
    ("features", feature_engineer),
    ("rare_titles", RareCategoryGrouper(column="title", min_count=10)),
    ("preprocess", preprocessor_v2),
    ("model", HistGradientBoostingClassifier(random_state=42)),
])

scores = cross_validate(full_pipeline, X_train, y_train, cv=cv,
                        scoring=["accuracy", "roc_auc"], n_jobs=-1)
print(f"ROC AUC: {scores['test_roc_auc'].mean():.3f}")

The raw DataFrame still goes in; the pipeline extracts titles, groups rare ones using counts from each fold's training rows, imputes, encodes and models. You can now add "rare_titles__min_count": [5, 10, 20] to a parameter grid and let cross-validation decide where to draw the line.

This is why add_features and RareCategoryGrouper live in features.py and are imported into the notebook. Pickle saves a reference to where a function or class was defined, not its code. Anything defined in a notebook cell is recorded as belonging to __main__, and a serving process cannot find it there, even if it imports a copy under the same name. Lambdas are worse: standard pickle cannot save them at all.

Persisting and serving the pipeline

A fitted pipeline is the deployable unit: one file that contains every learned statistic, encoder vocabulary and model weight. The serving code loads it and calls predict on raw input, so training and serving can never drift apart.

Fit on all available data, then save

Before deploying, score the pipeline you are actually shipping, full_pipeline, on the held-out test set once. Then refit it on all labeled data so the deployed model learns from every example, and save it with joblib together with a small metadata file.

import json
import joblib
import sklearn
from sklearn.metrics import roc_auc_score

# One final look at the test set, for the pipeline being shipped
full_pipeline.fit(X_train, y_train)
test_auc = roc_auc_score(y_test, full_pipeline.predict_proba(X_test)[:, 1])
print(f"Test ROC AUC: {test_auc:.3f}")

final_model = clone(full_pipeline).fit(X, y)   # train + test combined

joblib.dump(final_model, "titanic_pipeline_v1.joblib")

# Record the environment alongside the artifact
metadata = {
    "sklearn_version": sklearn.__version__,
    "features": X.columns.tolist(),
    "cv_roc_auc": round(float(scores["test_roc_auc"].mean()), 4),
    "test_roc_auc": round(float(test_auc), 4),
}
with open("titanic_pipeline_v1.json", "w") as f:
    json.dump(metadata, f, indent=2)

The test-set score printed above is your performance estimate for the deployed model. Refitting on all the data usually helps a little, but you cannot measure that, because no held-out data is left. (The earlier test-set look at the plain hist_gb pipeline was for illustration; in a real project, keep the single final check for the pipeline you ship.)

Load and predict in a service

# serving/app.py
import joblib
import pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel

from features import add_features, RareCategoryGrouper  # same module training used

model = joblib.load("titanic_pipeline_v1.joblib")
app = FastAPI()

class Passenger(BaseModel):
    pclass: int
    name: str
    sex: str
    age: float | None = None
    sibsp: int = 0
    parch: int = 0
    fare: float | None = None
    cabin: str | None = None
    embarked: str | None = None

@app.post("/predict")
def predict(p: Passenger):
    row = pd.DataFrame([p.model_dump()])
    proba = float(model.predict_proba(row)[0, 1])
    return {"survival_probability": round(proba, 3)}

The request carries raw fields, including missing ones as None. The pipeline imputes them, extracts the title, encodes categories and scores the row exactly as it did during training. There is no second copy of the preprocessing logic to keep in sync.

Pitfalls when persisting

Version pinning. Pickled scikit-learn objects are only guaranteed to load with the same library version. Pin scikit-learn in the serving environment to the training version, and store the version in metadata as above.

Importable custom code. Pickle stores a reference to your function and class names, not their source. add_features and RareCategoryGrouper must live in a module (here features.py) that both the training code and the service import under the same name. Defining them in a notebook and importing a copy in the service is not enough.

Security. Loading a pickle or joblib file can execute arbitrary code. Only load artifacts from sources you trust. For a safer, more portable format, consider skops.io, or export to ONNX with skl2onnx when serving outside Python.

Schema checks. The pipeline will drop unknown columns and ignore unseen categories, which is robust but can hide upstream bugs. Validate incoming data (the Pydantic model above does this) and monitor the share of unknown categories and imputed values in production.

Best practices and common pitfalls

Most pipeline bugs come from a short list of habits. Use this as a review checklist before you trust a result or ship a model.

A few further habits keep pipelines clean as projects grow:

Set random_state everywhere randomness appears: splits, CV folds, models and randomized searches. Reproducibility is what makes comparisons trustworthy.

Use memory= for slow preprocessing. Pipeline(steps, memory="cache_dir") caches fitted transformers, so a grid search that only varies model parameters does not recompute expensive steps for every combination.

Name steps for the reader. preprocess, features and model beat auto-generated names when someone else reads your grid six months later.

Inspect the fitted pipeline. pipe.named_steps["preprocess"].get_feature_names_out() returns the final feature names, which you need for coefficients, feature importances and permutation importance.

Keep one pipeline per model version and treat it as the contract between data science and engineering. If the pipeline changes, the version number changes.

Conclusion

A clean scikit-learn pipeline is less about elegant code and more about correctness: it makes leakage structurally impossible, makes preprocessing tunable, and makes the trained model a single artifact that behaves identically in evaluation and production.

In this article you went from raw, messy data to a deployable model:

You saw how fitting preprocessing outside cross-validation can turn random noise into 80% "accuracy".

You routed numeric and categorical columns through tailored sub-pipelines with ColumnTransformer.

You chained preprocessing and a classifier into one estimator and compared models fairly against a baseline.

You evaluated with stratified cross-validation, multiple metrics and a single, final look at the test set.

You tuned preprocessing and model hyperparameters together with grid and randomized search.

You embedded feature engineering as stateless and stateful custom transformers.

You saved the pipeline and served it behind an API that accepts raw input.

The same structure scales well beyond Titanic. Swap in TfidfVectorizer for text columns, add SelectFromModel for feature selection, or replace the classifier with a regressor; the pattern of preprocess, model, evaluate and persist stays the same. Once every learning step lives inside the pipeline, your evaluation numbers become trustworthy, and your deployment becomes a file copy rather than a rewrite.

Further reading: the scikit-learn user guide chapters on , , and .

Back to all blogs