shiftshap

Monitor whether your model's SHAP explanations still hold as your data drifts.

shiftshap answers a question every team running a model in production eventually asks: are my model's explanations still trustworthy? Models live for months or years — they get retrained, upstream pipelines change, and feature distributions shift. When that happens, the model's reasoning quietly changes with it. The feature that drove your predictions last quarter may not be the one driving them today.

SHAP is excellent at explaining a model at a single point in time, but it has no built-in way to tell you how those explanations have changed between two points. Today people work around this by pickling explanation objects and writing their own comparison scripts, or by using data-drift tools that know nothing about SHAP's structure. shiftshap fills that gap.

---

Install

pip install shiftshap            # core
pip install shiftshap[plot]      # + matplotlib for the drift chart

Quickstart

If you already use SHAP, you already have everything you need. Take your SHAP values from two periods — training vs. production, or last month vs. this month — and pass them in:

import shiftshap

report = shiftshap.compare(reference_shap, current_shap)

print(report.summary())
#  2 of 5 features show HIGH explanation drift (0 medium).
#  Top driver changed from 'income' to 'balance'.
#  Overall rank stability (Spearman): 0.70.

print(report.details())          # plain-English narrative of the biggest movers
report.to_frame()                # full per-feature table
report.plot()                    # rank-drift bump chart
shiftshap.metric_definitions()   # what every metric means, in plain words

Inputs can be shap.Explanation objects, NumPy arrays, lists, or pandas DataFrames of shape (n_samples, n_features). Multi-class SHAP (3D arrays) is supported — by default classes are aggregated, or pass class_index=k to focus on one class. The two periods don't need the same number of samples.

Robust by design

shiftshap is built to survive real, messy production data. It handles NaNs (ignored with a note), zero-variance features, tiny samples (with an explicit "results unreliable" warning rather than false alarms), and multi-class outputs — and it fails with clear, actionable errors on genuinely broken input (infinities, mismatched feature counts, empty arrays) instead of cryptic stack traces.

What it tells you

For every feature, shiftshap reports:

• Importance drift — how the mean absolute SHAP value changed between periods.
• PSI (Population Stability Index) on the SHAP distribution — the industry-standard shift metric, with the accepted 0.2 threshold flagging significant drift.
• Rank drift — whether your most important features reordered, plus an overall Spearman rank-stability score.
• Severity — a high / medium / low label per feature, so the output is actionable at a glance.

And a bump chart showing how the feature-importance ranking shifted:

Why it matters

An explanation that has silently drifted is worse than no explanation — it gives false confidence. In regulated settings (finance, insurance, healthcare) teams are increasingly required to show that model explanations remain valid over time. shiftshap turns that check into two lines of code.

Roadmap

v0.1 deliberately does one thing well: compare two snapshots of SHAP explanations for tabular models. Planned next:

• Persistent explanation store for many time-points (not just two).
• Research-grade faithfulness-under-shift metrics beyond distributional PSI.
• Alerting hooks for monitoring pipelines.
• Support for image and text explanations.

Contributions and issues welcome.

License

MIT