Choosing Between LIME, SHAP, and Permutation Importance: A Practical Guide to Model-Agnostic Explainability

This guide compares LIME, SHAP, and Permutation Importance, three model-agnostic techniques for explaining black-box models. It helps you choose the right method based on whether you need local or global explanations, computational budget, and data characteristics.

Sarah Chen

October 17, 2026 · 6 min read

Choosing an explainability technique for a black-box model comes down to three questions: do you need to explain one prediction or the whole model, what kind of data are you working with, and how much computation can you afford? For most projects, the answer lands on one of three model-agnostic methods. LIME delivers fast, local explanations of individual predictions. SHAP offers mathematically grounded feature attribution that works both locally and globally, at a higher computational price. Permutation Feature Importance gives a straightforward global view of which features matter, with no per-instance explanations at all.

These methods share one defining property: they treat the model as a function that maps inputs to outputs, without needing to inspect its internal architecture, as practitioner guides to explainable AI emphasize. That is what makes them portable across algorithms. This guide compares the three techniques across interpretability scope, core mechanism, typical application, and computational cost, then walks through the decision points that should drive your selection.

LIME: Local Explanations for Any Model

Local Interpretable Model-Agnostic Explanations rests on a simple insight: while a complex model's global behavior may be incomprehensible, its local behavior around any specific prediction can be approximated by a simple, interpretable model. LIME fits a local linear model around the prediction you want to explain, and the coefficients of that surrogate tell you which features pushed the output in which direction.

The scope is strictly local. A perspective piece comparing SHAP and LIME, published on arXiv, notes that LIME generates one plot per instance, while SHAP produces several plots reporting outcomes both locally and globally. That per-instance character is LIME's strength and its limitation: it is ideal when a stakeholder asks "why did the model make this prediction?" but poorly suited to summarizing overall model behavior.

Its speed is a genuine advantage. The same comparison finds LIME much faster than SHAP, especially with tree-based models, with computational complexity on the order of O(NLD), where N is the number of perturbations, L the features, and D the dimensions. The trade-off is fidelity: because LIME fits a local linear model, it can fail to capture non-linear associations, whereas SHAP may detect them depending on the underlying model.

In practice, LIME has been used to assess prediction reliability in high-stakes settings. A systematic review of model-agnostic XAI in finance, authored by Farhina Sardar Khan and colleagues and published in Artificial Intelligence Review, cites work by Çelik et al. (2023) that used LIME to assess prediction reliability and prevent erroneous decision-making in stock market forecasting on the KOSPI dataset. That pattern—spot-checking individual predictions before acting on them—is where LIME fits best.

SHAP: Game Theory for Global and Local Insights

SHapley Additive exPlanations takes a different foundation: cooperative game theory. It attributes each feature's contribution to a prediction using Shapley values, which consider all combinations, or coalitions, of features to cover every case where a feature acts alone or alongside subsets of the others. A biomarker-identification study published through PMC describes Shapley value-based methods like SHAP as providing localized feature characterization based on game theory.

The payoff is consistency. Because every coalition is accounted for, the attribution is fair by construction, and the same framework yields both local explanations for individual predictions and global summaries of model behavior. The arXiv perspective notes SHAP has been applied widely across domains to explain model outcomes either locally or globally, and that it may capture non-linear associations depending on the model used.

The cost is computational. Exact Shapley values grow in complexity as the number of features increases, which motivated the Kernel SHAP approximation. Even so, the PMC study characterizes Shapley value-based methods as computationally intensive, and the arXiv comparison gives Kernel SHAP complexity on the order of O(TLD), where T is the number of samples, L the features, and D the coalitions. For large datasets, this can become a bottleneck.

Two caveats deserve attention. The arXiv authors flag that SHAP is a model-dependent method in how its outputs should be interpreted, and that its behavior warrants care when features are collinear. The PMC study adds that Shapley-based methods do not guarantee a valid statistical test of importance. SHAP is nonetheless the method of record in domains where rigor matters: the finance review by Khan and colleagues cites Bussmann et al. (2020) using Shapley values to interpret AI predictions for peer-to-peer lending risk management.

Making Your XAI Choice: A Decision Framework

Select the XAI technique that best aligns with the project's interpretability goals (local vs. global), data characteristics, and available computational resources. Work through the decision points in order.

Model-agnostic XAI techniques compared across selection criteria
Technique	Interpretability Scope	Core Principle	Typical Application	Computational Cost
LIME	Local only; one plot per instance	Approximates the black-box model locally with an interpretable linear model	Assessing reliability of individual predictions, e.g., stock market forecasting (KOSPI dataset)	Lower; roughly O(NLD), faster than SHAP especially on tree-based models
SHAP	Both local and global	Shapley values from game theory; considers all feature coalitions	High-stakes interpretation, e.g., peer-to-peer lending risk management	High; Kernel SHAP roughly O(TLD), a bottleneck on large datasets
Permutation Feature Importance	Global only; no per-instance explanations	Shuffles a feature's values and measures the increase in prediction error	Global feature relevance audits, e.g., financial forecasting models	Relatively low; repeated model evaluations per feature

Local or global question? If you must justify individual predictions—to a regulator, a domain expert, or an affected party—LIME or SHAP is required. If you only need to know which features drive the model overall, permutation importance is sufficient and cheaper.

Both scopes needed? If you need per-instance explanations and global summaries from one framework, SHAP is the only technique of the three that provides both, provided you can absorb its computational cost.

Computational budget? With constrained resources, the arXiv comparison indicates LIME is much faster than SHAP, especially with tree-based models. Kernel SHAP's O(TLD) complexity can be a bottleneck on large datasets.

Non-linear relationships suspected? LIME's local linear approximation may fail to capture non-linear associations; SHAP may detect them depending on the model. If feature importance is your goal and non-linearity is a concern, PFI sidesteps the surrogate-model question entirely.

Audience and stakes? The finance review by Khan and colleagues links these choices to transparency and regulatory compliance in high-stakes domains, where SHAP's rigorous attribution has been the method of record in applications like peer-to-peer lending.

A common pattern is to combine methods: run permutation importance first for a global audit, then apply LIME or SHAP to the individual predictions that matter most. The measurable indicator of a sound choice is simple: the chosen XAI technique successfully generates explanations that are understandable by the target audience and meet the project's specific interpretability requirements. If your explanations answer the question your stakeholders actually ask—about one prediction or about the model as a whole—within your compute budget, the selection is right.

Choosing Between LIME, SHAP, and Permutation Importance: A Practical Guide to Model-Agnostic Explainability

LIME: Local Explanations for Any Model

SHAP: Game Theory for Global and Local Insights

Permutation Feature Importance: Simplicity and Robustness

XAI Technique Selection Matrix

Making Your XAI Choice: A Decision Framework

Sources

Tags

Related articles

Why AI Agents Can't Trust Their Own Reflection: Choosing a Self-Correction Architecture

Sparse vs. Dense Attention: How Transformers Trade Cost for Context

Choosing Between On-Device and Split Inference: A Practical Guide to Edge AI Optimization

CoT, Self-Consistency, or RAG: Choosing the Right Advanced Prompting Technique for Your Enterprise LLM