The Hitchhiker's Guide to
Actionable Interpretability

Understanding models is important—but understanding alone isn't enough. We make the case that interpretability research should be evaluated by what it enables: concrete decisions, interventions, and improvements beyond the field itself.
📄 Blog companion to: “Interpretability Can Be Actionable” [PDF] 📅 February 2026
This post is a companion to our paper. It covers the key ideas; see the full paper for the complete analysis and references. Our framing draws in part on discussions from the ICML 2025 Workshop on Actionable Interpretability, which aimed to foster dialogue on leveraging interpretability insights to drive tangible advancements in AI.
Interpretability has grown rapidly as a research area, but most of it hasn't led to practical improvements in models, deployments, or policy. This post summarizes our position arguing that actionability—the extent to which insights enable concrete decisions and interventions—should become a core evaluation criterion for interpretability work.

The post walks through the barriers, where the opportunities are, and gives a concrete checklist to make your own work more actionable.

The Problem

Actionability checklist for interpretability research
Actionability checklist for interpretability research.

There are hundreds of interpretability papers published every year. Probing, circuits, sparse autoencoders, feature attribution — the field is thriving. This growth is driven by the intuition that if we understand how models work, we should be able to make them safer, more reliable, and better aligned with what we actually want.

But here's the uncomfortable truth: most of this work hasn't translated into things used outside of interpretability research itself. Insights rarely inform changes to models, training procedures, deployment decisions, or policy. Papers get published, methods get cited—but predominantly in a conceptual way. Most citations don't credit interpretability work for changes to training, architecture, or evaluation.Mosbach et al. (2024) conducted an extensive analysis showing that while interpretability papers are frequently cited, their practical influence on downstream ML work remains limited.

This has motivated growing calls to focus on clearly demonstrable outcomes beyond "understanding" itself. We argue that what is missing is not methods, but evaluation criteria: a shared framework for determining when interpretability research is successful from a practical, decision-oriented perspective.

Defining Actionable Interpretability

We define an interpretability-oriented work as actionable if it produces insights about an AI model that inform or guide actions toward non-interpretability objectives. In plain terms: your work is actionable if someone can take what you found and do something useful with it—improve a model, make a deployment decision, inform a policy, or help a domain expert.

Actionability isn't binary. We characterize it along two key dimensions:

Two Dimensions of Actionability

📐 Concreteness

How specific is the proposed action?

Vague
Precise

“could inform safety research” → exact implementation with code

✅ Validation

Has anyone actually tested it?

Untested
Validated

“this might help” → quantitative evidence it actually does

Most existing work clusters in the low-concreteness, low-validation region—providing directional insights that motivate future work, but without articulating or testing specific actions. The field needs more work at the high end of both axes: precise, validated actions informed by interpretability. In the paper, we map existing work onto this space to show where current contributions land and what the gaps look like.

💡 What high actionability looks like in practice

For more examples, see the posters from the ICML 2025 workshop.

Why Isn't Interpretability Actionable Yet?

Several barriers reinforce a cycle where actionability isn't prioritized, methods lack validation, and deployment yields little feedback. We discuss these in detail in Section 3 of the paper; here's a summary.

🏛 Misaligned Incentives

Publication standards don't require actionability. Application-focused work is under-rewarded, often dismissed as "merely engineering." Unlike mainstream ML with its benchmarks, interpretability lacks clear signals of success.

🔬 Methodological Limitations

Many studies use oversimplified setups and small models. Rigorous comparisons against non-interpretability baselines (like prompting or fine-tuning) are rare. For example, AxBenchWu et al. (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025. showed these simpler methods often outperform interpretability methods for LLM steering. Recent benchmarks like MIBMueller et al. (2025). MIB: A Mechanistic Interpretability Benchmark. ICML 2025. have begun addressing these gaps by enabling head-to-head comparisons.*These issues are not unique to interpretability, but unlike applied ML—where benchmark performance provides immediate feedback—interpretability lacks a forcing function that drives practical validation. Without head-to-head comparisons against non-interpretability baselines, it's hard to know whether insights are genuinely useful.

🚀 Deployment Challenges

Employing interpretability methods requires deep expertise in model internals and specialized libraries—a barrier that keeps most practitioners from adopting them, especially when simpler alternatives exist. Additionally, most techniques assume open access to weights and activations, creating a fundamental tension: interpretability is most urgently needed for powerful frontier models, yet these are precisely the models that remain proprietary and resistant to such analysis.

Five Domains Where Interpretability Has Real Leverage

Where should the field focus? We identify five domains where interpretability provides a fundamental advantage—where answering why questions about models unlocks improvements that other approaches cannot.

We further discuss these opportunities in Section 4 of the paper

Five opportunities for actionable interpretability: Problems scaling does not solve, Alignment, Translation to meaningful concepts, Architectural design, and Surgical interventions
Five domains where interpretability offers unique leverage to drive concrete improvements.
🧩

Problems Scaling Won't Fix

Certain failure modes persist or even worsen with model and data scale, including hallucinations, catastrophic forgetting, biases, and adversarial brittleness. The persistence of these failures across model scales suggests they are fundamental to our current modeling paradigm rather than due to limited capacity. Interpretability offers a path forward precisely because it can identify why models fail.

🎯

Alignment

As AI systems become more capable, ensuring they behave as intended becomes more critical and more difficult. Alignment today still relies on fine-tuning and data curation rather than understanding-driven interventions, but as AI progresses, verifying that AI goals match human goals will shift from aspiration to necessity.

🔧

Surgical Interventions

Retraining a flawed model is expensive and risks introducing other unexpected outcomes. Interpretability enables targeted modifications; identifying components responsible for unwanted behaviors allows surgical fixes while preserving other functionality.

🏗

Architecture Design

Current improvements emerge largely through trial and error—an inefficient, opaque process where success may not scale or transfer to new domains. Interpretability could accelerate progress by narrowing the space of plausible architecture modifications, reducing both labor and compute required.

🔍

Meaningful Explanations

The most natural role of interpretability is explaining model behavior, yet translating internal signals into meaningful concepts remains a critical bottleneck. In high-stakes domains like healthcare, a radiologist needs to know if an AI-assisted diagnosis depends on clinically relevant features, not which pixels activate; automated methods that translate technical explanations into domain-appropriate, actionable concepts could unlock interpretability's core promise.

Who Takes the “Action” in “Actionable”?

Different stakeholders have different capabilities and motivation. An interpretability work becomes more actionable if it is explicit about its intended audience and the decisions it aims to support.

Audience Example Action What They Need
AI Developers Curate data, edit model behavior Data-point analysis, modification methods
Deployment Engineers Debug application failures Explanations for model errors
Domain Experts Validate reasoning, refine workflows Explanations tied to domain features
End Users Trust or override model output High-level rationale in human terms
Policymakers Enforce compliance and transparency System-level summaries

These actors rarely operate in isolation. A clinician's feedback about unreliable explanations may reveal failure modes to engineers. A policymaker's compliance requirements may drive developers toward specific mitigations.

What Actions Does Interpretability Enable?

We classify actions by what they affect. Click each category to explore the specific actions interpretability unlocks.

⚙️ Modify Model Output ▼

Decisions that directly change model behavior—modifications to training data, inputs, weights, or internal computations. Primarily made by developers and researchers with access to model internals.

🗂 Data Curation

Interpretability can identify which training examples help and which hurt.

  • Influence Functions trace model performance back to individual training examples, enabling targeted data selection.
    • Removing detrimental robot demonstrations achieved state-of-the-art results with only 33% of the original data.Agia, C. et al. (2025). CUPID: Curating Data Your Robot Loves with Influence Functions. CoRL 2025.
🎛 Direct Control

Interpretability can identify components responsible for specific behaviors, enabling targeted interventions.

  • Model Editing — Modifies weights to correct behaviors without full retraining.Meng, K. et al. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022.
  • Runtime Interventions — Steer activations along interpretable directions at inference time.Li, K. et al. (2023). Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. NeurIPS 2023.Turner, A.M. et al. (2023). Steering Language Models with Activation Engineering.

These are just two examples — we cover additional actions including model input selection, training decisions, safety interventions, and more in Section 5 of the paper.

🚀 Deployment & Use ▼

These change what humans do with model predictions—when to trust them, when to override them, and how to integrate them into workflows. Actions here are taken by end users, domain experts, and deployment engineers.

🩺 End User Decisions

Interpretability helps users understand when to trust and when to override model outputs.

  • Neuro-Symbolic Systems — Combining LLMs with rule-based expert systems provides the transparency radiologists need to confidently use AI while maintaining oversight.Prenosil, G.A. et al. (2025). Neuro-Symbolic AI for Auditable Cognitive Information Extraction from Medical Reports. Communications Medicine.
  • Uncertainty Estimation — Internal representations enable users to detect potential errors and decide when to trust model outputs.Kadavath, S. et al. (2022). Language Models (Mostly) Know What They Know.
🔀 Deployment Decisions

Internal mechanisms support routing decisions—whether to return a model's answer or escalate to alternatives.

  • OOD Detection — Internal causal mechanisms can identify out-of-distribution failures.Huang, J. et al. (2025). Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors.
  • Error Prediction — Interpretability methods can predict errors on unseen distributions, informing deployment boundaries.Li, V.R. et al. (2025). Can Interpretation Predict Behavior on Unseen Data?

More examples — including uncertainty-based routing and backdoor detection — in Section 5 of the paper.

🔮 Shape Future Practice ▼

Beyond immediate interventions, interpretability informs how the field builds and governs future systems. This has longer-term, broader impact across policy, science, and architecture.

🧠 Learning from Superhuman Models

When models exceed human expertise, interpretability becomes a mechanism for transferring knowledge from AI back to humans.

  • Concept Vectors — Vectors extracted from AlphaZero surfaced novel chess strategies that human grandmasters could learn from.Schut, L. et al. (2025). Bridging the Human–AI Knowledge Gap Through Concept Discovery and Transfer in AlphaZero. PNAS.
🏗 Development of Future Models

Interpretability can shift architecture design from trial-and-error toward principled engineering.

  • Induction Heads — The discovery of induction heads in TransformersOlsson, C. et al. (2022). In-Context Learning and Induction Heads. Transformer Circuits Thread. provided a mechanism for in-context learning that traditional state-space models lacked, directly influencing the selective state-space design of the Mamba architecture.Gu, A. & Dao, T. (2024). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. COLM 2024.

We also discuss policy and regulation implications — including the EU AI Act and GDPR — in Section 5 of the paper.

How to Evaluate Actionability

Now to a core part of creating actionable interpretability work: being able to evaluate whether it's actually actionable. Current practice often evaluates interpretability methods against each other—"grading on a curve." This is insufficient. We need metrics that measure whether insights actually enable better decisions and outcomes. Here are the criteria we propose — we discuss each in detail, including how to measure them in practice, in Section 6 of the paper:

Modify Outputs
Deployment
Future Practices

Comparative Utility

Does the interpretability-based method outperform standard baselines like prompting or fine-tuning? Actionability means marginal leverage over simpler methods.

Mechanistic Faithfulness

Does intervening on identified components produce predicted changes—altering target behavior while leaving unrelated behavior intact?

Generalization

Does the insight hold across seeds, perturbations, architectures, and scales without requiring rediscovery?

Specificity

When you intervene on an identified component, does it affect only the targeted behavior—or does it also disrupt other capabilities? Broad side effects signal that the finding is entangled, not specific.

Task Enhancement

The most direct user-facing test: do explanations improve how humans perform the task the model supports — their accuracy, speed, or ability to know when to trust or override the model? This typically requires human-subject evaluations, and prior work suggests the bar is harder to clear than it sounds.

Understandability

Can the target audience actually understand the explanation? A technically faithful explanation is useless if a clinician or policymaker can't make sense of it. Understandability is orthogonal to correctness — an explanation can accurately reflect model behavior and still be completely unusable.

Reliability

Are explanations stable across random seeds and minor perturbations? Even explanations that are faithful and understandable become useless if they fluctuate unpredictably.

Governance Utility

In a policy context, interpretability isn't a scientific diagnostic — it's an institutional lever. Does the method enable practical governance actions: safety audits, compliance verification, or detecting dangerous mechanisms? Does it reduce monitoring costs compared to blunt instruments like pausing deployment? Can regulators and safety teams actually use it?

What We're Not Saying

We are not arguing that all interpretability work must immediately yield actionable outcomes, or that purely exploratory work lacks value. Curiosity-driven research is vital—we don't yet know which techniques will ultimately prove useful.

What we are saying is that tracking actionability as a yardstick will strengthen the field's impact, hold methods to higher standards, and provide evidence that findings reflect genuine model behavior rather than analysis artifacts. Methodological novelty and application demonstration are not at odds—grounding findings in real-world actions provides stronger evidence that the interpretability insights are real.

The burden now falls on the research community: to reward actionable contributions alongside explanatory depth, to establish evaluation criteria that track the utility of interpretability insights, and to build infrastructure that connects understanding to impact.Haklay et al. (2025). 1st Actionable Interpretability Workshop at ICML 2025. We discuss several counter-arguments — including whether safety should be the only actionable goal, and whether there's decisive evidence interpretability methods outperform alternatives — in Section 7 of the paper.

The Actionability Checklist

If you're working on interpretability, ask yourself these questions:

📋 Actionability Checklist for Interpretability Research

Click on each step to learn more.

1
Define a clear goal Identify a specific problem that your interpretability question aims to eventually solve.
Don't start with a method—start with a problem. What failure mode, safety concern, or practical limitation does your work address? For example, rather than "we analyze attention patterns," ask: "Can we identify why the model hallucinates on medical queries, and use that to reduce hallucination rates?" A clear goal anchors your research in something that matters beyond interpretability itself and makes it easier to evaluate whether you've succeeded.
▼
2
Identify your audience Communicate insights according to different stakeholders: developers, practitioners, policymakers.
Your interpretability insights may be acted upon by very different stakeholders. AI developers need data-point level analysis and behavior modification methods. Domain experts like clinicians need explanations tied to domain-specific features. Policymakers need system-level summaries. Each audience requires different framing, language, and levels of abstraction. An interpretability work becomes more actionable when it is explicit about who can act on its findings and how.
▼
3
Propose concrete actions Articulate what decisions or interventions your insights enable.
Go beyond "this could be useful for safety." Specify: does your insight enable data curation (identifying harmful training examples), model editing (surgically correcting a behavior), deployment decisions (routing uncertain inputs to human review), or policy compliance (auditing for fairness)? The more precise the proposed action, the easier it is for others to build on your work. Provide code or explicit instructions where possible—technical complexity is a major barrier to adoption.
▼
4
Validate empirically Implement the proposed action yourself and demonstrate its effects.
Don't just propose an action—carry it out. If you claim your method can remove a bias, show the bias is reduced. If you claim it identifies failure-prone inputs, demonstrate improved routing decisions. It also provides evidence that your insights reflect genuine model behavior rather than artifacts of a particular analysis setup.
▼
5
Evaluate in realistic settings Apply methods to large-scale models and non-synthetic datasets.
Much interpretability research uses simplified tasks and small models as controlled testbeds, but insights from these settings may not transfer. Test on frontier-scale models and naturalistic data where possible. For example, many mechanistic studies focus on single next-token predictions, whereas real usage involves multi-token generation. Bridging this gap is essential for demonstrating that your findings are practically relevant beyond toy setups.
▼
6
Use actionable success criteria
  • Surpasses standard baselines (prompting, fine-tuning)
  • Generalizes across setting variations and seeds
  • Produces targeted effects without degrading other capabilities
  • Yields useful explanations for the target audience
For example, don't just compare your interpretability method against other interpretability methods—compare against standard ML baselines like prompting or LoRA fine-tuning. Does steering with SAEs improve refusal behavior more than targeted prompting? Also check that your findings generalize (across seeds, architectures, scales), are specific (interventions don't cause broad side effects), and produce explanations that your target audience can actually understand and use.
▼
This post covers the key ideas — the paper has the full picture. 📄 Read the paper

Citation

@article{orgad2026actionable,
  title     = {Interpretability Can Be Actionable},
  author    = {Orgad, Hadas and Barez, Fazl and Haklay, Tal
               and Lee, Isabelle and Mosbach, Marius
               and Reusch, Anja and Saphra, Naomi
               and Wallace, Byron C. and Wiegreffe, Sarah
               and Wong, Eric and Tenney, Ian and Geva, Mor},
  year      = {2026},
  url       = {https://actionable-interpretability.github.io}
}