TransformerLens

Use it when

• You need to cache, edit, ablate, or replace internal activations of an open-source language model to reverse engineer the algorithms it learned
• You are running mechanistic interpretability experiments such as activation patching, logit attribution, or induction head analysis on GPT-2 style models

What it solves

Not the fit when

• serving or fine-tuning models in production
• inspecting closed API models where weights are unavailable
• training sparse autoencoders, which now lives in SAELens
• llm-math-mismatch
• structured-output-drift
• multi-model-routing

Install

pip install transformer_lens (pip install 'transformer_lens~=2.0' for Python 3.8/3.9)

Invoke

from transformer_lens.model_bridge import TransformerBridge; bridge = TransformerBridge.boot_transformers("gpt2", device="cpu"); logits, activations = bridge.run_with_cache("Hello World")

Alternatives

No reviewed alternatives recorded yet.