SAELens Sparse Autoencoder Training
Train and analyze sparse autoencoders to find interpretable features inside language models.
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Stars
- 14,195
- Licence
- MIT
- Files
- 4 files
What it does
Sparse autoencoders (SAEs) are a research technique for breaking a language model's internal activations into separate, human-readable features, such as concepts like legal language or code syntax. Once this skill is installed, the agent can guide you through loading pre-trained SAEs, training a custom one with the SAELens library, and analyzing or steering individual features. It is a guidance document for interpretability research rather than a standalone tool.
Great for
- ML researchers studying mechanistic interpretability in transformer models
- Students learning how superposition and sparse features work
- Engineers who already use TransformerLens and want to add SAE analysis
Skip it if
- Users who just want a chatbot or general coding help, since it only applies to model interpretability work
- Production safety decisions based on feature labels, because SAE features are research artifacts and their interpretations can be wrong
- People without GPUs or a Python environment, since the workflows assume CUDA and substantial compute for training
Try asking
- “Load gpt2-small and a pre-trained SAE for layer 8 and show the top active features for each token in a sentence”
- “Help me set up a custom SAE training run with an expansion factor of 8 and L1 warm-up”
- “Explain how to check whether my SAE has too many dead features and what L0 target to aim for”
- “Walk me through analyzing and ablating a single feature to see its effect on model output”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 4 original files, the MIT licence and an attribution note.
saelens.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip saelens.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip saelens.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Check that the SAE you load matches the model and hook point exactly, since mismatched SAEs produce meaningless features
- Treat the 70% interpretability figure as a reference from Anthropic's research, and verify feature labels yourself before relying on them
- Start with a pre-trained SAE before committing compute to training a custom one
What's inside
- references/api.md
- references/README.md
- references/tutorials.md
- SKILL.md
Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.