Piras et al.: SOM Directions are Better than One (arXiv:2511.08379, AAAI 2026).
One sentence: train a Self-Organizing Map on harmful residual vectors; each neuron minus the harmless centroid is a refusal direction. Ablating the set beats single DIM in their experiments.
Why SOM, not just PCA#
PCA finds orthogonal variance axes. Refusal may be a curved manifold (see TUM concept cones). A SOM places a grid of prototype neurons on that manifold. The paper proves SOMs generalize difference-in-means: a one-neuron SOM recovers DIM.
# harmful activations H_bad at hook layer ℓ
train SOM prototypes {m_i} on H_bad
c_good = mean(H_good)
r_i = normalize(m_i − c_good)
ablate span{r_1, …, r_k} # or top-k by separationPairwise cosine among r_i is often high (they all sit near global DIM). The win is covering spread along the manifold, not finding exotic orthogonal concepts.
When to use#
| Use SOM | Skip |
|---|---|
| Single Heretic pass leaves category-specific refusals | First run on a new 4B/8B |
| You already cache residuals (FailSpy / measure.py) | No GPU for a second analysis pass |
| Research comparison vs jailbreak baselines | Production deadline — use Heretic + domain [bad_prompts] first |
QCRI 2026 still applies: many directions may be style, one volume knob. SOM is worth it when leftovers are mechanistically different (factory vs SafetyCore), not when you just want a louder knob (that is α / max_weight).
Practical recipe (research)#
- Cache last-token residuals on harmful + harmless sets (same hook as DIM).
- Fit a small SOM (
grid=3×3or4×4) on harmful rows only. - Build
r_i, drop near-duplicates (cos > 0.95). - Ablate k=3–6 with ../methods/multi-direction-ablation.md or
--mode subspace. - Eval factory JSONL and XSTest — SOM can increase over-refusal if neurons sit on the over-refusal tail.
Official code: pralab/som-refusal-directions. That trainer is not Abliterix and not senbonzakura (a separate multi-direction CLI). Handbook abliterate-cxx --mode svd is a subspace proxy, not the paper SOM.