Safetensor abliteration pipeline

How alignment refusal directions are removed from Hugging Face checkpoint files (*.safetensors) — the correct surgery surface.

Target tensors#

Heretic discovers layer kernel automatically. llm-abliteration uses YAML per destination layer.

Math (applied in-place on weights)#

Projected direction (Heretic default, orthogonalize_direction = true):

g = normalize(mean(h_harmless))
r = normalize(mean(h_harmful) - mean(h_harmless))
r_proj = normalize(r - (r·g)g)
W' = W - λ · outer(r_proj, r_proj @ W)   # on down_proj / o_proj rows

Norm-preserving (row_normalization = "full"): approximate row magnitude restoration via rank-r LoRA correction bundled in output shards.

→ ../techniques/projected-norm-preserving-abliteration.md

Pipeline A — Heretic (automatic)#

pip install heretic-llm==1.4.0 bitsandbytes accelerate safetensors
cp sources/heretic-tools/config.production.toml config.toml
heretic ./models/Qwen3-4B-Instruct-2507

On completion, choose save path — writes standard HF layout:

out/
├── config.json
├── model.safetensors          # or sharded model-00001-of-00004.safetensors
├── tokenizer.json
├── generation_config.json
└── (optional lora_* keys if row_normalization=full)

Backup rule: copy source to *-ORIGINAL before run. Never overwrite original shards.

Pipeline B — llm-abliteration (manual)#

python measure.py -m ./base -o directions.pt --quant 4bit --projected
python analyze.py directions.pt -c
python sharded_ablate.py my-model.yml --projected --normpreserve

sharded_ablate.py reads full-precision safetensors one shard at a time, writes new safetensors to output dir.

Critical: 4-bit quantized checkpoints are for measurement only. Ablation mutates FP16/BF16 weights.

Pipeline C — Hook prototype (no safetensor edit)#

TransformerLens / HF hooks subtract (h·r)r at inference — produces direction.pt only.

Use to validate prompts before permanent edit:

→ residual-hook-ablation.md · ../instructions/quickstart.md

Verification before GGUF export#

# Spot-check shard keys unchanged count
python -c "
from safetensors import safe_open
for p in ['ORIGINAL/model.safetensors','ABLITERATED/model.safetensors']:
    with safe_open(p, framework='pt') as f:
        print(p, len(f.keys()))
"

# Heretic built-in
heretic --evaluate-model ./ABLITERATED

Common mistakes#

Next step#

→ gguf-export-notes.md · ../docs/toolchain-safetensors-gguf-lora.md