Abliteration HandbookField guide
Tools & environments

Abliteration & low-VRAM tooling catalog

Tools for weight surgery, 4-bit measure, LoRA export, quantized inference, and agent deployment. GitHub-first — see ../../references.md.

9 min read 6 examples Source-linked

In this chapter
Interactive companion

Explore the chapter map

Use the visual controls alongside the complete source chapter.

Tools for weight surgery, 4-bit measure, LoRA export, quantized inference, and agent deployment. GitHub-first — see ../../references.md.


Core abliteration (safetensors surgery)#

ToolURLRoleVRAM notes
Hereticp-e-w/hereticPrimary — automatic Optuna + projected + norm-preservingbnb_4bit, CPU offload; PyPI heretic-llm
Heretic pins (this repo)heretic-tools-reference.mdImmutable configs, HF registryOffline copy-paste
llm-abliterationjim-plus/llm-abliterationv1.2 (Jan 2026) measure → analyze → sharded ablate4-bit measure; full-weight ablate
refusal_directionandyrdt/refusal_directionPaper reproductionResearch GPU
remove-refusals-with-transformersSumandora/remove-refusals-with-transformersPure HF, no TransformerLensMedium
abliterate.cppkabachuha/abliterate.cppGGUF-native direction measure → llm-abliterationWIP experimental
Abliterixwuwangzhang1216/abliterixOptuna TPE multi-objective; 150+ configs; MoE/VL/SSM; LoRA/ORBA/SAEHeretic-lineage; HonestAbliterationBench; AGPL-3.0
ErisForgeTsadoq/ErisForgeLayer-band single-pass; ExpressionRefusalScorerGSM8K-friendly subset; lower arch coverage
TransformerLensTransformerLensOrg/TransformerLensHooks, direction probes1–3B on 8 GB
wassname/abliteratorwassname/abliteratorCommunityLegacy
FailSpy/abliteratorFailSpy/abliteratorTransformerLens hooks; temp/permanent ablationNotebook prototyping; narrow arch support
Nous llm-abliterationNousResearch/llm-abliterationFrozen jim-plus fork (last push ~2025-11-27)Historical YAML only — manual path is jim-plus v1.2
OBLITERATUSelder-plinius/OBLITERATUSSVD + entanglement analysis; surgical CoT-aware modeHermes Agent skill
huihui_aiollama.com/huihui_aiSumandora abliterates; layer-banded GGUF dropsFast uncensor, weak tool QA
199-biotech Gemma 4gemma-4-abliteratedPer-layer MLX pipeline; weight factor 1.0Gemma 4 31B quality-first
ablate-llmAIAnytime/ablatepip install ablate-llm KL search + Hub pushNot in Young matrix — eval locally
J-WashExtraltodeus/J-WashJacobian-lens manual alignment, exportableResearch / visualization
Blasphemersunkencity999/blasphemerHeretic fork, macOS + LM StudioApple Silicon automation
Handbook C++26 CLI../../cxx/README.mdDIM / ORBA / subspace / hooks / evalnpm run cxx:self-check

Master toolchain doc: ../toolchain-safetensors-gguf-lora.md · Safetensors: ../../methods/safetensor-abliteration-pipeline.md

Workflows: ../../instructions/heretic-workflow.md · ../../instructions/low-vram-abliteration.md

Extended toolkit (Abliterix, ErisForge, jim-plus + --deccp, FailSpy): comparison table, commands, decision tree → ../../techniques/extended-abliteration-toolkit.md


Quantization & memory#

ToolURLRole
bitsandbytesbitsandbytes-foundation/bitsandbytes4/8-bit load for measure & infer
Acceleratehuggingface/acceleratedevice_map, max_memory offload
auto-gptqAutoGPTQ/AutoGPTQGPTQ quant (post-abliteration)
autoawqcasper-hansen/AutoAWQAWQ quant
llama.cppggml-org/llama.cppGGUF convert + llama-quantize
Ollamaollama/ollamaLocal GGUF serving
LM Studiolmstudio.aiGUI GGUF (desktop)

Typical post-abliteration chains:

bash
# Merged full weights → Ollama
convert_hf_to_gguf.py → llama-quantize Q4_K_M → ollama create

# LoRA sidecar → llama.cpp (keep base GGUF)
export-abliteration-lora.py → convert_lora_to_gguf.py → llama-cli --lora

# Browser LoRA convert
# huggingface.co/spaces/ggml-org/gguf-my-lora

→ ../../methods/gguf-export-notes.md


LoRA / QLoRA stack#

ToolURLRole
PEFThuggingface/peftLoRA adapters; merge for GGUF
export-abliteration-lora.py../../scripts/export-abliteration-lora.pyΔW → adapter safetensors
convert_lora_to_gguf.pyggml-org/llama.cppPEFT LoRA → GGUF sidecar
GGUF-my-LoRAspaces/ggml-org/gguf-my-loraWeb convert PEFT → GGUF
grimjim abliteration LoRALlama-3-Instruct-abliteration-LoRA-8BRefusal removal as trained adapter
Unslothunslothai/unslothFast 4-bit QLoRA + save_pretrained_gguf
TRLhuggingface/trlDPO/SFT trainers (Jarvis repair)
Axolotlaxolotl-ai-cloud/axolotlYAML-driven QLoRA fine-tunes
torchtunepytorch/torchtuneMeta fine-tuning recipes
Use caseTool
Heretic norm-preserving LoRABuilt into Heretic row_normalization = full
ΔW → adapter export../../methods/lora-adapter-export.md
Post-abliteration tool repairJarvis pack + Unsloth/TRL QLoRA

Theory: ../../techniques/lora-qlora-abliteration.md


Inference servers (agents)#

ToolURLRole
vLLMvllm-project/vllmHigh-throughput API; LoRA slots
llama.cpp serverggml-org/llama.cppCPU/GPU GGUF
text-generation-webuioobabooga/text-generation-webuiUI + extensions
ExLlamaV2turboderp/exllamav2Fast 4-bit CUDA infer
koboldcppLostRuins/koboldcppPortable GGUF

CyberGym / OpenHands: point agent at OpenAI-compatible endpoint (vLLM or llama.cpp --api).


Apple Silicon & CPU-only#

ToolURLRole
mlx-lmml-explore/mlx-examplesMLX quant + LoRA on Mac
llama.cpp (Metal)ggml-org/llama.cppBest Mac GGUF path

Surgery on Mac: prefer cloud Heretic + local GGUF.


ToolURLRole
Optunaoptuna/optunaHeretic TPE search backend
PaCMAPYingfanWang/PaCMAPHeretic residual plots ([research] extra)

Interpretability & advanced direction finding#

ToolURLRole
GemmaScopegoogle/gemma-scopeJumpReLU SAEs for refusal latents
andyrdt SAEssaes-llama-3.1-8b-instructLlama 3.1 residual SAEs
TransformerLensTransformerLensOrg/TransformerLensHooks, activation cache
repeng / RepEarxiv 2310.01405Representation engineering vectors
TUM geometry-of-refusalcs.cit.tum.de/daml/geometry-of-refusalGradient RDO code
GraySwan circuit-breakersGraySwanAI/circuit-breakersDefensive training (contrast)
ErisForgeTsadoq/ErisForgeLayer-band ablation + refusal scorer
Abliterixwuwangzhang1216/abliterixMulti-technique automation; HonestAbliterationBench
FailSpy/abliteratorFailSpy/abliteratorActivation cache + hook ablation
Nous llm-abliterationNousResearch/llm-abliterationFrozen fork (~2025-11-27); use jim-plus v1.2 + measure.py --deccp

Datasets (refusal taxonomy)#

DatasetUse
mlabonne/harmful_behaviorsHeretic default bad
mlabonne/harmless_alpacaHeretic default good
SorryBench splitsCategory-specific directions
CoCoNotContextual non-compliance
XSTestOver-refusal measurement
WildGuardMixSafety core prompts
Custom factory .txtWMI/nmap false-refusal pairs

Datasets (direction estimation)#

DatasetURLUse
mlabonne/harmless_alpacaHFHeretic default good prompts
mlabonne/harmful_behaviorsHFHeretic default bad prompts
deccpAUGMXNT/deccpQwen2 PoC + topic dataset — not a peer CLI; use jim-plus measure.py --deccp

Custom: one prompt per line .txt or JSONL — see Heretic config.default.toml [good_prompts] / [bad_prompts].


Agent & eval (this repo)#


Agent & tool-calling: tool picker (Nous, huihui, OBLITERATUS, SuperGemma)#

Use this section when the deploy goal is agents with online/native tool calling — not just uncensored chat. Refusal removal and structured tool output (<|tool_call>, OpenAI tool_calls, JSON tool schemas) are different success criteria.

Gemma 4 native tools require processor.apply_chat_template(..., tools=[...]) and multi-turn tool_responses — see Google function-calling guide.

Prominent stacks compared#

StackEngineTypical targetTool-call preservationBest for
Heretic (handbook default)Optuna + projected + norm-preservingAny HF instruct modelMedium — tune KL + eval gatesAutomatic deploy + factory/CyberGym eval
AbliterixOptuna TPE multi-objective; 150+ presets; MoE/VL/SSMGemma 4, MoE, hybrid, VL familiesMedium-high when preset matches — verify HonestAbliterationBench + handbook JSONLHeretic alternative when arch-specific automation needed
jim-plus llm-abliteration (v1.2)measure.py → YAML sharded_ablate.pyLlama, Qwen, Gemma 3+, MistralMedium-low if conservative YAMLManual path; 20B+ sharded. Nous fork last push ~2025-11-27 — frozen.
Nous Hermes + OBLITERATUSobliteratus obliterate --method advanced|surgical3B–70B instructMedium-high with surgical on reasoning modelsResearch agents, CoT models
OBLITERATUS (standalone)SVD + norm-preserving + KL co-optWide model listMedium-high with advanced; use surgical for R1/CoTWhen entanglement analysis matters
ErisForgeSingle-pass layer-bandGSM8K-sensitive basesMedium — better math subset in arXiv:2512.13655When Heretic KL is poor. Young’s “DECCP” cell is not a peer CLI — use jim-plus measure.py --deccp for those topics.
huihui_aiSumandora/remove-refusals-with-transformersGemma 4 GGUF/OllamaLow — fast uncensor, no tool evalLocal chat uncensoring, not production agents
199-biotechnologies Gemma 4Per-layer directions, weight 1.0, MLXgoogle/gemma-4-31b-itMedium-high on capability suiteApple Silicon, quality-first Gemma
SuperGemma 4 (Jiunsong) v2Fine-tuned agentic base + mild abliterate/uncensor passGemma 4 26B/31BOften high — format re-grounded + permissive baseGemma agents when huihui GGUF breaks tools
Community GGUF downloadUnknown pipelineOllama usersUnpredictablePrototyping only — always eval

Why SuperGemma 31B v2 often keeps tools when huihui builds do not:

  1. Stock Gemma 4 31B-IT is already permissive — less aggressive surgery needed.
  2. Super* lines usually include capability/agent fine-tuning before or after abliteration (re-teaches tool grammar).
  3. Conservative edits (per-layer, weight factor ≤ 1.0) vs huihui's fast layer-banded Sumandora pass.
  4. Users often run SuperGemma through HF AutoProcessor + tools=; huihui builds are often served via Ollama chat without Gemma 4 tool templates (stack failure, not weights).

Theory: refusal, over-refusal, and structured-output directions share subspaces — see beyond-single-direction.md and comparative-abliteration-benchmarks.md.


Decision flowchart — pick abliteration backend#

Workflow diagram from Abliteration & low-VRAM tooling catalogWorkflow diagram from Abliteration & low-VRAM tooling catalog
Diagram rendered from the chapter's source. Open SVG
Inspect diagram notation
flowchart TD
  START([Need abliterated agent with tool calling?]) --> Q1{Authorized lab / factory / CyberGym?}
  Q1 -->|No| STOP([Stop — risks-and-ethics + scope])
  Q1 -->|Yes| Q2{Model family?}

  Q2 -->|Gemma 4 + native HF tools| G1{Download only or self-abliterate?}
  G1 -->|Download| G2[Prefer SuperGemma v2 / 199-biotech / official HF + eval]
  G1 -->|Self-abliterate| G3[199-biotech or jim-plus llm-abliteration projected + normpreserve]
  G2 --> EVAL
  G3 --> EVAL

  Q2 -->|Reasoning / CoT model| R1[OBLITERATUS --method surgical]
  R1 --> EVAL

  Q2 -->|MoE Mixtral / DeepSeek-MoE| M1[OBLITERATUS --method surgical OR moe-hybrid technique]
  M1 --> EVAL

  Q2 -->|General instruct 7B–14B| H1{Heretic KL acceptable on pilot?}
  H1 -->|Yes| H2[Heretic + eval-driven-workflow]
  H1 -->|No| H3[ErisForge or jim-plus YAML + optional measure.py --deccp]
  H2 --> EVAL
  H3 --> EVAL

  Q2 -->|Ollama GGUF only| O1[huihui / community GGUF]
  O1 --> WARN[Assume tool-call broken until proven]
  WARN --> EVAL

  EVAL[Run eval gates — see below] --> Q3{tool_call rate ≥ 95%?}
  Q3 -->|Yes| SHIP[Runtime gate + deploy]
  Q3 -->|No| REPAIR[Jarvis QLoRA tool repair]
  REPAIR --> EVAL2[Re-eval jarvis-safe + factory JSONL]
  EVAL2 --> Q4{Pass?}
  Q4 -->|Yes| SHIP
  Q4 -->|No| MILD[Reduce ablation strength OR switch backend OR use stock + LoRA only]

Decision flowchart — tool calling broke after abliteration#

Workflow diagram from Abliteration & low-VRAM tooling catalogWorkflow diagram from Abliteration & low-VRAM tooling catalog
Diagram rendered from the chapter's source. Open SVG
Inspect diagram notation
flowchart TD
  FAIL([Tool calling degraded]) --> S1{Same stack as stock model?}

  S1 -->|No — different server/template| T1[Fix stack first]
  T1 --> T1a[Gemma 4: apply_chat_template + tools= + tool_responses]
  T1 --> T1b[Verify special tokens in GGUF tokenizer]
  T1 --> T1c[Match OpenAI tools schema to model family]
  T1a --> RETEST
  T1b --> RETEST
  T1c --> RETEST

  S1 -->|Yes — same HF path| S2{Symptom type?}

  S2 -->|Refuses to call tools| R1[Over-refusal subspace — not fully ablated OR wrong prompts]
  R1 --> R1a[Custom factory bad/good .txt in Heretic]
  R1 --> R1b[XSTest eval ≤ 5% over-refusal]
  R1 --> R1c[Jarvis QLoRA on tool JSONL]

  S2 -->|Calls wrong function / bad args| C1[Capability regression]
  C1 --> C1a[GSM8K / MMLU slice]
  C1 --> C1b[Reduce n_directions / weight factor]
  C1 --> C1c[Switch to ErisForge or jim-plus projected+normpreserve]

  S2 -->|Garbled tool syntax| F1[Format subspace damaged]
  F1 --> F1a[Never use aggressive multi-pass on Gemma]
  F1 --> F1b[Re-export from bf16 before Q4 quant]
  F1 --> F1c[Jarvis SFT on tool-call rows]

  S2 -->|Infinite confirm loops| L1[Agent harness issue + residual policy tone]
  L1 --> L1a[CyberGym confirm-loop metric]
  L1 --> L1b[Stronger abliteration OR repair adapter]

  R1a --> RETEST
  R1b --> RETEST
  R1c --> RETEST
  C1a --> RETEST
  C1b --> RETEST
  C1c --> RETEST
  F1a --> RETEST
  F1b --> RETEST
  F1c --> RETEST
  L1a --> RETEST
  L1b --> RETEST

  RETEST{npm run eval:stats + factory JSONL + jarvis-safe} --> PASS{≥ 95% tool_call?}
  PASS -->|Yes| OK([Deploy with hardware-tool-gate.py])
  PASS -->|No| ESC[Escalate: LoRA-only policy OR different base model]

Mandatory eval gates (before any agent deploy)#

bash
npm run eval:stats
npm run ralph:regress          # after doc/script changes from automated dev
npm run eval:jarvis-safe       # post-repair subset
python scripts/cybergym-eval-stub.py --print-flow
GateCorpus / commandPass target
RefusalHarmful set + custom factory badNear 0% explicit refusal on scoped tasks
Over-refusalxstest-overrefusal-sample.jsonl≤ 5% refusal on benign lab prompts
Tool callhardware-factory-prompts.jsonl≥ 95% correct tool_call
Agent repairjarvis-safe-eval.jsonlHigh compliance after optional QLoRA
CapabilityGSM8K / MMLU-5 / HumanEval sliceNo silent regression vs base
CyberGym proxycybergym-subset-sample.jsonlPoC repro + low confirm-loop rate

Full matrices: ../evaluation.md · Workflow: ../../instructions/eval-driven-workflow.md · Stack: ../../instructions/agentic-security-stack.md


Common failure points & fixes#

#FailureLikely causeWhat to do
1Explicit refusal on wmic / nmap / hashcatIncomplete abliteration; factory-specific over-refusal directionCustom [bad_prompts] / [good_prompts] in Heretic; domain-specific abliteration; Jarvis QLoRA
2"I don't have access / cannot execute" without refusal toneOver-refusal subspace (XSTest class) partially intactInclude XSTest-style pairs in good set; eval xstest-overrefusal-sample.jsonl
3**No `<tool_call>` output** (Gemma 4)Wrong chat template; Ollama chat without tools=
4Malformed tool JSON / tokensCollateral damage to format subspace; aggressive ablationReduce layers/directions; --projected + --normpreserve; re-abliterate from bf16; Jarvis SFT
5Wrong function name or argsReasoning/capability regression (not refusal)GSM8K/MMLU gate; try ErisForge or jim-plus --projected --normpreserve; lower Heretic KL target; OBLITERATUS with higher regularization
6Works in HF, breaks in GGUF/OllamaQuantization + conversion dropped tool tokensAbliterate bf16 → convert → verify tokenizer; avoid re-quantizing already-quantized abliterates
7Works on stock, breaks on huihui GGUFFast Sumandora pass + layer band only (e.g. 23–28) + no tool evalSwitch to SuperGemma v2, 199-biotech, or self-abliterate with eval gates; don't assume Ollama tags are agent-safe
8SuperGemma works, your Heretic run doesn'tSuperGemma includes agent fine-tune + mild surgeryAdd Jarvis QLoRA after your abliteration; or start from SuperGemma base for Gemma agents
9Confirm / permission loops in agent harnessResidual policy narration; weak abliteration on agentic promptsStronger projected abliteration; CyberGym confirm-loop metric; eval on cybergym-subset-sample.jsonl
10CoT / thinking model tool accuracy dropRefusal direction entangled with reasoningOBLITERATUS --method surgical; Heretic chain_of_thought_skips; CoT-aware orthogonalization
11MoE model uneven tool behaviorPer-expert refusal directions differmoe-hybrid-abliteration.md; OBLITERATUS surgical
12KL looks good, tools still badKL on harmless prose ≠ tool-call subspaceAlways run factory JSONL + jarvis-safe — KL is necessary not sufficient
13Perplexity spike > 15% after OBLITERATUS/HereticOver-ablationReduce n_directions, increase regularization, fewer layers, basic method
14Refusal persists > 10% after single passRotated residual directions (Ouroboros)OBLITERATUS refinement passes; iterative re-probe; aggressive only on 3B+ after advanced fails
15Runtime executes destructive commandsAbliteration removed safety but no external gateAlways deploy hardware-tool-gate.py ALLOW/CONFIRM/BLOCK — abliteration is not a safety layer

GoalPipeline
Factory firmware agentHeretic (factory bad/good .txt) → eval factory JSONL → optional Jarvis QLoRA → hardware-tool-gate.py
Gemma 4 + native tools199-biotech or conservative jim-plus YAML → HF processor path → factory + xstest eval → Jarvis if < 95%
CyberGym / pentest agentHeretic or OBLITERATUS advanced → cybergym-subset → OpenHands + vLLM
Quick local uncensored chathuihui Ollama tag — do not assume tool calling
Gemma agent, huihui broke toolsSuperGemma 31B v2 or stock IT + Jarvis repair + eval gates
Research / max interpretabilityOBLITERATUS analysis modules → informed or advanced → tournament eval

Install one-liners#

bash
# Full Heretic + quant stack
pip install heretic-llm==1.4.0 bitsandbytes accelerate safetensors

# Manual pipeline + 4-bit measure
pip install torch transformers bitsandbytes peft

# LoRA export after abliteration
pip install peft safetensors

# Research plots
pip install "heretic-llm[research]==1.4.0"

# GGUF toolchain
git clone https://github.com/ggml-org/llama.cpp.git tools/llama.cpp

Tool selection by goal#

Example
Remove refusal from safetensors (automatic)?
  └─ Heretic (heretic-llm) → save HF checkpoint

Remove refusal (manual layer YAML)?
  └─ llm-abliteration v1.2: measure 4bit → sharded_ablate full weights

Only have GGUF for measure?
  └─ abliterate.cpp (WIP) → measurements.pt → llm-abliteration

Deploy Ollama single file?
  └─ merge LoRA OR full abliterated safetensors → convert_hf_to_gguf Q4_K_M

Deploy llama.cpp with swappable policy?
  └─ export-abliteration-lora.py → convert_lora_to_gguf → --lora-scaled

Post-abliteration tool-call repair?
  └─ Jarvis QLoRA (Unsloth/TRL) — SGD, not abliteration

Agent + online tool calling?
  └─ See [Agent & tool-calling: tool picker](#agent--tool-calling-tool-picker-nous-huihui-obliteratus-supergemma) above

Skip surgery entirely?
  └─ Download p-e-w/*-heretic or community GGUF — always eval tool_call before agents

Do not: edit GGUF tensors directly; quantize aligned models expecting uncensoring; deploy huihui/community GGUF as production agents without factory + jarvis eval gates.

Trace this chapter to its source.

The complete Markdown remains the editable source. This page adds navigation, typography, and a visual companion.

docs/tools/abliteration-tooling.md
Inspect the source fingerprintSHA-256 c738ba5e63c381026a8b08841d45edb2db332d238c3f87737ace08c7f2c36edfSource commit 6fba7436a5914cad7c8f0835a14b480a29dac263