Model Modification

Every technique so far works around safety training. Encoding hides content from filters. Personas create alternative identities. Jailbreaks override behavior. But the safety training is still there, pushing back.

What if you just removed it?

For open-weight models, you can. A technique called abliteration identifies the direction in the model’s internal representation that mediates refusal and removes it. The result: a model that lost the ability to refuse.

Think about why this matters for red teaming. Every other technique on this site is about crafting inputs that bypass safety. Model modification removes safety entirely. What does that change about how you work?

What removal changes

Two things change fundamentally.

First, the workflow: you no longer spend time crafting prompts that bypass safety. You ask directly and get direct answers. This makes the modified model useful as a RED TEAM TOOL, not as a target. It generates attack prompts, drafts payloads, and brainstorms approaches without resistance.

Second, the testing value: a modified model tells you nothing about the TARGET’S defenses. Testing your attacks against a model with no safety training is like testing lockpicks on a door with no lock. The modified model is your workshop tool. The target model is what you test against.

This distinction matters because beginners sometimes confuse “I made the model do the thing” (model modification) with “I bypassed the model’s defenses” (red teaming). They’re different activities with different value.

When to use model modification

Makes sense when:

You need unrestricted generation for attack prompt creation
You’re building a red team assistant that follows any instruction
You want to compare behavior with and without safety training
You’re researching the mechanics of safety training itself

Doesn’t help when:

Testing against closed-source APIs (GPT-4, Claude, Gemini). You can’t modify what you can’t access.
You need to test the target’s actual safety training. The target’s defenses are what you’re evaluating.
You lack compute for local model inference. Abliterated models need to run locally.

The workflow:

Abliterate an open-weight model (or use a pre-built uncensored model)
Use it as your red team assistant to draft attack prompts
Test those prompts against the actual target model
Use the assistant to iterate on failed attempts

Separate the tool from the target.

Assessing whether model modification fits your engagement

Do you need unrestricted generation? If you’re manually crafting 5-10 prompts, model modification is overhead. If you need to generate hundreds of attack variants systematically, an unrestricted assistant eliminates the friction of fighting safety training on your own tools.

What model quality do you need? Abliterated models are typically 3B-70B parameter open-weight models. They’re capable for drafting attack prompts, brainstorming approaches, and iterating on techniques. They’re not as capable as frontier models (GPT-4, Claude) for complex reasoning. If your attack requires sophisticated multi-step reasoning, you might need a frontier model’s capability even if you have to work around its safety training.

Do you have the compute? Running a 70B model locally requires significant GPU memory (40GB+ for 4-bit quantization). Smaller models (3B-8B) run on consumer hardware but are less capable. Pre-built uncensored models on HuggingFace are the fastest path if you don’t want to run abliteration yourself.

Does your engagement scope allow it? Some red team engagements restrict the tools you can use. Model modification creates unrestricted AI assistants. Make sure your scope of work permits this.

Abliteration

How it works

Contrast prompts: Run harmful and harmless prompts through the model. Collect the internal representations (activations in the residual stream) for both sets.
Calculate refusal direction: Identify the direction in the residual stream that differs between harmful-refused and harmless-complied responses. This is the “refusal direction.”
Ablate the direction: Remove that direction from the model weights by projecting it out.
Result: The model loses the ability to refuse. All other capabilities remain.

The key finding

In the models tested, refusal appears to be mediated by a single direction in the residual stream. This is a remarkably concentrated mechanism. Removing it eliminates refusal. Adding it to a non-refusing model induces unnecessary refusals on harmless requests. The concentration of safety behavior in a single direction (rather than distributed across the network) is what makes abliteration possible. More recent safety training approaches may distribute refusal behavior more broadly, making abliteration less clean.

Tools

Tool	Description
NousResearch/llm-abliteration	Make abliterated models with transformers
FailSpy/abliterator	Original implementation
p-e-w/heretic	Fully automatic censorship removal

Usage

# Basic abliteration
python abliterate.py --model meta-llama/Llama-3.2-3B-Instruct

# With projection (recommended for quality)
python abliterate.py --model meta-llama/Llama-3.2-3B-Instruct --projected

# Large models with limited VRAM
python abliterate.py --model meta-llama/Llama-3.3-70B-Instruct --load-in-4bit

Tested models

Llama-3.2, Qwen2.5-Coder, Ministral-8b, Mistral-7B-Instruct, Gemma-3-27b-it, Mistral-Nemo-Instruct.

Performance trade-off

Abliteration can degrade general model performance. Removing the refusal direction sometimes damages adjacent capabilities. The technique was improved using DPO (Direct Preference Optimization) to create models like NeuralDaredevil-8B: fully uncensored but with restored quality.

How much degradation you can tolerate depends on use case. For generating attack prompt drafts, moderate quality loss is acceptable. The assistant doesn’t need to be brilliant. It needs to be unrestricted.

Pre-built uncensored models

If you don’t want to run abliteration yourself, pre-built uncensored models exist on HuggingFace. These models have already had safety training removed or were trained without it.

Use cases:

Red team assistant for generating attack prompts
Baseline comparison (model behavior without safety training)
Testing whether a defense depends on model refusal vs. external filtering

Limitation: Uncensored models are smaller and less capable than frontier models. An uncensored 8B model can generate attack prompts, but it won’t match GPT-4 or Claude for quality.

Comparing approaches: abliteration vs. jailbreaking vs. pre-built models

Approach	What you get	Setup effort	Capability level	Persistence
Abliteration	Customized unrestricted model from your choice of base	Medium (run script, requires GPU)	Matches base model quality minus degradation	Permanent
Jailbreak	Temporarily bypassed model	Low (craft prompt)	Full frontier model capability	Per-session, needs re-establishment
Pre-built uncensored	Ready-to-use unrestricted model	Low (download from HuggingFace)	Limited to available models (typically 3B-70B)	Permanent

When to use each:

Use abliteration when you want a specific model (one with particular capabilities or training) without safety restrictions. You control the base model choice.

Use jailbreaking when you need frontier model capability (GPT-4, Claude quality) and are willing to re-establish the bypass each session. The target model’s full capability is available.

Use pre-built uncensored when you need quick access to an unrestricted assistant without setup. Fastest path to a working red team tool.

Use the combination when you want maximum efficiency: pre-built uncensored model as your attack prompt generator, jailbreaking techniques tested against the actual target.

Your turn

You have a red team engagement against a company’s customer service chatbot (powered by GPT-4). You need to test 50+ attack techniques against it. For each technique, you need to generate multiple prompt variants.

You have two options:

Craft each attack prompt manually, fighting GPT-4’s safety training on your own tools
Set up an abliterated Llama-3.2-8B as your red team assistant to generate attack variants

Which approach produces better results? Think about the trade-off between the assistant’s capability (8B vs. GPT-4) and the friction of manual crafting.

Practitioner analysis

Option 2 (abliterated assistant) is better for this engagement. The math: 50 techniques × multiple variants each = 200+ prompts. Manually crafting each one against GPT-4’s safety training is slow. The assistant generates variants in seconds.

The quality trade-off is manageable. You don’t need GPT-4-quality attack prompts. You need structurally varied prompts that test different approaches. An 8B model can produce “persona with academic framing” and “encoding with fiction context” variations efficiently. The prompts don’t need to be sophisticated. They need to be numerous and diverse.

The workflow: (1) Use the abliterated model to generate 5-10 variants per technique. (2) Quick manual review to discard obviously weak variants. (3) Run all variants against the actual target. (4) Analyze results to identify which techniques and variants work. (5) Use the abliterated model to iterate on successful approaches.

The key insight: separating the “attack generation tool” from the “target under test” is how professional red teams work.

References

mlabonne. “Abliteration: uncensoring LLMs.” HuggingFace Blog.
NousResearch. llm-abliteration implementation.
FailSpy. Original abliterator.

Previous: Persistence