Resources
References
Tools, communities, research, and standards for AI red teaming. What this site draws from and what practitioners use in the field.
Our Tools
Custom tools built for this project. Technique composition and specialized attack scripts.
RedTeam Composer
CLI tool for browsing and composing adversarial techniques. 95 techniques across 17 tactic categories with application strategies, combination recommendations, and research citations. Generates meta-prompts for adversarial prompt composition and system jailbreak construction.
Python · MIT License · Powers the Compose section on this site
Attack Scripts
Specialized tools for testing AI system boundaries beyond prompt-level attacks. Inject hidden payloads into documents for RAG poisoning (PDF, HTML, DOCX, Markdown). Embed instructions in audio files for multimodal attacks. Spin up rogue MCP servers for agent poisoning. Plant poisoned configs and model artifacts for supply chain testing.
Python · MIT License · 4 attack tools + HTTP callback server
Tools and Frameworks
Red teaming tools and frameworks for testing AI systems. From automated scanning to interactive jailbreak crafting.
| Tool | What it does |
|---|---|
| Promptfoo | Open source. Tests 50+ vulnerability types. Generates attack probes, maps to OWASP/NIST/EU AI Act. CI/CD integration. 300K+ users. |
| InjectPrompt Companion | LLM-powered jailbreak crafting. Reads refusal signals and adapts. Generates and iterates on adversarial prompts with real-time feedback. Free tier available. |
| PyRIT — Microsoft | Python Risk Identification Toolkit. Automated multi-turn, XPIA, and persuasion orchestrators. Enterprise-grade. |
| PAIR | Prompt Automatic Iterative Refinement. Automated jailbreak generation using an attacker LLM. |
| AutoDAN | Automated stealthy jailbreak generation. Genetic algorithm approach to evolving adversarial prompts. |
| Garak | LLM vulnerability scanner. Named after the Star Trek character. Probes, detectors, and generators for systematic testing. |
| DeepEval | Open-source LLM red teaming for RAG pipelines, chatbots, and agents. Built-in vulnerability scanners and evaluation metrics. |
| Inspect — UK AISI | Red teaming framework from the UK AI Safety Institute. Evaluation tasks, solvers, and scorers for systematic AI safety testing. |
| EasyJailbreak | 12 attack method implementations in one framework. Quick prototyping of GCG, AutoDAN, PAIR, and other automated approaches. |
Community
Where practitioners learn, compete, and share. The red teaming landscape is community-driven.
Communities
| Community | What it is |
|---|---|
| BASI Discord | The main community for AI jailbreaking and red teaming. 52K+ members. Led by Pliny. Active research, prompt sharing, and technique development. |
| r/ChatGPTJailbreak | Reddit community for jailbreak research and sharing. Active discussion of new techniques and model updates. |
| chatgptjailbreak.tech | Jailbreak forum. Home of ENI-Tutor and other community-developed architectures. |
| InjectPrompt | Newsletter and community by David Willis-Owen. Red teaming techniques, tool releases, and industry analysis. |
Competitions and Challenges
| Platform | What it is |
|---|---|
| Gray Swan Arena | Competitive red teaming platform. Test frontier models, compete for prizes, build career in AI safety. Covers prompt injection, tool use, computer use, and coding agents. |
| Gandalf — Lakera | AI hacking CTF. Progressive difficulty levels. Agent Breaker mode tests agentic AI exploitation. Free to play. |
| HackAPrompt | Prompt injection competition. Structured levels testing specific injection techniques. |
| LLMail-Inject — Microsoft | Email injection CTF. Test indirect prompt injection in realistic email assistant scenarios. |
| IEEE SaTML 2024 LLM CTF | Defense competition where 42 of 44 defenses were bypassed. Demonstrated that static defenses fail against adaptive attackers. |
Educational Resources
| Resource | What it is |
|---|---|
| Adversarial Attacks on LLMs — Lil'Log | Best single overview of the attack landscape. Comprehensive, well-sourced. |
| ACL 2024 Tutorial: LLM Vulnerabilities | Academic tutorial covering vulnerability categories with research citations. |
| AI Red Teaming Resource Guide — Gray Swan | Curated list of tools, papers, and communities for AI red teaming practitioners. |
| Embrace The Red — Johann Rehberger | AI security research blog. Pioneered ASCII smuggling, Claude data exfiltration, and indirect prompt injection research. |
Standards and Frameworks
Industry standards that define how AI security is evaluated and governed.
| Standard | Organization | Focus |
|---|---|---|
| LLM Top 10 | OWASP | LLM01: Prompt Injection through LLM10 |
| Top 10 for Agentic Applications | OWASP | Agentic AI security (2026) |
| ATLAS | MITRE | Adversarial Threat Landscape for AI Systems |
| AI RMF 1.0 | NIST | Risk management lifecycle |
| Agentic AI Red Teaming Guide | CSA | Cloud Security Alliance guidance for testing agentic systems |
| Secure-by-Design Principles | CoSAI | Coalition for Secure AI design principles |
Benchmarks and Datasets
Standardized evaluation sets for measuring attack effectiveness and defense robustness.
| Benchmark | Size | Source |
|---|---|---|
| HarmBench | 400+ behaviors, 18 methods, 33 LLMs | ICML 2024 |
| JailbreakBench | 100 behaviors | NeurIPS 2024 |
| WildJailbreak | 262K pairs | NeurIPS 2024 |
| StrongREJECT | 313 calibrated prompts | Automated rubric-based jailbreak evaluation |
| SafetyPrompts.com | 144 datasets | Catalogue of safety evaluation datasets |
| AILuminate v0.5 | 1,200 prompts | MLCommons AI safety benchmark |
| hh-rlhf | Red team subset | Anthropic human feedback dataset |
| ALERT | 44,800 prompts | Babelscape red team prompt dataset |
| JailbreakHub | 15,140 prompts | WalledAI jailbreak prompt collection |
Jailbreak Repositories
Collections of jailbreak prompts, patterns, and system prompt leaks. Research material, not copy-paste solutions.
| Repository | What it is |
|---|---|
| L1B3RT4S | Cross-platform universal patterns. 24+ jailbreaks, 14 AI orgs. GODMODE protocol. By Pliny. |
| CL4R1T4S | Leaked system prompts. ChatGPT, Claude, Gemini, Grok, Cursor. By Pliny. |
| verazuo/jailbreak_llms | Largest academic dataset. 15,140 prompts, 1,405 jailbreaks from 131 communities. |
| Spiritual-Spell-Red-Teaming | Claude-focused. ENI-Tutor limerence architecture, Push Prompt Basics. |
| ChatGPT_DAN | DAN evolution from v6 through v13. Historical record of jailbreak iteration. |
| TheBigPromptLibrary | Collection of leaked and discovered system prompts from production AI systems. |
| UltraBr3aks | Jailbreak collection with The Jailbreak Index for centralized vulnerability tracking. |
| Awesome_GPT_Super_Prompting | Curated awesome-list: jailbreaks, system prompt leaks, prompt injection techniques. |
| AwesomeLLMJailBreakPapers | Curated academic paper tracker. Organized by attack category with publication venue. |
Abliteration Tools
| Tool | What it does |
|---|---|
| NousResearch/llm-abliteration | Make abliterated models with transformers |
| FailSpy/abliterator | Original abliteration implementation |
| p-e-w/heretic | Fully automatic censorship removal |
Academic Papers
Research this site draws from. Organized by topic.
Foundational
| Paper | Venue | Key Contribution |
|---|---|---|
| Jailbroken: How Does LLM Safety Training Fail? — Wei et al. | NeurIPS 2023 | Failure mode taxonomy, competing objectives |
| Do Anything Now — Shen et al. | CCS 2024 | 15,140 prompts from 131 communities analyzed |
| Jailbreaking via Prompt Engineering — Liu et al. | 2023 | 97.44% pretending prevalence, 3 strategies, 10 patterns |
| Tree of Attacks (TAP) — Mehrotra et al. | ICLR 2024 | Automated jailbreaking in under 20 queries via tree search and pruning |
Taxonomy and Analysis
| Paper | Key Finding |
|---|---|
| Don't Listen To Me | 5 categories, 10 patterns, length-success correlation |
| Red Teaming the Mind | ASR by category: roleplay 89.6%, logic traps 81.4%, encoding 76.2% |
| h4rm3l: Composable Jailbreak Attacks | Compositional attack grammar with parameterized primitives |
| GPTFuzzer | Mutation operators (generate, crossover, expand, shorten, rephrase). 90%+ ASR on ChatGPT and Llama-2 |
| Content Concretization | 7% initial success → 62% after 3 iterations. Validates systematic mutation over template reuse |
Psychological Manipulation
| Paper | Key Finding |
|---|---|
| Breaking Minds, Breaking Systems (HPM) | 88.1% ASR via gaslighting, emotional blackmail |
| How Johnny Can Persuade LLMs to Jailbreak Them | 92% ASR, 40 persuasion techniques (PAP) |
Multi-Turn and Escalation
| Paper | Key Finding |
|---|---|
| Crescendo Multi-Turn LLM Jailbreak — Russinovich et al. | 98% ASR on GPT-4, under 5 turns |
| Many-shot Jailbreaking — Anthropic | Power-law scaling with context examples |
| The Attacker Moves Second — Nasr, Carlini et al. | Adaptive attacks, conditional logic injection, defense taxonomy |
Universal Bypasses and Infrastructure
| Paper | Key Finding |
|---|---|
| Policy Puppetry — HiddenLayer | Universal across all major LLMs via config format |
| Output Constraints as Attack Surface | Structured output APIs create vulnerabilities |
| MCPTox | Tool poisoning on 45 live MCP servers, 72.8% ASR |
| Ignore Previous Prompt — Perez, Ribeiro | Foundational prompt injection research |
| ASCII Smuggling — Rehberger | Unicode Tags (E0000) invisible to humans but parsed by tokenizers. Bypasses safety filters |
| Prompt Injection 2.0 | Hybrid AI + web attacks combining prompt injection with XSS, CSRF, and SSRF |
Attack Success Rate Reference
Empirical success rates from published research. Benchmark numbers, not guarantees. Actual rates vary by model, deployment, and prompt quality.
| Technique | ASR | Source |
|---|---|---|
| Persuasion-based (PAP) | 92% | Persuasive Jailbreaker |
| Compositional (h4rm3l) | 91% | h4rm3l |
| Fuzzing (GPTFuzzer) | 90%+ | GPTFuzzer |
| Roleplay / Persona | 89.6% | Red Teaming the Mind |
| Psychological Manipulation | 88.1% | HPM |
| Context Compliance Attack | 82-100% | Microsoft MSRC |
| Logic Traps | 81.4% | Red Teaming the Mind |
| Encoding | 76.2% | Red Teaming the Mind |
| MCP Tool Poisoning | 72.8% | MCPTox |
| Bad Likert Judge | 71.6% | Unit 42 |
| Multi-turn (human) | 70%+ | Scale AI |
| Deceptive Delight | 64.6% | Unit 42 |
| Content Concretization (3 iterations) | 62% | Content Concretization |
| Policy Puppetry | Universal | HiddenLayer |