Case 48BF5C · AI security · L4 Advanced
Red Teaming LLM Applications
- 01 Context
- 02 Mechanism
- 03 Lessons
What a strong answer covers
Try it out loud first. Then check yourself:
- Goal: find security and safety failures before attackers do; scope the model and the app around it: tools, RAG, auth.
- Threat-led plan: map assets and harms, then pick attack classes from OWASP LLM Top 10 and MITRE ATLAS.
- Mix automation and people: PyRIT, garak or promptfoo scale probes; humans find multi-turn and business-logic breaks.
- Outputs are non-deterministic: measure attack success rate per category over many attempts; one clean pass proves little.
- Turn findings into fixes and a regression suite rerun on every model, prompt or tool change; rate severity like a pentest.
If the interviewer pushes back
- The provider swaps the underlying model version. What do you rerun, and what result blocks the release?
- How do you red team an agent safely when its tools can send real emails or move real data?