Model Extraction Attacks

Try it out loud first. Then check yourself:

1. Attacker queries a model API at scale and trains a surrogate on the outputs, cloning capability (distillation).
2. Returned logprobs/confidence scores leak far more signal than top-1 text and speed up extraction.
3. Stolen surrogate enables IP theft and offline crafting of adversarial or jailbreak inputs that transfer back.
4. Defenses: per-key rate limits and quotas, query-pattern anomaly detection, limit or round logprobs, ToS enforcement.
5. Output watermarking or canary responses can give evidence (not proof) that a model was trained on your outputs.

If the interviewer pushes back

• How would you distinguish a heavy legitimate customer from an extraction campaign using only API telemetry?