Security teams haven't suddenly forgotten how to protect endpoints. What's changed is that the attack surface grew sideways — into AI assistants, agent loops, and inference endpoints — and the threat models built for traditional web apps don't quite cover it.
The new vectors worth understanding
Three risks that didn't exist (or didn't matter) two years ago:
- Prompt injection. A user-supplied prompt instructs the model to ignore its system instructions and exfiltrate data, send unintended emails, or call unauthorised tools. If your AI agent has access to APIs, the surface area is significant.
- Model exfiltration via inference. Attackers query a customer-facing model in patterns designed to reconstruct training data or proprietary prompts. Especially nasty for fine-tuned models trained on internal documents.
- Deepfake social engineering. Voice-cloned CFO calls, AI-generated phishing tailored to your internal vocabulary. The kit is cheap, the success rate is up.
What the playbook looks like now
In customer security reviews, we keep coming back to the same checklist:
- Treat the model as an untrusted user. Its outputs flow into systems that should validate, log, and rate-limit them the same as any user input.
- Separate read from write capability. AI agents that can read a customer record are different from agents that can update one. Permissions per tool, with audit logs on every privileged call.
- Input sanitisation extends to prompts. User content concatenated into prompts is the new SQL string concatenation. Same hygiene, same consequences.
- Egress controls on inference output. What can the model output? Block PII patterns, secrets, system prompts. Don't let the model return what it shouldn't have access to.
The defenders' upside
It's not all asymmetric. The same models that help attackers help defenders detect anomalies faster, triage alerts at scale, and write better detection rules. SOC teams that integrate ML into their workflow have seen real reduction in mean-time-to-detect.
The era isn't unwinnable. It's just one where the threat models need to be refreshed, and the team has to keep the security conversation in every product review — including the AI features.