The S3 Brief · Issue 14 · AI Security

Your AI Risk Assessment Is Missing Prompt Injection. Here's What to Test.

Prompt injection ranks OWASP #1. Your risk assessment needs specific controls to detect and mitigate it in production.

Your risk assessment lists AI models and data sources. It doesn't test for prompt injection. Neither does anyone else's, and regulators notice.

Prompt injection is ranked number one by OWASP in both 2023 and 2025. It's not a fringe attack. The International AI Safety Report 2026 found that sophisticated attackers bypass best-defended models roughly 50 percent of the time with just 10 attempts. Google Gemini, after applying its best defenses including adversarial fine-tuning, still suffered 53.6 percent attack success rates. Microsoft Copilot, GitHub Copilot, and Cursor IDE all fielded critical CVEs above severity 9.0 this year.

Yet most risk assessments skip it entirely. Here's why that costs you money, and how to fix it.

What makes prompt injection different from traditional input validation is architectural. Large language models cannot reliably distinguish between trusted system instructions and untrusted user data or external content. An attacker embedding malicious instructions in a email, document, or calendar invite can force an agent to ignore its intended task and execute arbitrary commands instead. That agent may have access to your databases, file systems, and APIs. One successful injection becomes a persistent compromise if the agent stores data across sessions.

The Salesloft Drift breach in August 2025 reached more than 700 organizations through OAuth tokens stolen from one AI chat integration. It was not a prompt injection, but it shows how much access a single AI integration holds. The attack surface scales with deployment speed. Thousands of AI agents are deployed weekly without IT or security oversight, creating inventory gaps that compound the threat.

Your compliance frameworks already require you to test for this. ISO/IEC 42001 requires you to assess and treat AI risks (Clause 6.1), and prompt injection belongs in that assessment. GDPR Article 32 requires appropriate technical and organizational measures to secure personal data, including data an AI system touches. SOC 2 auditors now sample input validation and output policy checks as a matter of standard fieldwork. The NIST AI Risk Management Framework treats prompt injection as a top-tier threat alongside OWASP and MITRE ATLAS.

No complete fix exists. The NCSC's December 2025 assessment was explicit: prompt injection may never be fully mitigated the way SQL injection was, because the LLM is an inherently confusable deputy. Defense-in-depth is the only viable posture.

Here's what your risk assessment should test:

Start with inventory. Identify every AI agent deployed in your environment, including its data access scope. Catalog all input vectors, including direct user prompts, documents fed to retrieval-augmented generation systems, emails and calendar invites processed by the agent, and API responses that flow into the model. Evaluate whether your existing controls actually address prompt injection as a threat model. Most do not.

Then test. Run quarterly assessments using a structured framework. Identify AI assets and their data access scope. Catalog attack surfaces for each one. Evaluate existing controls against prompt injection threat scenarios. Quantify residual risk using likelihood multiplied by impact. Prioritize remediation based on risk scores and business criticality.

Implement controls in layers. Input validation alone won't work, but it's a start. Implement strict input validation for any external data flowing into prompts. Encode or remove special characters. Apply output validation and policy-aligned response checks before any AI output triggers downstream action. Require human-in-the-loop for high-risk actions. Every AI decision in a regulated context needs a named human owner who can explain why the AI acted. Document who approved the model, who monitors it, and who has authority to shut it down.

Your access control needs teeth. Agents granted 10 times more access than their workflows actually need are 10 times more dangerous when a prompt injection succeeds. Restrict what each agent can do to the minimum required function. Audit access logs. Enforce authentication and approval for sensitive operations.

Log everything. Model version, timestamp, calling user or service, prompt hash or redacted prompt text, full output, and token counts. If a successful injection compromises your system, you need to reconstruct exactly what the AI was told and what it did. Per-call logging is the single most-requested artifact in AI-inclusive SOC 2 audits. Field reports on 2026 audits list it alongside immutable model registries and rollback proof as the top items examiners sample.

Monitor for drift. A model's behavior changes after updates or data changes. Continuous evaluation should compare live output with approved baseline results. Reviews should cover quality, safety, fairness, reliability, and operational alignment. If an injection attack is degrading the quality of outputs, monitoring catches it before downstream systems act on bad data.

Your risk assessment is due. Add prompt injection. Start with the AI assets handling your highest-risk data. Assign a lead with AI security expertise. Include security professionals for threat modeling, data scientists who understand model behavior, and domain experts from the business units where agents run. Set a cadence. Continuous monitoring of access and alerts. Monthly review of performance and incident trends. Quarterly permission reviews and vendor assessments.

This is not optional. Your auditor will ask for it. Your buyers will require it. Your incident response plan will fail if you have not already tested the controls that should have stopped the attack.

Start building your prompt injection risk assessment now. Grab the free AI Security Checklist for a practical list of controls for personal and company AI use: https://s3advisorygroup.com/resources/ai-security-checklist

Sources

More past issues

All issues