The S3 Brief · Issue 4 · AI Governance

AI Incident Reporting Is Now Mandatory. Your Audit Trail Isn't Ready.

EU AI Act, Colorado, Illinois, and federal rules now require mandatory AI incident reporting. Build your logging and documentation system today.

Your organization runs AI in production. An AI model flags a loan applicant incorrectly. An LLM leaks customer data through a prompt injection. A foundation model output causes a hiring decision error. You have hours to report it. Your audit trail doesn't exist.

AI incident reporting is becoming law in the EU and in several U.S. states.

What triggers mandatory reporting.

EU AI Act Article 73 requires providers of high-risk AI systems to report serious incidents to national authorities. After the AI Omnibus, most high-risk duties apply from December 2, 2027. High-risk systems include AI used in employment, credit decisions, education, law enforcement, and critical infrastructure. The definition of a serious incident spans safety breaches, unexpected model behavior, data breaches, and security vulnerabilities.

In the U.S., California's frontier AI law took effect January 1, 2026, with a 15-day reporting deadline. New York's RAISE Act and Illinois's AI Safety Measures Act take effect January 1, 2027, with 72-hour reporting for critical safety incidents. All three require reports within 24 hours when there is imminent risk of death or serious injury. Colorado's separate AI Act covers high-risk automated decisions in areas like employment, lending, housing, and healthcare.

At the federal level, CISA proposed mandatory cyber incident reporting under CIRCIA, though the final rule was delayed to May 2026. The proposed framework requires covered entities to report covered cyber incidents within 72 hours. DOD separately proposed an AI incident reporting program for military AI systems. Congress signaled support through the proposed AI Incident Reporting Act.

What most organizations get wrong.

Incident reporting requires an audit trail. You need continuous logs of model inputs and outputs, decision chains, model artifact integrity, and training pipeline events. You need documentation of when the incident occurred, what the AI system did, who was affected, and what action was taken.

Most organizations logging AI systems today log only final outputs, not inputs or decision rationale. Few instrument their training pipelines. Fewer still have baseline models of normal AI behavior to detect deviation. When an incident happens, you have no timeline to work backward from.

NIST identified this gap in March 2026. Their report on monitoring deployed AI systems found immature information-sharing ecosystems for incident data, a lack of trusted guidelines for AI monitoring methodologies, and the absence of standardized approaches to post-deployment oversight. The tooling that works for cyber incidents (endpoint detection, security information and event management, network traffic analysis) does not yet cover AI-specific compromise.

What you need to build now.

Start with an AI inventory. Catalog every model, tool, and data source in production. Classify each system by risk tier: Is it high-risk under EU AI Act definitions? Does it make consequential decisions about people? Tag it.

Define what constitutes a serious incident for your business. A data leak from a model. A bias spike in credit decisions. An unauthorized model change. A prompt injection that bypasses your instructions. Write these down. Make the list specific and narrow. Overly broad definitions create noise. Overly narrow ones hide risk.

Build logging discipline. Instrument your AI systems to capture inputs, outputs, model artifacts, and decision chains at a temporal resolution you can later query. You do not need to log every token in a foundation model call, but you need enough telemetry to reconstruct what happened. Store these logs separately from application logs. Treat them as forensic evidence.

Define roles and escalation. Who detects an AI incident? Who classifies it as serious? Who notifies legal, compliance, or external authorities? Who documents the remediation? A RACI matrix clarifies this. Ambiguous accountability kills response speed.

Test your response. Run a tabletop exercise. Assume an AI model training data leak is discovered. Walk through your detection, classification, documentation, and reporting workflow. Do you have the logs you need? Is your incident classification clear? Does your escalation path work? Gaps found in an exercise stay found until you fix them.

Why this matters to you.

California's rules already apply. New York and Illinois start January 1, 2027. The EU's high-risk duties follow on December 2, 2027. If you operate AI in production, or sell AI systems to enterprises, build your incident reporting process before those dates.

Non-compliance costs are material. EU AI Act penalties for high-risk obligations reach 15 million euros or 3% of global annual turnover. U.S. state regulators can impose civil penalties. Beyond the fine, a public incident report erodes trust with customers, regulators, and boards.

Compliance is also an early indicator of AI governance maturity. Organizations that can audit their own AI incidents defensibly signal to customers and vendors that they take AI risk seriously. That becomes a competitive advantage in regulated markets.

Start your incident response framework this week. Identify your AI systems. Log your inputs and outputs. Name your incident triggers. Run a tabletop test. The enforcement clock is real.

Sources

More past issues

All issues