# Postmortem: {{incident_title}}

> Template for incidents involving an AI feature: wrong or harmful outputs, prompt injection, data exposure through a model, cost spikes, provider outages, or an agent taking an unintended action. Copy this file per incident, fill the `{{...}}` fields, and delete these block-quote notes. Write blameless: describe what the system and process allowed, not who to blame.

| Field | Value |
|---|---|
| Incident ID | {{incident_id}} |
| Status | Draft / In review / Final |
| Severity | {{severity}} (use your normal incident scale) |
| Date detected | {{detected_at}} (with timezone) |
| Date resolved | {{resolved_at}} |
| Author(s) | {{authors}} |
| Feature / system | {{feature_name}} |
| Model and version at the time | {{model_id}} |
| Prompt version at the time | {{prompt_version}} (commit SHA or tag) |

## 1. Summary

> Three to five sentences a non-engineer can read: what happened, who was affected, how long, and what we changed.

## 2. Impact

- **Users or customers affected:** {{count_or_estimate_and_how_measured}}
- **What they saw or experienced:**
- **Data exposure:** None / Suspected / Confirmed. If suspected or confirmed, which data classes and whether notification obligations apply (involve legal or privacy).
- **Actions taken by the system:** (for agents and tool use: emails sent, records changed, payments, tickets closed)
- **Cost impact:** (tokens or spend above normal, if relevant)
- **Duration:** first bad output at ___, detected at ___, mitigated at ___, resolved at ___.

## 3. Timeline

> Use exact times. Include when the triggering change shipped, even if it was days earlier.

| Time | Event |
|---|---|
| | |

## 4. Failure classification

Tick every category that applies.

- [ ] **Hallucination / fabrication**: output stated something false as fact.
- [ ] **Instruction failure**: model ignored a rule in the prompt.
- [ ] **Prompt injection**: instructions inside user, document, web or tool content were followed.
- [ ] **Data leakage**: model output revealed system prompt, other users' data, or secrets.
- [ ] **Unsafe or off-brand content**: toxic, discriminatory or policy-violating output.
- [ ] **Unintended action**: an agent or tool call did something it should not have.
- [ ] **Format / parsing failure**: downstream code could not handle the output.
- [ ] **Model or provider change**: a model version, default or deprecation changed behavior.
- [ ] **Retrieval failure**: wrong, stale or missing context was retrieved.
- [ ] **Capacity / cost**: rate limits, timeouts, runaway loops or token spend.
- [ ] **Other:** ___

## 5. Root cause analysis

> Ask "why" until you reach something in the system or process you can change. Typical layers to examine: the input that triggered it, the prompt, the model, retrieval or tools, output validation, guardrails, monitoring, and the release process.

**Triggering input(s):** (redact personal data; link the stored copy if retention rules allow)

**Why the model produced this output:**

**Why our guardrails or validation did not stop it:**

**Why we did not detect it sooner:**

**Why it was not caught before release:** (was there an eval case that should have covered this?)

## 6. What went well

-

## 7. Where we got lucky

-

## 8. Action items

> Each item has one owner and a date. Prefer changes that prevent a whole class of failure over a fix for this one input.

| # | Action | Type | Owner | Due | Tracking link |
|---|---|---|---|---|---|
| 1 | Add the triggering input(s) to the eval set as regression cases | Detect | | | |
| 2 | | Prevent | | | |
| 3 | | Mitigate | | | |

Types: **Prevent** (stop it happening), **Detect** (notice sooner), **Mitigate** (reduce impact next time), **Process** (release, review or on-call change).

## 9. Eval and monitoring changes

- New eval cases added (IDs or file path):
- New or changed alert:
- Kill switch or fallback exercised? Did it work?

## 10. Follow-up review

Date to confirm action items are done and the regression cases pass: {{follow_up_date}}