AI Incident Disclosure / Liability / Risk Transfer Playbook
AI incident disclosure / liability / risk transfer 不是基础 incident response, 而是把 AI 事故从“技术故障”升级为事实保全、客户影响界定、披露支持、责任边界、保险权利保留、供应商追偿和 board-ready 证据架构。
AI Incident Disclosure / Liability / Insurance / Risk Transfer Architecture Playbook
配对阅读:本手册的原理/架构解读版是
docs/ai-foundations/papers/126-ai-incident-disclosure-liability-risk-transfer-architecture.md。先读 paper 建立机制与取舍,再用本手册落地为模板、RACI 与门禁,两者不需要重复精读。
核心导读: AI incident disclosure / liability / risk transfer 不是基础 incident response, 而是把 AI 事故从“技术故障”升级为事实保全、客户影响界定、披露支持、责任边界、保险权利保留、供应商追偿和 board-ready 证据架构。
学习主线: 从 incident taxonomy、materiality triage、notification workflow、liability boundary、vendor/insurance mapping、loss quantification、evidence pack、tabletop exercise 到 operating model, 建立一条可复盘、可审查、可追偿的 AI incident 决策链。
系统边界: 本文不自动判断是否披露、是否承保、是否赔付或是否通知客户;它定义支持 Legal、Risk、Compliance、Security、Insurance、Vendor Management、Board 和审计流程的事实、证据、控制和治理结构。
核心判断:
AI incident management is not complete when the model is fixed. It is complete when facts are preserved, affected customers are understood, disclosure decisions are supported, liability boundaries are mapped, insurance rights are preserved, vendor recovery is evaluated, and evidence is ready for counsel, regulators, auditors and executives.
0. Disclaimer
本文是系统学习和架构训练材料, 不构成法律意见、证券披露意见、保险覆盖意见、监管意见、合规结论、审计结论、理赔建议或客户通知建议。
正式项目必须由 Legal、Securities Counsel、Disclosure Committee、Compliance、Privacy、Cyber、Operational Risk、Model Risk、Third-Party Risk、Insurance/Risk Management、Finance、Communications、Business Owner、Internal Audit、Insurance Broker、Coverage Counsel 和必要的外部顾问共同判断。
适用性取决于 entity type、public-company status、jurisdiction、product/customer segment、incident facts、customer impact、cyber/privacy impact、regulated workflow impact、vendor contract、insurance policy terms、notice timing、exclusions、sublimits、retentions、endorsements、board/counsel/regulator/insurer views。
本文避免给出“必须披露”“一定覆盖”“必然赔付”之类结论。它提供的是 decision support architecture。
1. Executive Framing
金融零售 AI incident 会跨越多个传统边界:
- Technology incident。
- Cyber/privacy incident。
- Customer harm incident。
- Misleading claim / marketing content incident。
- Conduct risk / fair lending / suitability incident。
- Third-party vendor incident。
- Board governance incident。
- Insurance notice and claim preservation event。
- Legal hold and evidence preservation event。
传统 IT incident 关注 restore service。AI incident 还要关注:
What did the AI say or do?
Who saw it?
Which business decision was influenced?
Which customer population may be harmed?
Was a regulated claim or disclosure affected?
Which facts are confirmed, unknown or disputed?
Which vendor or internal control boundary failed?
Which notification, disclosure or insurance notice pathways may be implicated?
What evidence is required before anyone can responsibly conclude anything?
成熟的 AI incident operating model 不应替代律师、证券披露委员会、保险 broker 或理赔团队。系统设计要把以下能力前置:
- 把 incident facts 结构化。
- 把 AI runtime evidence 保全。
- 把客户影响和业务影响量化。
- 把 vendor and insurance boundary 映射清楚。
- 把 board / regulator / customer / insurer 所需 evidence 提前准备。
- 把补救、恢复、risk transfer 和 CAPA 放进同一个 operating model。
Executive principle:
The institution cannot transfer AI risk it cannot evidence,
cannot disclose risk it cannot measure,
and cannot defend AI conduct it cannot replay.
2. Source Anchors
| Anchor | Official link | 本文使用方式 |
|---|---|---|
| SEC Cybersecurity Disclosure Rules final rule page | https://www.sec.gov/news/press-release/2023-139 | 用 registrant cybersecurity incident disclosure、materiality、risk management、strategy and governance disclosure 的语言训练 public-company disclosure support;AI incident 是否进入该路径取决于事实 |
| FTC AI claims guidance | https://www.ftc.gov/business-guidance/blog/2023/02/keep-your-ai-claims-check | 用 AI claim substantiation、deceptive overclaim、risk disclosure 和 marketing truthfulness 组织 AI claims incident |
| NAIC Model Bulletin on use of AI systems by insurers | https://content.naic.org/sites/default/files/inline-files/Final%20Model%20Bulletin%20-%20Adopted%20by%20the%20Executive%20Committee%20and%20Plenary_12.4.23.pdf | 用 insurer AI governance、risk management、third-party AI system、consumer outcome 和 regulatory oversight 语言组织 insurance/underwriting scenarios |
| NIST AI RMF | https://www.nist.gov/itl/ai-risk-management-framework | 用 Govern / Map / Measure / Manage 组织 incident taxonomy、risk measurement、control selection、monitoring 和 improvement |
| FFIEC Management booklet | https://ithandbook.ffiec.gov/it-booklets/management.aspx | 用 board and senior management oversight、risk management、third-party management、audit、information security management 语言连接金融机构治理 |
| ISO/IEC 42001 overview | https://www.iso.org/standard/42001 | 用 AI management system、policy、roles、operation planning、performance evaluation、internal audit 和 continual improvement 建立 operating model |
Source nuance:
- SEC cyber rule 不是所有 AI incident 的通用披露规则;prompt log exposure、tool token compromise、unauthorized access 或 cyber business disruption 才可能进入该路径分析。
- FTC AI claims 不只影响广告;金融产品中的 AI-generated benefit、approval、risk、fee、speed、accuracy claim 都需要 substantiation thinking。
- NAIC bulletin 面向 insurer AI use;银行、lender、broker 或 fintech 不能机械套用, 但 third-party AI governance 和 consumer outcome 思维可复用。
- NIST AI RMF 和 ISO/IEC 42001 是 structured governance language, 不是单一法律答案。
3. AI Incident Taxonomy
AI incident taxonomy 要按 business consequence, not only technical symptom。
| Class | Definition | Typical trigger |
|---|---|---|
| AI claim incident | AI 生成、选择、改写或发送的 claim 可能误导客户或缺乏 substantiation | marketing copy, chatbot answer, product page, sales script |
| Customer harm incident | AI output or automation caused or contributed to customer loss, denial, delay, unfair treatment or confusion | fee, credit, fraud, complaint, servicing, wealth |
| Regulated decision support incident | AI influenced a regulated workflow with weak evidence, wrong source, biased output or missing human review | underwriting, adverse action, AML, suitability, insurance |
| Cyber/privacy AI incident | AI system exposes, mishandles or enables unauthorized access to sensitive data, prompt logs, embeddings, tokens or tool payload | prompt trace leak, tool credential abuse, vector ACL failure |
| Model behavior incident | drift, hallucination, unsafe instruction following, jailbreak, prompt injection or evaluation failure affects production | unexpected answer pattern, eval failure, guardrail bypass |
| Vendor AI incident | model provider, RAG vendor, data supplier, SaaS copilot or system integrator causes outage, quality loss, data breach or evidence gap | vendor notice, unexplained model change, SLA miss |
| Evidence and record incident | required AI records cannot be preserved, exported, replayed or linked to final customer action | log retention gap, vendor portal unavailable, missing source IDs |
| Governance and oversight incident | AI risk was not escalated, approved, monitored or reported according to policy | shadow AI, unapproved deployment, board reporting gap |
Severity dimensions:
| Dimension | Questions |
|---|---|
| Customer impact | Were customers misled, delayed, denied, overcharged, exposed, disadvantaged or deprived of recourse? |
| Regulated workflow impact | Did AI influence credit, insurance, investment, AML, fraud, complaint, privacy or account access decisions? |
| Data sensitivity | Were PII, account data, KYC/AML notes, complaint details, authentication tokens or proprietary models exposed? |
| Scale and duration | How many customers, messages, accounts, decisions, branches, channels, jurisdictions and days? |
| Reversibility | Can the customer impact be corrected, reversed, compensated or appealed? |
| Evidence quality | Can the institution replay prompt, source, model, tool, approval and final output? |
| External visibility | Has the issue reached customers, media, regulators, plaintiffs, vendors, insurers or investors? |
| Control weakness | Is this isolated misconfiguration or systemic governance failure? |
AI-specific incident objects:
| Object | Why it matters |
|---|---|
ai_run_id | Connects prompt, model, retrieval, tool plan and output |
content_object_id | Links generated text to approved claims and customer communication lifecycle |
model_route_id | Shows provider, model, endpoint, version, region and fallback |
prompt_bundle_id | Shows system prompt, policy prompt, template and release version |
source_manifest_id | Shows RAG corpus, document version, chunk IDs and content hashes |
tool_action_id | Shows real business side effects such as CRM update, account hold or customer notice |
approval_packet_id | Shows human review, authority, evidence seen and reason |
final_channel_event_id | Shows what the customer actually received |
vendor_ticket_id | Connects incident evidence to contract escalation and SLA |
insurance_notice_id | Connects incident facts to risk management notice workflow |
4. Materiality Triage
Materiality triage is decision support, not automated legal conclusion。
Triage principles:
- Separate confirmed facts from assumptions。
- Keep quantitative and qualitative factors together。
- Preserve uncertainty explicitly。
- Record who made each decision and under what privilege or governance process。
- Do not let product, engineering or comms create final disclosure conclusions outside authorized process。
- Re-run triage when facts change。
Workflow:
incident intake
-> preservation and privilege gate
-> preliminary classification
-> affected population estimate
-> financial impact estimate
-> qualitative impact assessment
-> disclosure / notification pathway screen
-> counsel / disclosure committee / risk committee review
-> decision log and reassessment cadence
Decision support matrix:
| Factor | Evidence to gather | Owner |
|---|---|---|
| Public-company status | registrant status, reporting obligations, committee charter | Securities Counsel |
| Cyber nexus | unauthorized access, data exposure, system compromise, ransomware, token abuse | Cyber / Privacy / Legal |
| Operational impact | critical operations affected, outage duration, manual backlog, service degradation | Operational Risk |
| Customer harm | affected customer count, harm type, remediation estimate, vulnerable population | Business Owner / Compliance |
| Financial impact | direct cost, lost revenue, remediation, defense, reserves, vendor recovery | Finance / Risk Management |
| Qualitative significance | trust, regulator attention, board concerns, repeated control weakness | Legal / Risk / Executive |
| Vendor responsibility | contract obligation, SLA, notice, breach, evidence, limitation | Third-Party Risk / Legal |
| Insurance notice | policy line, notice trigger, timing, known facts, claim definition | Insurance / Broker / Counsel |
Public company cyber nuance:
- A cyber-adjacent AI incident may require disclosure committee process if facts indicate material cybersecurity impact。
- Architecture should provide timeline、cyber nexus support、affected systems/data、financial estimate、qualitative impact、control state、known/unknown facts。
- Architecture should not state “material,” “not material,” “no disclosure,” or “covered by insurance.”
Qualitative factors:
| Factor | Why it can matter |
|---|---|
| Customer trust | AI harm may damage trust even with modest direct cost |
| Regulatory sensitivity | Complaint, credit, insurance, AML, privacy and vulnerable customer incidents can draw scrutiny |
| Repeated weakness | Similar prior incidents may indicate systemic control failure |
| Board visibility | Board or committee awareness changes governance record |
| Misleading public claims | Prior AI marketing claims may increase external scrutiny |
| Evidence gap | Inability to reconstruct events may raise defensibility and governance risk |
| Vendor concentration | One provider failure affecting multiple products may indicate systemic dependency |
5. Notification Workflow
Notification workflow must coordinate customer, regulator, board, vendor and insurer channels without allowing inconsistent narratives。
detection
-> incident command starts
-> legal / privilege / preservation gate
-> initial fact ledger
-> notification pathway screen
-> customer communication
-> regulator / examiner communication
-> board / committee notification
-> vendor notice / demand
-> insurer notice
-> approved message control
-> evidence and decision log
-> updates and corrections
Customer notification decision support:
| Question | Evidence |
|---|---|
| Did customers receive incorrect, misleading or incomplete content? | final channel event, content hash, customer exposure list |
| Did customers suffer financial, access, privacy, credit, insurance or complaint harm? | case records, account impact, transaction or decision ledger |
| Can harm be corrected without customer action? | remediation workflow, reversal capability, account update |
| Is customer action required? | identity step-up, account monitoring, complaint path, appeal path |
| Is communication legally or regulatorily prescribed? | legal/compliance determination |
| What can be said with verified facts? | approved fact ledger and message version |
Regulator / examiner binder:
| Topic | Evidence pack |
|---|---|
| System scope | AI use case, products, channels, customer segments, jurisdictions |
| Incident timeline | detection, containment, escalation, decision times |
| Impact | affected population, harm type, financial estimate, operational impact |
| Controls | pre-incident controls, failed controls, compensating controls |
| Evidence | prompt, model, source, tool, approval, final output, audit trail |
| Remediation | customer correction, control fix, testing, CAPA, governance update |
| Vendor role | vendor dependency, notice, response, contract controls |
| Management oversight | committee updates, board notification, decision log |
Board brief should cover: what happened, known/unknown facts, customers and operations affected, cyber/privacy/disclosure/litigation/insurance/vendor exposure under assessment, decisions already made, decisions needed, residual risk, next update cadence, CAPA and risk transfer strategy。
Insurance notice workflow:
| Step | Output |
|---|---|
| Identify potentially implicated policies | cyber, E&O, tech/professional liability, D&O, media, crime, FI bond, property/BI where relevant |
| Capture notice trigger facts | claim, circumstance, wrongful act, cyber event, privacy event, first-party loss, third-party demand |
| Coordinate with broker and counsel | notice wording, privilege, timing, reservation concerns |
| Submit notice under policy process | notice record, acknowledgement, claim number |
| Track consent requirements | defense counsel, forensics, PR, remediation, settlement, vendor costs |
| Update loss runs | paid, incurred, reserve, recovery, vendor offset |
6. Liability Boundary Mapping
Liability boundary mapping asks who controlled which risk and which evidence supports that boundary。
| Layer | Liability questions |
|---|---|
| Institution product layer | Did the institution make a customer promise, regulated communication or decision? |
| AI application layer | Was the output within approved use, prompt, model, source and tool boundary? |
| Human oversight layer | Was human review meaningful, documented and authorized? |
| Vendor layer | Did a vendor breach data, security, availability, model change, subprocessor, service description or support obligation? |
| Data/source layer | Was source data accurate, current, licensed and authorized for AI use? |
| Channel layer | What did the customer actually see, hear, sign or receive? |
| Policy/insurance layer | Which policy line, contract term, exclusion or indemnity may respond? |
| Governance layer | Was the use case approved, monitored, reviewed and escalated according to policy? |
Causation and contribution map:
| Relationship | Evidence needed |
|---|---|
| AI-caused | output/tool action led to customer impact without meaningful human intervention |
| AI-contributed | AI draft, score or summary influenced human decision |
| Human override | human changed or ignored AI output, with documented reason |
| Vendor-caused | vendor outage, model change, data handling or security failure maps to contract obligation |
| Data-caused | incorrect source data or stale policy drove output |
| Control-caused | missing approval, weak guardrail, evidence gap or release failure allowed incident |
Boundary diagram:
customer impact
<- final communication / business action
<- human approval or automation rule
<- AI output / recommendation / tool plan
<- prompt and policy bundle
<- RAG source and data quality
<- model route and vendor behavior
<- governance and release controls
The strongest liability boundary evidence is not a narrative. It is a replayable chain。
7. Vendor Indemnity / SLA / Insurance Mapping
Vendor risk transfer is only useful if contract metadata and runtime evidence are connected before the incident。
| Contract term | AI incident relevance | Evidence needed |
|---|---|---|
| Service description | Defines what vendor promised to provide | service exhibit, model capability, region, support scope |
| SLA / service credits | Availability, latency, support response, recovery time | uptime logs, ticket timestamps, affected workflow evidence |
| Security obligations | Controls for data, access, vulnerability, incident response | security incident timeline, access logs, audit reports |
| Data use restrictions | Training, retention, telemetry, subprocessors, deletion | data flow, vendor settings, DPA, subprocessor list |
| AI model change notice | Notice for model version, behavior, policy, deprecation | release notes, model route logs, change notifications |
| Indemnity | Defense/hold harmless for specified claims | claim facts, covered claim category, tender record |
| Limitation of liability | Caps and carve-outs | damages category, cap table, exclusions from cap |
| Insurance requirements | Vendor policies and limits | certificate of insurance, additional insured, policy type |
| Audit rights | Ability to inspect controls and evidence | audit request, SOC report, incident evidence export |
| Termination/exit | Rights after repeated failure or material breach | breach notice, transition plan, data return/deletion proof |
Vendor types:
| Vendor | Key incident questions |
|---|---|
| Foundation model provider | Did model behavior, availability, data retention, training setting or safety filter change? |
| RAG platform | Were source ACLs, index freshness, document versions or retrieval logs correct? |
| Data provider | Was input data accurate, licensed, timely and fit for purpose? |
| SaaS copilot | Where are prompts, outputs, logs and admin controls stored? |
| System integrator | Did configuration, prompt design, testing or deployment fall below agreed scope? |
| Observability / evidence vendor | Can traces be exported under incident pressure? |
Contract-to-evidence link:
contract obligation
-> control requirement
-> runtime telemetry
-> incident evidence
-> notice / tender / SLA claim
-> recovery or dispute
Example:
| Obligation | Runtime evidence | Incident use |
|---|---|---|
| Vendor must not use customer data for training | model gateway setting, vendor admin export, DPA | breach investigation and customer/regulator response |
| Vendor must notify material security incident promptly | vendor notice time, detection time, ticket | contract compliance and insurer notice |
| Vendor must maintain audit logs | trace export, retention setting, missing log report | evidence gap and indemnity discussion |
| Vendor must provide availability SLA | synthetic monitor, workflow outage, customer impact | SLA credit and business interruption analysis |
8. Risk Transfer Options
Risk transfer is a portfolio of mechanisms. None substitutes for control design。
| Option | What it transfers | Limits |
|---|---|---|
| Contract indemnity | Some third-party claims or losses tied to vendor conduct | scope, caps, exclusions, notice, defense control |
| SLA credits | Part of service fee for availability/support failure | rarely covers full business or customer loss |
| Limitation carve-outs | Higher or uncapped liability for confidentiality, data breach, IP, gross negligence where negotiated | difficult to obtain; wording matters |
| Insurance | First-party and third-party financial consequences depending on policy | coverage depends on facts, wording, exclusions, notice and consent |
| Captive / self-insurance | Retained risk funded internally | requires actuarial and capital discipline |
| Reserves | Financial recognition of expected loss | accounting and legal review needed |
| Vendor diversification | Reduces concentration and operational impact | adds complexity and integration risk |
| Technical risk reduction | Prevents or limits loss before transfer | not transfer, but often best economic option |
| Customer contract terms | Allocates some responsibilities and disclaimers | consumer protection limits and fairness considerations |
Risks often retained despite transfer language:
- Product claim accuracy。
- Board and management oversight。
- Primary regulator relationship。
- Customer remediation before recovery。
- Evidence gaps。
- Vendor concentration。
- Systemic control weakness。
Design principle:
Transfer follows evidence.
If the institution cannot prove vendor route, failed obligation, affected output,
loss amount and mitigation actions, transfer mechanisms become negotiation,
not architecture.
9. Cyber vs E&O vs Tech / Professional Liability Mapping
Insurance mapping should be done with broker and coverage counsel. The architecture role is to supply clean facts and loss categories。
| Policy line | AI incident fit | Typical evidence |
|---|---|---|
| Cyber | unauthorized access, privacy event, network security failure, data breach, cyber BI, incident response cost | forensic report, data map, notice timeline, affected records, access logs |
| E&O / professional liability | alleged error, omission, negligence or failure in professional/financial services | service promise, customer claim, decision record, standard of care evidence |
| Tech E&O | technology product or service failure causing client loss | system logs, service contract, defect evidence, outage impact |
| Media / advertising liability | misleading content, advertising injury, IP, defamation-like content depending on wording | content object, claim substantiation, publication record |
| D&O | governance, securities claim, misstatement, oversight allegations | board materials, disclosure record, management decisions |
| Crime / FI bond | employee dishonesty, certain fraud or computer crime depending on wording | fraud evidence, access logs, loss proof |
Coverage-adjacent fields:
| Field | Why it matters |
|---|---|
| Date first discovered | policy period, notice timing, prior knowledge |
| Claim vs circumstance | policy trigger may differ |
| Wrongful act | E&O / D&O wording may require alleged wrongful act |
| Cyber event | cyber policy may require security failure or privacy event |
| Professional service | E&O may require service within covered professional services |
| Technology product | tech E&O may require product/service failure |
| Retroactive date | incident before retro date may be excluded |
| Prior acts / prior notice | past related facts can affect coverage |
| Exclusions | fraud, contractual liability, unauthorized collection, discrimination, IP, fines/penalties depending on wording |
| Consent | insurer consent may be needed for counsel, forensics, PR, settlement or remediation costs |
| Retention and sublimit | economic recovery depends on retention and sublimits |
Notice pack:
| Item | Include |
|---|---|
| Basic facts | who, what, when, where, how known |
| Policyholder entity | legal entity and product line |
| Potential claimants | customers, regulators, partners, vendors |
| Incident category | cyber, privacy, E&O, media, D&O, vendor failure, mixed |
| Loss categories | response, remediation, defense, business interruption, vendor recovery |
| Current actions | containment, preservation, counsel, customer care |
| Known uncertainty | facts still under investigation |
| Reservation | notice preserves rights without admitting liability or coverage conclusion |
10. Loss Quantification
Loss quantification supports materiality, reserves, insurance notice, vendor recovery and remediation planning。
| Category | Examples |
|---|---|
| Customer remediation | refunds, fee reversals, interest, account correction, goodwill credits |
| Operational response | overtime, manual review, call center surge, complaint handling |
| Legal and professional fees | outside counsel, coverage counsel, forensic experts, consultants |
| Customer notification | printing, mailing, call center, identity monitoring if applicable |
| Regulatory response | exam support, data production, remediation program, independent review |
| Defense and settlement | litigation defense, arbitration, settlement, judgment |
| Business interruption | lost revenue, delayed launches, suspended products, reduced automation |
| Reputational and retention impact | churn, reduced conversion, higher acquisition cost |
| Vendor recovery | SLA credits, indemnity, damages claim, offset |
| Control remediation | new tooling, monitoring, validation, staff training |
| Insurance cost | retention, premium impact, uncovered costs |
Model:
gross_loss =
customer_remediation
+ response_cost
+ legal_professional_cost
+ regulatory_response_cost
+ defense_settlement_estimate
+ business_interruption_estimate
+ control_remediation_cost
net_loss_range =
gross_loss
- expected_vendor_recovery
- expected_insurance_recovery_after_retention
+/- uncertainty_adjustment
Minimum affected population query dimensions:
- model version。
- prompt version。
- source document version。
- approved claim id。
- tool action type。
- customer segment。
- product。
- jurisdiction。
- channel。
- employee role。
- time window。
- final customer message status。
- remediation status。
Loss evidence:
| Loss item | Evidence |
|---|---|
| Fee refund | account ledger, customer list, approval, payment record |
| Manual review cost | queue volume, reviewer hours, rate, case count |
| Legal cost | engagement letter, invoice, matter code |
| Vendor recovery | contract clause, breach evidence, tender letter, credit memo |
| Insurance recovery | notice, claim number, coverage correspondence, proof of loss |
| Business interruption | baseline, incident period, causation analysis, mitigation record |
11. Evidence Pack
Evidence pack should support counsel, regulator, insurer, vendor recovery, audit and internal learning。
Evidence principles:
- Preserve before cleanup。
- Capture raw evidence and curated explanation separately。
- Keep chain-of-custody。
- Maintain privilege where directed by counsel。
- Preserve final customer-visible content, not only drafts。
- Preserve tool side effects and human approvals。
- Avoid overwriting model, prompt or RAG source versions。
- Track evidence gaps explicitly。
Minimum evidence fields:
| Field | Purpose |
|---|---|
| incident_id | common reference |
| detection_source | alert, complaint, audit, vendor, regulator, employee |
| first_detected_at | timeline and notice timing |
| incident_class | taxonomy and routing |
| use_case_id | product and owner |
| customer_population | affected or potentially affected |
| ai_run_ids | prompt/model/retrieval/output chain |
| model_route | provider, model, endpoint, version, region |
| prompt_bundle | system prompt, template, release id |
| source_manifest | RAG document ids, versions, hashes |
| tool_actions | calls, parameters, side effects, approvals |
| final_content | rendered customer message or employee action |
| human_review | reviewer role, evidence seen, decision |
| vendor_dependency | provider, ticket, SLA, contract reference |
| control_state | which controls passed, failed or were degraded |
| containment_action | safe-stop, rollback, disable route, customer hold |
| legal_hold_status | hold id, scope, affected systems |
| insurance_notice_status | policy line, notice date, claim number |
| communication_versions | customer, board, regulator, vendor, insurer messages |
| CAPA | corrective and preventive actions |
Fact ledger states:
| State | Meaning |
|---|---|
| Confirmed | supported by evidence and reviewed by owner |
| Likely | supported but still under review |
| Unknown | material question not yet answered |
| Disputed | conflicting evidence or parties disagree |
Replay package:
customer context allowed at time
prompt and policy bundle
retrieved source IDs and source hashes
model/provider/version
AI output
claim scan or policy decision
human edit and approval
tool action and side effect
final customer communication
complaint or remediation record
incident containment and correction
12. Tabletop Exercises
Exercises should test decision rights, evidence, communication and risk transfer, not only technical recovery。
Scenario 1: Misleading AI Product Claim
An AI-generated credit card landing page variant says customers are "guaranteed approval"
after a soft-check prequalification flow. The variant ran for 18 hours in two states.
Complaints mention annual fee confusion and approval denial after application.
Expected actions: freeze campaign, preserve content object/prompt/approval/audience/applications, build affected cohort, route to claim substantiation and remediation review, screen customer/regulator/board/insurer/vendor pathways。 Evidence: final pages, claim library entry, campaign approvals, customer exposure list, application outcomes, complaints, model/prompt/version, correction message。
Scenario 2: Prompt Trace Data Exposure
A vendor support portal exposes AI prompt traces containing account context and complaint notes.
The exposure window is uncertain. Vendor states no evidence of misuse.
Expected actions: activate cyber/privacy and AI incident bridge, legal hold, determine data fields and affected population, evaluate notification pathways, provide cyber/E&O notice support as appropriate, review vendor breach/data/subprocessor obligations。 Evidence: vendor notice, access logs, trace samples, data field inventory, customer mapping, DPA and contract terms, insurer notice record, remediation plan。
Scenario 3: Lending AI Decision Support Error
A lending policy assistant summarized an exception policy incorrectly for three weeks.
Underwriters used the summaries in decline rationale drafts.
Expected actions: stop affected route, preserve AI runs/edits/final notices, query affected applicants, review fair lending/adverse action/remediation implications, map AI contribution vs human approval vs source issue, quantify customer remediation and defense cost。 Evidence: prompt/source manifests, approval packets, final notice versions, applicant cohort, policy effective date, QA findings, remediation decisions。
Scenario 4: Vendor Model Change
The model provider silently changes safety and summarization behavior.
Complaint response drafts become more definitive and omit conditional language.
Expected actions: freeze route, compare output drift and employee copy behavior, preserve vendor release notes/model route logs, check contract model change notice/audit rights, analyze exposure and correction need。 Evidence: model route IDs, output samples, employee edit deltas, final sent emails, vendor communications, contract clauses, complaint links。
Scenario 5: Evidence Export Failure
During a customer-facing AI incident, the observability vendor cannot export traces
for high-risk workflows. Application logs are partial but final channel messages exist.
Expected actions: enable local evidence ledger, move Tier 1 workflows to draft-only or safe-stop, preserve final channel records, escalate vendor audit/export rights, assess defensibility gap, capture insurance and vendor recovery implications。 Evidence: failed export logs, local ledger, final messages, missing trace report, vendor SLA/ticket, recovery gate record。
13. Operating Model
Decision rights:
| Decision | Accountable | Consulted |
|---|---|---|
| Declare AI incident | Incident Commander / Operational Risk | AI Product, Cyber, Legal |
| Open legal hold / preservation | Legal | Records, IT, Vendor Owner |
| Classify cyber/privacy pathway | Cyber / Privacy Legal | Security, Product, Vendor |
| Trigger materiality support process | Legal / Securities Counsel | Finance, Risk, Product |
| Notify board / committee | Executive Owner | Legal, Risk, Communications |
| Customer communication | Business Owner + Legal/Compliance | Communications, Operations |
| Regulator / examiner response | Legal / Compliance | Business, Risk, Audit |
| Vendor notice / tender | Third-Party Risk + Legal | Product, Procurement |
| Insurance notice | Insurance/Risk Management | Broker, Coverage Counsel, Legal |
| Approve remediation | Business Owner / Risk Committee | Legal, Compliance, Finance |
| Restore AI capability | Business Owner + AI Governance | Platform, Model Risk, Compliance |
| Close incident | Incident Commander + Legal/Risk | Audit, Business, Insurance |
RACI:
| Activity | Accountable | Responsible | Consulted | Informed |
|---|---|---|---|---|
| Taxonomy and intake | Operational Risk | AI Governance | Legal, Cyber, Product | Business owners |
| Evidence preservation | Legal | Records / Platform | Privacy, Vendor Owner | Audit |
| Affected population query | Business Owner | Data / Analytics | Compliance, Legal | Risk |
| Materiality support pack | Legal / Securities Counsel | Product / Process Analysis / Finance | Cyber, Risk, Product | Disclosure committee |
| Vendor contract mapping | Third-Party Risk | Procurement / Legal Ops | Product, Security | Risk Committee |
| Insurance notice pack | Risk Management | Insurance team / Broker | Legal, Finance, Product | Executive Owner |
| Customer remediation | Business Owner | Operations | Legal, Compliance, Finance | Frontline |
| Board reporting | Executive Owner | Risk / Legal | Product, Finance, Cyber | Board committee |
| CAPA tracking | Operational Risk | Product / Platform | Audit, Compliance | Senior management |
Governance cadence:
| Cadence | Forum | Output |
|---|---|---|
| Hourly during critical incident | Incident bridge | facts, containment, evidence, decisions |
| Daily during active response | Executive risk huddle | impact, customer, regulator, insurer, vendor status |
| Weekly until closure | AI incident review | loss estimate, remediation, CAPA, recovery |
| Monthly | AI governance committee | trends, KRIs, control gaps, vendor issues |
| Quarterly | Board risk or technology committee | material themes, risk transfer, systemic dependencies |
| Annual | AI incident tabletop | disclosure, liability, insurance, vendor recovery drill |
14. Metrics and KRIs
Incident metrics:
| Metric | Why it matters |
|---|---|
| Time to classify AI incident | delayed classification weakens preservation and notice |
| Time to legal hold | evidence may be deleted or overwritten |
| Time to affected population estimate | supports customer, board and materiality decisions |
| Evidence completeness rate | measures defensibility |
| Final-channel capture rate | proves customer-visible exposure |
| Vendor evidence export SLA | measures third-party resilience |
| Insurance notice elapsed time | preserves potential rights |
| Customer remediation cycle time | reduces harm and complaint escalation |
| CAPA closure age | shows governance follow-through |
KRIs:
| KRI | Example threshold |
|---|---|
| High-risk AI outputs without trace | zero tolerance for Tier 1 workflows |
| Customer-visible content without claim ID | escalation threshold after first confirmed event |
| Vendor incidents without contract mapping | management action if any critical vendor missing matrix |
| AI incidents with unknown affected population after 48h | executive escalation |
| Insurance notice not assessed within defined window | risk management escalation |
| Repeated incidents by prompt/source/model route | systemic review |
| Evidence export failures during incident | vendor risk committee review |
| Materiality support pack missing financial estimate | disclosure process blocker |
| Board update without known/unknown distinction | governance quality defect |
Dashboard views: incident class trend, use case heatmap, customer impact, remediation status, vendor contribution, SLA status, insurance notice and claim status, loss estimate range, evidence completeness, CAPA aging, repeat control failures。
15. 30-Day Lab
目标: 30 天内完成一个 AI Incident Disclosure / Liability / Insurance / Risk Transfer system evidence pack。推荐选择 credit card AI marketing claim、lending AI decision support、complaint response agent、insurance underwriting AI 或 prompt trace exposure。
| Day | Task | Artifact |
|---|---|---|
| 1 | 选择 scenario, 明确 product, channel, customer segment | Scenario Boundary Card |
| 2 | 写 incident taxonomy and severity dimensions | AI Incident Taxonomy |
| 3 | 画 AI runtime dependency graph | Model/RAG/Tool/Vendor Graph |
| 4 | 定义 evidence objects and trace IDs | Evidence Object Catalog |
| 5 | 设计 legal hold and preservation gate | Preservation Workflow |
| 6 | 写 materiality support workflow | Materiality Support Map |
| 7 | 设计 affected population query dimensions | Exposure Query Spec |
| 8 | 设计 customer harm classification | Customer Harm Matrix |
| 9 | 写 customer notification decision support | Customer Communication Matrix |
| 10 | 写 regulator / examiner response binder outline | Regulator Binder |
| 11 | 写 board brief template | Board One-Pager |
| 12 | 做 liability boundary map | Liability Boundary Diagram |
| 13 | 建 vendor contract matrix | Vendor Risk Transfer Matrix |
| 14 | 写 vendor evidence request workflow | Vendor Evidence Request |
| 15 | 建 insurance policy map | Policy Line Mapping |
| 16 | 写 insurance notice card | Notice Support Pack |
| 17 | 建 loss quantification model | Loss Model |
| 18 | 写 remediation and correction workflow | Customer Remediation Plan |
| 19 | 设计 communication approval workflow | Message Control Flow |
| 20 | 写 tabletop scenario 1: misleading claim | Scenario Script |
| 21 | 写 tabletop scenario 2: prompt trace exposure | Scenario Script |
| 22 | 写 tabletop scenario 3: vendor model change | Scenario Script |
| 23 | 运行 90 分钟 tabletop, 记录 decisions | Decision Log |
| 24 | 生成 gaps and CAPA backlog | CAPA Register |
| 25 | 设计 metrics and KRIs dashboard | KRI Dashboard Spec |
| 26 | 写 operating model and RACI | Operating Model |
| 27 | 写 executive incident brief | Executive Brief |
| 28 | 写 system case narrative | Decision Narrative |
| 29 | 整理 8 个 design review questions | Design Review Questions |
| 30 | 打包 artifact index and evidence storyline | System Evidence Pack |
16. Decision Narrative
Disclosure Support Architecture
核心判断:
AI incident disclosure architecture 不让系统自动决定是否披露。它把事故事实、客户影响、业务影响、控制状态、vendor dependency、insurance notice 和证据链组织成 decision-ready pack, 支持 legal、disclosure committee、risk、board、regulator 和 insurer-facing workflow。架构边界是提供证据和结构, 不做法律或覆盖结论。
系统推演:
每个 incident record 要连接 ai_run_id、source manifest、model route、tool action、approval packet、final channel event、affected population query、loss estimate、vendor contract reference 和 insurance notice assessment。决策者看到的不是“模型坏了”这一句, 而是 confirmed facts、unknowns、customer exposure、control weakness、timeline、open decisions 和 evidence gaps。
Cyber Disclosure Boundary
核心判断:
是否进入 cybersecurity disclosure analysis 取决于事实是否存在 cyber nexus, 例如 unauthorized access、data exposure、system compromise、tool token abuse 或 cyber business disruption。SEC cyber disclosure anchor 面向 registrants 的 material cybersecurity incidents, 不是所有 AI incident。
系统推演:
AI incident intake 要把 cyber/privacy facts 与一般 model quality issue 分离: access path、data exposure、credential/token status、system compromise、business disruption、financial impact、qualitative impact、control status 和 uncertainty 都要进入 materiality support pack, 由 securities counsel 和 disclosure process 判断。
AI Claims Incident Control
核心判断:
AI 功能 claim、收益 claim、approval claim、accuracy claim、risk claim 都需要 substantiation。内容系统不能只生成流畅文案, 还要管理 approved claim library、forbidden claim、evidence source、pre-use review、final-channel capture 和 correction workflow。
系统推演:
Marketing、product page、chatbot、advisor script 和 customer email 都应连接 claim ID、supporting evidence、approval status、publication channel、audience、expiry 和 correction owner。事故发生时, 系统能定位谁看到了哪个 claim、依据是什么、是否过期、如何下架或更正。
Liability Boundary Reconstruction
核心判断:
Liability boundary 从客户影响往回追: final customer communication or business action、human approval、AI output、prompt/policy bundle、RAG source、tool action、model route、vendor dependency 和 governance controls。责任分析需要区分 AI-caused、AI-contributed、human override、vendor-caused、data-caused 和 control-caused。
系统推演:
Evidence pack 不只保存 output, 还要保存 source version、retrieval reason、tool payload、approval evidence、policy decision、vendor SLA state、exception approval 和 final customer artifact。这样才能把客户损害、系统行为、人类决策和第三方边界连接起来。
Insurance Mapping Discipline
核心判断:
Insurance mapping 的第一原则是不先下“赔不赔”结论。事实应被映射到可能相关的 policy lines: cyber、E&O、tech E&O、media、D&O、crime 等, 再准备 notice trigger、date discovered、claim/circumstance、loss category、evidence、consent requirements 和 uncertainty。
系统推演:
Coverage conclusion 由 broker、coverage counsel 和 insurer process 确认。架构要确保 notice support pack 可以在时限内产出: facts, timeline, affected systems, loss categories, mitigation steps, vendor contribution, known/unknown distinction, preservation status and communication approvals。
Vendor Recovery And Insurance Rights
核心判断:
Vendor indemnity 和 insurance 是不同 risk transfer channels。Vendor indemnity 依赖合同义务、breach、notice、cap 和 evidence;insurance 依赖 policy wording、facts、notice、exclusions 和 retention。成熟架构同时保留 vendor recovery 和 insurance notice 权利, 但不假设任何一方自动补偿全部损失。
系统推演:
Vendor contract matrix 要记录 SLA、audit right、subprocessor、indemnity、limitation of liability、notice path、required evidence、insurance requirement 和 cooperation obligation。Insurance map 要记录 policy line、notice window、loss bucket、consent-to-settle or remediation constraints, 并与 incident timeline 对齐。
Evidence Gap As Incident
核心判断:
Evidence gap 本身就是 AI incident。无法复原 prompt、source、model、tool、approval 和 final customer message 时, 机构无法证明发生了什么、客户是否受影响、谁应负责、是否需要通知、保险或 vendor recovery 是否成立。
系统推演:
Evidence export failure、missing source IDs、vendor portal unavailable、retention mismatch、trace ID discontinuity 和 final-channel capture gap 都应进入 incident taxonomy。它们会削弱 disclosure support、defense、regulator response、claim recovery 和 CAPA prioritization。
Risk Transfer Value
核心判断:
AI 风险不能只靠买保险转移。治理层需要知道哪些风险由 product controls 降低, 哪些由合同转移给 vendor, 哪些通过 insurance notice 保留, 哪些必须自留并准备 reserves。
系统推演:
Risk transfer architecture 的价值是把证据、合同、保单、损失和治理放到同一张图上。它让 incident response 不只恢复服务, 还同步完成 fact preservation、loss quantification、recovery option preservation、communication control 和 corrective action prioritization。
17. System Evidence Pack
| Evidence object | System value |
|---|---|
| AI Incident Taxonomy | 按 business consequence 分类 customer、cyber、claim、vendor、governance and evidence incidents |
| Materiality Support Worksheet | 组织 quantitative、qualitative、known/unknown、timeline and decision owner fields |
| Liability Boundary Map | 连接客户影响、AI runtime、人类审批、数据来源、工具动作和 vendor responsibility |
| Vendor Contract Matrix | 把 TPRM、SLA、indemnity、audit rights、notice and cooperation duties 变成操作资产 |
| Insurance Policy Map | 区分 cyber/E&O/tech E&O/D&O/media 等 policy lines, 同时避免越权给 coverage opinion |
| Loss Quantification Model | 量化 customer remediation、operations、legal、business interruption、vendor recovery、insurance recovery and retained risk |
| Evidence Pack Schema | 定义 replayable, defensible, cross-functional incident evidence |
| Communication Workflow | 控制 customer、board、regulator、vendor、insurer messages 的版本、审批和最终发送记录 |
| Tabletop Exercise Scripts | 验证 decision rights, preservation gates, vendor escalation and risk transfer readiness |
| Executive One-Pager | 用 governance language 表达 AI incident exposure、options、unknowns and residual risk |
| CAPA Register | 把事故发现连接到 funded corrective and preventive actions |
| Decision Narrative | 说明系统如何保全事实、量化影响、映射责任、支持决策并保留追偿路径 |
System narrative:
The AI incident risk transfer architecture preserves facts before conclusions,
quantifies customer and financial impact before communication,
maps liability before recovery actions,
and keeps disclosure, vendor, insurance and board workflows tied to the same evidence base.
18. Design Review Questions
| Review area | Questions | Evidence to inspect |
|---|---|---|
| Incident boundary | Does the taxonomy classify business consequence, not just model symptom? | taxonomy, severity dimensions, intake form |
| Preservation | Can legal hold and evidence export run before cleanup or model rollback? | preservation workflow, hold log, export test |
| Customer impact | Can affected population be bounded by model, prompt, source, tool, channel and time? | exposure query spec, population estimate, uncertainty log |
| Disclosure support | Are quantitative and qualitative materiality inputs separated from legal conclusions? | materiality worksheet, decision log |
| Liability | Can customer action be traced back to AI output, human approval, source, tool and vendor route? | liability map, trace IDs, approval packets |
| Risk transfer | Are vendor notice and insurance notice rights preserved without assuming recovery? | contract matrix, policy map, notice card |
| Communication | Are customer, regulator, board, vendor and insurer messages consistent but separately approved? | communication workflow, final-channel capture |
| CAPA | Does the incident produce funded control improvements, not only a closure note? | CAPA register, owner, due date, validation evidence |
19. Self-Assessment Checklist
| Check | Passing evidence |
|---|---|
| Taxonomy exists | incidents classified by customer, cyber, claim, vendor, governance and evidence consequence |
| Preservation gate works | legal hold and evidence export occur before cleanup |
| Affected population can be bounded | exposure query by model/prompt/source/tool/channel/customer |
| Materiality support is structured | quantitative, qualitative, known/unknown and decision owner fields exist |
| Customer notification workflow exists | approved message, affected population, remediation and capture |
| Regulator binder exists | scope, timeline, impact, controls, remediation and governance evidence |
| Board pack exists | concise decision-ready view with risk transfer and residual exposure |
| Vendor matrix exists | SLA, indemnity, notice, limitation, audit and insurance fields mapped |
| Insurance notice pack exists | policy line, notice basis, facts, loss categories, uncertainty |
| Loss model exists | gross loss, net loss, vendor recovery, insurance recovery and retained risk |
| Tabletop tested | misleading claim, prompt exposure, vendor change and evidence failure scenarios exercised |
| CAPA linked | incident findings tied to funded corrective and preventive actions |
20. Final Operating Principle
AI incident maturity can be tested with one question:
If a serious AI incident happens today, can the institution preserve evidence,
bound customer harm, support disclosure decisions, notify the right parties,
map liability, preserve insurance rights, pursue vendor recovery,
quantify loss, communicate consistently, and prove every step later?
如果答案不清楚, 问题不是少一个 incident form。问题是 AI product architecture、third-party risk、insurance, legal, compliance, board governance and evidence engineering 还没有被真正连接起来。