AI 底层逻辑 / 经典论文

AI Risk Quantification：场景损失与控制 ROI 架构

AI risk quantification 的价值不是把所有风险精确算成一个数字，而是把“AI 可能出错”翻译成可以比较、可以挑战、可以投资、可以复核的经济决策语言。

552 行ai-foundations/papers/170-ai-risk-quantification-scenario-loss-control-roi-architecture.md

AI 风险量化架构：Scenario Loss / Control ROI / Residual Risk

配对阅读：本篇的操作手册版（模板/RACI/门禁/runbook）是 docs/AI_RISK_QUANTIFICATION_SCENARIO_LOSS_CONTROL_ROI_PLAYBOOK.md。第一遍读本篇建立原理与架构判断；第二遍做案例时再用 playbook 查表落地，两者不需要重复精读。

Date: 2026-06-30 Status: evergreen Audience: 已具备 CBAP 级业务分析基础的金融零售 AI 产品、架构与治理从业者。 Output: 一套把 AI risk scenario 转成 expected loss range、residual risk、control ROI、investment priority 和 management action 的决策架构笔记。

Source Anchors

Source	Link	在本文中的用法
NIST AI Risk Management Framework	https://www.nist.gov/itl/ai-risk-management-framework	用 Govern / Map / Measure / Manage 组织 AI risk scenario 的识别、度量、处置和治理证据；正式项目需按访问日期复核版本状态。
NIST AI RMF Generative AI Profile	https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence	用于 GenAI 特有 scenario：hallucination、grounding failure、misuse、supply-chain dependency、content risk 和 evaluation gap。
ISO/IEC 23894:2023	https://www.iso.org/standard/77304.html	参考 AI risk management 如何进入组织活动、AI lifecycle 和风险处置过程。
ISO/IEC 42001:2023	https://www.iso.org/standard/81230.html	用 AI management system 的视角连接 policy、objective、risk treatment、operation、performance evaluation 和 continual improvement。
NIST SP 800-30 Rev. 1	https://csrc.nist.gov/publications/detail/sp/800-30/rev-1/final	参考 risk assessment、likelihood、impact、residual risk、control mitigation 和 senior leader decision support。
BIS Principles for Operational Resilience	https://www.bis.org/bcbs/publ/d516.htm	用 operational resilience 思维处理 vendor outage、technology failure、wide-scale disruption 和 business service impact。

核心导读

AI risk quantification 的价值不是把所有风险精确算成一个数字，而是把“AI 可能出错”翻译成可以比较、可以挑战、可以投资、可以复核的经济决策语言。

在金融零售里，定性风险等级只能回答“这个场景看起来严重不严重”。量化架构要进一步回答：

这个 AI 场景暴露了多少业务事件？
哪些 failure mode 会转化成真实 loss event？
频率和严重度假设来自历史事件、eval、shadow mode、QA、红队、vendor SLA 还是专家估计？
控制措施到底降低了频率、严重度、检测延迟、恢复成本，还是只是增加了治理外观？
控制成本、控制摩擦和残余风险下降是否匹配？
上线、扩展、缩小范围、补控制、接受残余风险或暂停，哪个动作最合理？

AI failure mode
  -> business loss scenario
  -> exposure
  -> frequency distribution
  -> severity distribution
  -> gross expected loss / tail loss
  -> control mechanism and cost
  -> residual expected loss / tail loss
  -> management action

1. 问题定义：从风险标签到经济决策

本文不是法律、审计、精算、资本计量或监管意见，也不展开 risk appetite policy、board reporting、customer harm、incident liability 和 continuous control monitoring 的完整框架。本文聚焦一个高级问题：如何把 AI 风险场景转成可比较、可投资、可挑战、可复核的决策语言。

AI 项目常见风险讨论停在三个层次：

High / Medium / Low
严重 / 一般 / 可接受
加一个 guardrail / 加一个 human review / 做一些监控

这些表达对初筛有帮助，但不足以支持路线图和架构投资。真正需要决策的是：

同样预算应投 RAG grounding、agent permission gateway、human review capacity、vendor redundancy 还是 eval coverage？
某个控制能把高频低损失压下来，还是能压缩低频高损失的 tail risk？
一项控制降低了多少风险，又引入了多少延迟、误报、人工负荷和客户放弃？
哪些 residual risk 需要限期接受，哪些必须降低，哪些应通过 product scope 避免？

1.1 量化不是精确预测

高级风险量化不是制造虚假精度。它强调 range、percentile、assumption quality、sensitivity 和 review cadence。一个粗糙但透明的区间，通常比一个看似精确但无法解释来源的单点数字更有治理价值。

弱表达	强表达
这个 AI 风险很高	在 35k/month 暴露量下，unsupported regulated recommendation 的 gross monthly expected loss 约 105k，stress case 可达 1m；citation gate + boundary routing 可把 base residual 降到 32k-58k，但会降低部分 advisor adoption。
加 human review 就安全	Human review 主要降低最终客户影响的 severity，但如果 queue SLA 不够或 reviewer 没有证据包，frequency 不一定下降，detection delay 可能上升。
模型 accuracy 达标	Model accuracy 不等于业务损失，必须把 defect 映射到 workflow exposure、customer impact、remediation cost 和 tail scenario。

2. 架构模型：Scenario Loss Decision Architecture

AI 风险量化不是一个 spreadsheet，而是一套 decision architecture。它将 use case boundary、scenario register、assumption set、control catalog、residual risk model 和 decision record 连接到 release gate 与 investment prioritization。

flowchart LR
  A[AI Use Case Scope] --> B[Scenario Register]
  B --> C[Frequency Assumption]
  B --> D[Severity Assumption]
  C --> E[Gross Loss Range]
  D --> E
  E --> F[Control Catalog]
  F --> G[Control Effectiveness Estimate]
  G --> H[Residual Loss Range]
  H --> I[Risk Appetite Threshold]
  H --> J[Control ROI]
  I --> K[Management Action]
  J --> L[Investment Prioritization]
  K --> M[Release / Scale Decision]
  L --> M
  M --> N[Evidence Packet]

ASCII 版：

scenario -> likelihood x severity -> gross loss range
gross loss range -> control effectiveness -> residual loss range
residual loss range -> appetite threshold -> management action
control cost vs loss reduction -> ROI -> architecture/product priority

2.1 Core Objects

Object	作用	关键字段	Owner
Use case boundary	限定量化范围	system、channel、user group、decision/action boundary、traffic volume、model/vendor、release stage	Product + Architecture
Scenario register	列出可量化风险场景	scenario id、failure mode、business event、loss event、affected population、trigger	Product + Risk
Assumption set	记录频率和严重度假设	baseline frequency、stress frequency、severity components、confidence、source、review date	Risk + Finance + Ops
Gross loss model	估算控制前损失	expected loss、P50/P90/P95、tail scenario、confidence interval	Risk / Finance
Control catalog	列出可选控制组合	control id、type、target mechanism、cost、latency、coverage、owner	Architecture + Control Owner
Effectiveness model	估算控制如何降低损失	frequency reduction、severity reduction、detection reduction、recovery reduction、dependency	EvalOps + Risk
Residual risk model	估算控制后风险	residual expected loss、residual P95、threshold status、risk owner、expiry	Risk Owner
Decision record	把结论转成行动	invest / defer / reduce scope / accept / transfer / avoid、rationale、evidence	Governance Forum

2.2 Loss Event Taxonomy

风险量化的单位应是 business loss event，而不是抽象模型错误。

Loss event type	示例	量化口径
Direct financial loss	payment fraud false negative 导致欺诈损失	confirmed fraud amount、recovery rate、chargeback / reimbursement
Operational cost	AML missed escalation 后补查、QA、case reopening	analyst hours、QA hours、manager review、backlog cost
Compliance remediation cost	KYC wrongful rejection 或 regulated advice hallucination 后整改	re-review population、notification、external counsel、audit support
Revenue / conversion loss	KYC 误拒导致开户流失	lost margin、expected lifetime value、reactivation rate
Resilience cost	vendor outage 导致 AI channel 降级	manual fallback capacity、SLA penalty、lost service productivity
Control friction cost	控制带来人工复核、延迟、误报	incremental review cost、abandonment、AHT increase

2.3 Four Loss Views

同一个 scenario 至少看四个视图，避免只盯平均损失。

View	Formula	用途
Expected loss	frequency x average severity	投资优先级、预算和常规 trade-off
Stress loss	stress frequency x stress severity	韧性设计、incident planning、release gate
Tail loss	P95 or P99 severity range	高影响上线决策和管理层承受度
Net value	business benefit - residual expected loss - control cost - friction cost	产品组合、funding gate、scale decision

2.4 Quantification Grain

AI 风险不是只按系统量化，更要按 decision grain 量化：

Grain	用途
per response	客服 RAG、regulated advice、contact center misinformation
per case	AML investigation、KYC review、fraud triage
per transaction	payment fraud false negative、payment scam intervention
per customer journey	onboarding、complaint、wealth suitability support
per vendor dependency hour	model outage、embedding service outage、vector DB outage
per release	model upgrade、prompt policy change、tool permission change

3. 关键机制与生命周期

3.1 Step-by-Step Method

1. Define business decision boundary.
2. Write scenario as loss event, not model error.
3. Estimate exposed population and event frequency.
4. Estimate severity distribution by loss component.
5. Calculate gross expected loss and tail loss.
6. Map candidate controls to loss mechanisms.
7. Estimate control effectiveness and control cost.
8. Calculate residual loss range and ROI.
9. Compare against threshold and product value.
10. Record decision, assumptions and evidence.

3.2 Scenario Statement Pattern

弱表达：

模型可能 hallucinate。

强表达：

In the regulated wealth education copilot, the model may produce an unsupported
product-specific recommendation that a frontline advisor reuses in a customer
conversation. Loss occurs when the customer acts on advice outside approved
suitability workflow, requiring remediation review, complaint handling and
potential client correction.

字段化：

Field	内容
Scenario	regulated advice hallucination reused by advisor
Trigger	customer asks product-specific question; answer not grounded in approved source
Exposure	35k advisor-assisted responses / month
Frequency assumption	0.08%-0.18% unsupported high-risk answers before control
Severity components	review cost, complaint handling, remediation, potential lost relationship margin
Candidate controls	RAG source gating, recommendation phrase blocker, licensed advisor workflow, QA sample, eval gate
Decision use	whether to allow customer-specific product discussion in pilot

3.3 Frequency Estimate

Frequency should be estimated as a range and tied to observable evidence.

Evidence source	用法	注意点
Historical incidents	类似客服误导、AML miss、fraud loss、KYC rejection	历史系统不一定等同 AI system
Eval / red-team sample	从 prompt suite、golden set、adversarial set 推断 defect rate	样本设计偏差要记录
Production shadow mode	AI 输出不执行，与人工结果比对	需要足够覆盖高风险 segment
QA / manual review	抽样检查 output quality 和 control bypass	reviewer consistency 要校准
Vendor SLA / outage history	vendor outage scenario 的频率假设	要考虑 shared dependency concentration
Expert elicitation	新场景无历史数据时使用	必须记录 confidence 和复核日期

频率表达示例：

Confidence	Monthly frequency assumption	说明
Low	0.02%-0.20% of exposed events	只有红队和专家估计，未经过 shadow mode
Medium	0.04%-0.11% of exposed events	有 4 周 shadow mode 和 QA 抽样
High	0.05%-0.08% of exposed events	有 3 个月 production telemetry、review calibration 和 stable policy corpus

3.4 Severity Estimate

Severity 不应只填一个 dollar amount。高级做法是拆成组件，并明确哪些组件在 expected loss 中稳定出现，哪些只在 tail scenario 出现。

Component	Formula example
Direct loss	number of affected events x average direct loss x recovery adjustment
Review cost	reopened cases x hours per case x fully loaded hourly cost
Remediation cost	affected customers x notification / correction / rework cost
Opportunity cost	wrongly rejected customers x conversion loss x expected margin
Control friction	extra reviews x review cost + additional latency impact
Resilience cost	outage hours x manual fallback cost x business criticality multiplier

Percentile	用法
P50	正常预算和 backlog impact
P90	risk committee 和 release gate
P95	executive release decision / stress narrative
P99	resilience planning or capital-style sensitivity, not daily prioritization

3.5 Control Effectiveness

控制有效性要说明作用机制。不要只写“降低风险 70%”。

Mechanism	示例	量化方式
Frequency reduction	RAG citation checker blocks unsupported regulated answers	high-risk unsupported output rate from 0.15% to 0.06%
Severity reduction	human approval prevents final customer communication	average severity drops because error stays internal
Detection improvement	QA sample finds drift within 48h instead of 30 days	loss duration reduced
Recovery improvement	trace evidence accelerates case reconstruction	investigation hours reduced
Exposure reduction	product scope excludes product-specific advice	exposed high-risk queries reduced
Resilience improvement	vendor failover preserves critical workflow	outage impact hours reduced

3.6 Control ROI and Risk-Adjusted Net Value

基础口径：

gross_expected_loss = exposure x frequency x average_severity
residual_expected_loss = exposure x residual_frequency x residual_average_severity
loss_reduction = gross_expected_loss - residual_expected_loss
control_roi = (loss_reduction - annual_control_cost) / annual_control_cost

更适合架构投资的口径：

risk_adjusted_net_value =
  business_benefit
  - residual_expected_loss
  - control_cost
  - control_friction_cost

Tail-aware narrative:

The control does not only reduce expected loss.
It compresses the P95 tail from a management-action event to a contained operating event.

3.7 Management Action Lifecycle

Residual risk 不能停留在模型表格里。每个 material scenario 都应进入明确动作：

Action	适用条件	设计含义
Reduce	残余风险高于阈值，控制 ROI 明显	加控制、调 scope、提升 eval、改 workflow
Avoid	tail loss 不可接受或不符合 intended use	取消自动化、禁止某类 intent、移出 release
Transfer	风险可通过保险、合同、vendor obligation 或服务协议部分转移	不能替代内部控制，只能改变损失承担方式
Accept with expiry	残余风险低于阈值或短期可接受	指定 owner、review date、leading indicator 和退出条件
Defer scale	pilot 可控但证据成熟度不足	继续 shadow mode、增加样本、限制流量

3.8 Prioritization Matrix

Control	Annual cost	Expected loss reduction	Tail loss reduction	Friction	Priority
RAG citation enforcement	180k	420k-720k	medium	low	high
Agent tool gateway	350k	600k-1.4m	high	medium	high
Double human review for all cases	1.2m	700k-1.1m	high	high	selective
Prompt-only safety instruction	40k	50k-120k	low	low	supporting
Vendor redundancy	500k	250k-600k	high for outage	low	critical-service only

4. 证据与控制

4.1 Metric Contract

Metric	Definition	Evidence source	Decision use
Gross scenario exposure	number of AI-influenced events in scope	workflow telemetry, gateway logs, case system	denominator for loss estimate
Defect frequency	scenario-specific failure rate before control	eval, red-team, shadow mode, QA	gross loss and control target
Control coverage	share of exposed events where control operated	policy engine logs, tool gateway logs, workflow approval	control design and operating check
Control effectiveness	observed reduction in frequency / severity / detection delay	A/B, shadow comparison, before-after, expert estimate	residual loss and ROI
Residual expected loss	expected loss after controls	quantification model + evidence packet	release / scale / fund decision
Tail loss	P95/P99 scenario severity	stress test, incident simulation, expert estimate	high-impact decision and resilience planning
Control friction cost	cost introduced by the control	AHT, review volume, latency, abandonment, infra cost	net value calculation
Loss avoided	gross loss minus residual loss	quantification model	ROI and prioritization

4.2 Control Types by Loss Mechanism

Control type	Reduces frequency	Reduces severity	Reduces detection delay	Reduces recovery cost
RAG source gating	yes	partial	partial	yes
Citation verification	yes	partial	yes	yes
Agent permission gateway	yes	yes	yes	yes
Human approval	partial	yes	partial	partial
Eval gate	yes	partial	yes	partial
Trace and evidence capture	no direct	partial	yes	yes
Vendor failover	no	yes	yes	yes
Product scope exclusion	yes	yes	yes	yes

4.3 Evidence Packet

Evidence	Required fields
Scenario register	scenario id、owner、exposure、trigger、loss event、affected workflow
Frequency evidence	sample design、period、population、defect count、confidence、reviewer calibration
Severity evidence	loss components、unit cost source、finance owner、range rationale
Control evidence	design spec、control owner、event field、test result、coverage
Effectiveness evidence	pre/post comparison、A/B or shadow result、assumptions、limitations
Residual risk statement	residual range、threshold status、owner、decision、expiry
ROI sheet	control cost、friction cost、loss reduction、net value、sensitivity
Release decision record	selected architecture、alternatives、tradeoffs、review date

4.4 Architecture Mapping

Architecture area	Quantification focus	Typical controls	Decision tradeoff
RAG	unsupported claim frequency, stale source severity, source coverage	approved corpus, citation verification, freshness SLO, answer abstention	better grounding vs higher latency / refusal
Agent	unauthorized action severity, tool misuse frequency, reversibility	permission gateway, scoped tokens, approval workflow, idempotent tools, action ledger	autonomy value vs tail loss compression
Copilot	human overreliance, draft reuse, review cost	UI confidence cues, mandatory review for high-risk intent, edit diff, role-based guidance	productivity vs review burden
Eval	scenario frequency estimate, control effectiveness evidence	scenario eval set, red-team suite, segment tests, regression gate	eval coverage vs release speed
Governance	residual threshold, acceptance owner, action tracking	scenario register, residual risk memo, release decision record, evidence packet, management action pack	decision clarity vs governance overhead

4.5 Focused Examples

RAG scenario: contact center copilot gives wrong fee waiver policy.
Control: citation-required answer mode for fee / dispute / account closure intents.
Quant effect: unsupported claim rate drops from 1.8% to 0.4%; AHT increases by 9 seconds.
Decision: apply control to regulated intents only, not all FAQ intents.

Agent scenario: agent creates or closes a customer case incorrectly.
Control: tool gateway requiring human approval token for write actions.
Quant effect: severity distribution changes because errors become draft-only.
Decision: allow read + draft autonomy, require approval for writes until residual P95 falls below threshold.

Eval scenario: AML copilot misses new mule-account typology.
Control: typology-specific regression suite and monthly red-team refresh.
Quant effect: confidence in frequency estimate improves; release decision moves from "unknown" to "controlled pilot".
Decision: fund eval coverage before adding more model capacity.

5. 金融零售场景

5.1 Model Hallucination in Regulated Advice

Dimension	Quantification design
Scenario	Wealth or lending assistant generates unsupported product-specific advice that an employee reuses.
Exposure	advisor responses or policy-assistant answers involving regulated topics.
Gross frequency	unsupported regulated recommendation rate from eval + shadow mode.
Severity	case review, correction outreach, complaint handling, advisor retraining, potential revenue loss.
Controls	approved corpus manifest, citation support gate, recommendation phrase classifier, licensed advisor handoff, blocked generation for product-specific advice.
Residual narrative	residual risk is acceptable only if AI cannot directly send customer communication and all product-specific claims show source support.
Tradeoff	Stricter refusal reduces loss frequency but may lower adoption; route high-risk prompts to curated workflow instead of generic chat.

Example range:

Item	Conservative	Base	Stress
Monthly exposure	20,000	35,000	50,000
Gross defect frequency	0.06%	0.12%	0.25%
Avg severity	1,200	2,500	8,000
Gross monthly expected loss	14,400	105,000	1,000,000
Residual after controls	5,000-18,000	32,000-58,000	220,000-420,000

5.2 AML Missed Escalation

Dimension	Quantification design
Scenario	AML copilot summary or prioritization misses a suspicious pattern, causing delayed escalation.
Exposure	alerts where AI summary, typology suggestion or priority score influences analyst attention.
Gross frequency	missed red-flag rate by typology from shadow comparison and QA sample.
Severity	reopened investigation, suspicious activity escalation delay, QA remediation, analyst backlog, regulatory exam support.
Controls	typology-specific eval set, recall floor for high-risk patterns, mandatory red-flag checklist, analyst override reason, no auto-close, scenario sampling.
Residual narrative	expected loss may be moderate, but tail scenario is high when missed escalation clusters by typology or policy drift.
Tradeoff	Optimizing precision reduces false positives but may increase tail risk; architecture should preserve recall for high-risk typologies and manage review load elsewhere.

Decision rule:

If high-risk typology recall falls below release floor or confidence interval is wide,
do not expand automation.
Use AI for evidence assembly, not case prioritization, until recall evidence matures.

5.3 Payment Fraud False Negatives

Dimension	Quantification design
Scenario	AI fraud triage classifies risky transactions as low priority, delaying intervention.
Exposure	transactions routed through AI triage or risk explanation workflow.
Gross frequency	false negative rate by segment, channel, merchant category and new fraud pattern.
Severity	fraud amount, recovery rate, dispute operations, customer contact, downstream losses.
Controls	threshold by risk segment, champion/challenger model, rules fallback, velocity checks, risk-based human queue, post-event learning loop.
Residual narrative	residual risk must be compared against false-positive friction and customer abandonment, not optimized in isolation.
Tradeoff	Lower false negatives can increase false positives; control ROI should include saved fraud losses and additional review / customer friction cost.

Loss formula:

net_loss_reduction =
  avoided_fraud_loss
  - incremental_false_positive_review_cost
  - customer_friction_cost
  - engineering_and_runtime_cost

5.4 KYC Wrongful Rejection

Dimension	Quantification design
Scenario	AI document / identity / risk assistant contributes to rejecting legitimate applicants.
Exposure	applications where AI recommendation influences decision or manual queue priority.
Gross frequency	wrongful rejection rate from appeal uphold, manual audit and shadow adjudication.
Severity	lost funded accounts, rework, support contacts, appeal handling, reputational friction.
Controls	low-confidence routing, segment fairness test, reason-code consistency, second review for thin-file / document-quality cases, appeal evidence capture.
Residual narrative	expected loss includes lost conversion, while tail risk comes from concentrated rejection in a protected or vulnerable segment.
Tradeoff	Faster onboarding may increase false rejects if quality gates are too aggressive; product architecture should separate fraud risk escalation from final rejection.

5.5 Contact Center Misinformation

Dimension	Quantification design
Scenario	Customer service copilot provides incorrect fee, dispute, account closure, promotion or policy information.
Exposure	customer-visible or agent-assisted answers on regulated or contractual topics.
Gross frequency	unsupported claim rate, stale-source answer rate, agent acceptance rate.
Severity	recontact cost, complaint handling, fee corrections, goodwill credit, QA remediation.
Controls	approved-source RAG, citation required for policy claims, answer templates for regulated topics, escalation intents, agent confirmation UX.
Residual narrative	residual expected loss can be low, but high volume makes small defect rates material.
Tradeoff	Full blocking may hurt first-contact resolution; better architecture uses intent risk tiering and source-specific answer modes.

5.6 Vendor Outage

Dimension	Quantification design
Scenario	Foundation model, embedding, vector database or AI gateway vendor outage degrades a critical service.
Exposure	workflows dependent on vendor at runtime.
Gross frequency	vendor SLA, historical outage, shared dependency analysis, internal failover test.
Severity	manual fallback cost, backlog, SLA breach, customer delay, lost productivity.
Controls	graceful degradation, cached responses for stable policy, model fallback, queue prioritization, manual runbook, vendor concentration limit.
Residual narrative	expected loss may not justify full redundancy for low-criticality use cases, but P95 outage cost may justify redundancy for AML, fraud or contact center peak periods.
Tradeoff	Multi-vendor architecture adds eval, routing, security and consistency cost; use risk-adjusted criticality rather than blanket redundancy.

6. 反模式与修正

Anti-pattern	Why it fails	Better design
High / Medium / Low only	不能比较投资优先级，也不能解释为什么花这笔钱	用 expected loss range + tail scenario + confidence
False precision	写出 123,456.78 美元让人误以为精确	用 range、percentile、assumption quality 和 sensitivity
Model metric as risk metric	accuracy / F1 不等于业务损失	把 model error 映射到 loss event 和 workflow exposure
Control theater	增加 guardrail 但不估计有效性	对每个 control 写明降低频率、严重度、检测或恢复的机制
Ignoring friction cost	控制看似高 ROI，实际拖慢流程、增加放弃率	把 review load、latency、false positive 和 adoption impact 计入 net value
Average-only thinking	expected loss 低，tail loss 高	同时看 P50、expected、P95 和 stress narrative
Over-general scenario	“AI 给错答案”太宽	按业务事件、渠道、客户群、流程阶段拆场景
No confidence rating	所有假设看起来同等可靠	标注 evidence maturity: expert / eval / shadow / production
Control dependency blindness	两个控制依赖同一日志或供应商	建 control dependency map 和 common-mode failure scenario
ROI without action	算完数字但不上 roadmap / release decision	将结果进入 funding gate、release gate 和 management action

7. 最终心智模型

AI risk quantification 的核心不是让风险看起来“科学”，而是让决策假设显性化，让不同控制、不同场景和不同产品范围可以在同一语言下比较。

成熟表达不是：

这个 AI 风险可控。

而是：

在这些 assumptions 下，scenario exposure、defect frequency、severity range、
control coverage 和 residual P95 都低于当前阈值。
如果 leading indicators 突破，管理动作是 reduce scope、add control、
accept with expiry、defer scale 或 stop。

最终判断标准：

量化单位必须是 business loss scenario，不是抽象 model failure。
频率和严重度必须有来源、区间、置信度和复核日期。
控制必须说明降低频率、严重度、检测延迟、恢复成本或暴露面的机制。
ROI 必须同时看 loss reduction、control cost、friction cost 和 tail compression。
RAG、Agent、Copilot、Eval、Governance 和 Vendor resilience 可以用同一套 scenario-loss language 比较投资价值。
残余风险必须进入管理动作，而不是停留在表格里。

当这套结构建立起来，AI 风险管理就不再是上线前的合规附件，而是产品范围、架构路线图、控制投资和运营承诺的共同决策系统。

SOTA 状态标注 (2026-07-01)

本篇属于第二、三遍深读池（参考架构/深读笔记），未列入 12 周主线必读。时效基线为写作时点；引用前请按 CLAUDE.md 全局时效性硬规则复查最新进展。模块级 SOTA 对照见 docs/AI_SYSTEMATIC_LEARNING_ROADMAP_2026.md 各周「2026 SOTA 对照」行与文末「SOTA 检查」。