AI 风险量化架构：Scenario Loss / Control ROI / Residual Risk

Date: 2026-06-30 Status: evergreen Audience: experienced CBAP / financial retail PM / product architect / solution architect / AI governance lead Output: 一套把 AI risk scenario 转成 expected loss range、residual risk、control ROI、investment priority 和 management action 的决策架构。

1. Why Risk Quantification Matters for AI Product / Architecture

重要边界:

• 本文不是法律、合规、审计、精算、资本计量或监管意见。
• 本文不重复 risk appetite policy、board reporting、customer harm、incident liability 和 continuous control monitoring 的完整框架。
• 本文只解决一个高级问题: 如何把 AI 风险场景转成可比较、可投资、可挑战、可复核的经济决策语言。

AI 项目常见的风险讨论停在三个层次:

High / Medium / Low
严重 / 一般 / 可接受
加一个 guardrail / 加一个 HITL / 做一些监控

这对上线评审有帮助, 但不足以支持产品和架构投资决策。CRO、CFO、CTO、产品负责人真正需要回答:

• 这个 AI 场景的损失分布是什么, 不是单点风险等级是什么。
• 频率和严重度假设来自哪里, 是历史事件、专家估计、eval 样本、红队结果还是 vendor SLA。
• 某个控制能降低多少频率、严重度、检测延迟或恢复成本。
• 控制投入是否比残余损失下降更划算。
• 同样 100 万预算, 应该投 RAG grounding、agent permission gateway、human review capacity、vendor redundancy 还是 eval coverage。
• 哪些 residual risk 需要管理层接受, 哪些必须降低, 哪些应通过产品 scope 避免。

一句话:

AI Risk Quantification 是把“AI 可能出错”翻译成“哪些场景会以多大频率造成多大损失, 哪些控制用多少钱能降低多少损失, 现在是否值得上线、扩展、暂停或改架构”。

对 AI PM / BA / Architect 的价值:

---

2. Concept Diagram

flowchart LR
  A[AI Use Case Scope] --> B[Scenario Register]
  B --> C[Frequency Assumption]
  B --> D[Severity Assumption]
  C --> E[Gross Loss Range]
  D --> E
  E --> F[Control Portfolio]
  F --> G[Control Effectiveness Estimate]
  G --> H[Residual Loss Range]
  H --> I[Risk Appetite Threshold]
  H --> J[Control ROI]
  I --> K[Management Action]
  J --> L[Investment Prioritization]
  K --> M[ADR / Release Decision]
  L --> M
  M --> N[Evidence Packet]

ASCII 版:

scenario -> likelihood x severity -> gross loss range
gross loss range -> control effectiveness -> residual loss range
residual loss range -> appetite threshold -> management action
control cost vs loss reduction -> ROI -> architecture/product priority

关键转译:

AI failure mode
  -> business scenario
  -> loss event
  -> frequency distribution
  -> severity distribution
  -> gross expected loss / tail loss
  -> control effect
  -> residual expected loss / tail loss
  -> investment decision

---

3. Core Architecture Model

AI 风险量化不是一个 spreadsheet, 而是一套 decision architecture。它至少包含八个对象。

3.1 Loss Event Taxonomy

本文刻意不展开 customer harm 和 incident liability 的全套治理, 只把损失对象用于量化。

3.2 Four Loss Views

同一个 scenario 至少看四个视图:

3.3 Quantification Grain

AI 风险不是只按系统量化, 更要按 decision grain 量化:

---

4. Scenario Loss and Control ROI Method

4.1 Step-by-Step Method

1. Define business decision boundary.
2. Write scenario as loss event, not model error.
3. Estimate exposed population and event frequency.
4. Estimate severity distribution by loss component.
5. Calculate gross expected loss and tail loss.
6. Map candidate controls to loss mechanisms.
7. Estimate control effectiveness and control cost.
8. Calculate residual loss range and ROI.
9. Compare against threshold and product value.
10. Record decision, assumptions and evidence.

4.2 Scenario Statement Pattern

弱表达:

模型可能 hallucinate。

强表达:

In the regulated wealth education copilot, the model may produce an unsupported product-specific recommendation that a frontline advisor reuses in a customer conversation. Loss occurs when the customer acts on advice outside approved suitability workflow, requiring remediation review, complaint handling and potential client correction.

字段化:

4.3 Frequency Estimate

Frequency should be estimated as a range and tied to observable evidence.

频率表达示例:

4.4 Severity Estimate

Severity 不应只填一个 dollar amount。高级做法是拆成组件。

Severity range:

4.5 Control Effectiveness

控制有效性要说明作用机制。不要只写“降低风险 70%”。

4.6 Control ROI Formula

基础口径:

gross_expected_loss = exposure x frequency x average_severity
residual_expected_loss = exposure x residual_frequency x residual_average_severity
loss_reduction = gross_expected_loss - residual_expected_loss
control_roi = (loss_reduction - annual_control_cost) / annual_control_cost

更适合架构投资的口径:

risk_adjusted_net_value =
  business_benefit
  - residual_expected_loss
  - control_cost
  - control_friction_cost

Tail-aware narrative:

The control does not only reduce expected loss.
It compresses the P95 tail from a management-action event to a contained operating event.

4.7 Prioritization Matrix

---

5. Financial Retail Scenarios

5.1 Model Hallucination in Regulated Advice

Example range:

5.2 AML Missed Escalation

Decision rule:

If high-risk typology recall falls below release floor or confidence interval is wide,
do not expand automation.
Use AI for evidence assembly, not case prioritization, until recall evidence matures.

5.3 Payment Fraud False Negatives

Loss formula:

net_loss_reduction =
  avoided_fraud_loss
  - incremental_false_positive_review_cost
  - customer_friction_cost
  - engineering_and_runtime_cost

5.4 KYC Wrongful Rejection

5.5 Contact Center Misinformation

5.6 Vendor Outage

---

6. Metrics / Control / Evidence Model

6.1 Metric Contract

6.2 Control Types by Loss Mechanism

6.3 Evidence Packet

---

7. Anti-Patterns and Failure Modes

---

8. Architecture Mapping to RAG / Agent / Copilot / Eval / Governance

8.1 RAG Example

Risk scenario: contact center copilot gives wrong fee waiver policy.
Control candidate: citation-required answer mode for fee / dispute / account closure intents.
Quant effect: unsupported claim rate drops from 1.8% to 0.4%; AHT increases by 9 seconds.
Decision: apply control to regulated intents only, not all FAQ intents.

8.2 Agent Example

Risk scenario: agent creates or closes a customer case incorrectly.
Control candidate: tool gateway requiring human approval token for write actions.
Quant effect: severity distribution changes because errors become draft-only.
Decision: allow read + draft autonomy, require approval for writes until residual P95 falls below threshold.

8.3 Eval Example

Risk scenario: AML copilot misses new mule-account typology.
Control candidate: typology-specific regression suite and monthly red-team refresh.
Quant effect: confidence in frequency estimate improves; release decision moves from "unknown" to "controlled pilot".
Decision: fund eval coverage before adding more model capacity.

---

9. ADR Draft

---

10. Interview Answer

30 秒版本

我不会只用 High / Medium / Low 管 AI 风险。我的方法是把 AI failure mode 转成业务 loss scenario, 估算 exposure、frequency、severity, 得到 gross expected loss 和 P95 tail loss; 再评估每个控制能降低频率、严重度、检测延迟还是恢复成本, 计算 residual risk 和 control ROI。这样产品、架构、风险和财务可以共同决定: 上线、缩小范围、加控制、接受残余风险, 还是停止。

2 分钟版本

金融零售 AI 的难点不是知道“有风险”, 而是知道风险是否值得承担以及控制钱该花在哪里。我会先定义 use case boundary, 比如客服 RAG、AML copilot、KYC AI review、支付欺诈 triage 或 vendor outage。然后把模型错误写成业务场景: regulated advice hallucination、AML missed escalation、fraud false negative、KYC wrongful rejection、contact center misinformation。每个场景估算 exposure、频率范围和严重度组件, 形成 gross expected loss 和 tail loss。

接着我会把控制映射到损失机制。RAG citation gate 主要降低 unsupported answer frequency; agent tool gateway 降低 unauthorized action severity; human review 降低最终客户影响; eval suite 提高频率估计和变更门禁; vendor failover 降低 outage severity。最后比较 control cost、friction cost 和 loss reduction, 输出 residual risk、ROI、阈值状态和 management action。

这套方法的价值是把风险治理变成产品和架构决策: 哪些场景只能 assist-only, 哪些可以 automation, 哪些投资比模型升级更高 ROI, 哪些 residual risk 需要管理层限期接受或整改。

CTO 版本

CTO 关心的是工程投资是否压缩了真正的 tail risk, 而不是 dashboard 上多了几个绿色指标。我会要求高影响 AI 系统具备 scenario-loss architecture: 每个 release 都能说明 AI run 的业务 exposure、scenario defect rate、control coverage、residual P95 和 common-mode dependency。架构上优先做 deterministic control plane: model gateway、RAG source registry、tool permission gateway、trace evidence、eval regression 和 vendor degradation path。

投资排序不是“哪个控制看起来更合规”, 而是“哪个控制用最小 friction 压缩最多 expected loss 和 tail loss”。比如 regulated advice 场景, prompt instruction 的 ROI 很低, citation enforcement 和 product-scope routing 更有效; agent write-action 场景, tool gateway 比事后 review 更能降低严重度; vendor outage 场景, 不是所有 use case 都要 multi-vendor, 但 critical service 必须有 graceful degradation。我的 CTO-level takeaway 是: risk quantification should drive architecture runway.

---

11. 7-Day Practice Plan

评分标准:

---

12. Key Takeaways

1. AI risk quantification 的核心不是精确预测, 而是让决策假设显性化。
2. 量化单位必须是 business loss scenario, 不是抽象 model failure。
3. 控制 ROI 要同时看 loss reduction、control cost、friction cost 和 tail compression。
4. RAG、Agent、Copilot、Eval 和 Governance 的架构投资可以用同一套 scenario-loss language 比较。
5. 最成熟的表达不是“这个 AI 风险可控”, 而是“在这些 assumptions 下, residual expected loss 和 P95 tail loss 低于阈值; 如果指标突破, 管理层动作是 reduce scope / add control / accept with expiry / stop”。

---

Source Anchors