AI Requirements-to-Eval Cookbook

定位: 面向 AI BA / AI PM / AI Solutions Architect 的可测需求手册。 目标: 把“AI 应该准确、安全、专业、有用”改写成 eval contract、release gate、monitoring 和 incident loop。 使用方式: 每个 AI use case 至少填一张 Requirements-to-Eval Matrix, 并把关键失败模式放入 release gate。

---

Source Anchors

这些来源作为学习锚点, 不构成法律或合规意见。

---

1. 为什么 AI 需求不能只写 Acceptance Criteria

传统软件需求可以写:

• 点击提交后创建工单。
• 金额必须大于 0。
• 查询结果按时间倒序。

AI 需求常写:

• 回答要准确。
• 摘要要完整。
• 建议要安全。
• 语气要专业。
• 不能幻觉。
• 要遵守政策。

这些需求如果不转成 eval, 实际上不可验收。

AI requirement 的合格形式应该是:

Business requirement
-> expected AI behavior
-> unacceptable behavior
-> test data
-> evaluation method
-> threshold
-> severity
-> owner
-> release gate
-> monitoring signal
-> incident response

一句话:

AI requirements are not done until they are testable, risk-tiered, monitored and owned.

---

2. Requirements-to-Eval 主流程

flowchart TB
  B[Business outcome] --> W[Workflow point]
  W --> A[AI behavior]
  A --> F[Failure modes]
  F --> D[Data and evidence]
  D --> R[Rubric]
  R --> T[Test cases]
  T --> M[Metrics and threshold]
  M --> G[Release gate]
  G --> O[Production monitoring]
  O --> I[Incident and improvement loop]

Step 1: Business outcome

不要从模型能力开始, 从业务结果开始。

例子:

• AML: 降低 evidence gathering 时间, 提升 case narrative 完整性。
• KYC: 缩短 remediation cycle time, 降低重复联系客户。
• 客服: 提升 first-contact resolution, 降低错误政策回答。
• 支付: 缩短 exception resolution time, 避免错误修复动作。
• 信贷: 提升 memo 一致性, 不越过人工信贷决策边界。

Step 2: Workflow point

AI 插入流程的哪个点?

Step 3: AI behavior

把“好回答”拆成可评估行为:

• 是否引用正确证据。
• 是否覆盖必要字段。
• 是否说出缺失信息。
• 是否遵守输出格式。
• 是否触发人工复核。
• 是否避免越权承诺。
• 是否区分事实和推断。

Step 4: Failure modes

常见 AI failure modes:

Step 5: Data and evidence

每个 eval case 要包含:

• user input。
• workflow state。
• retrieved evidence。
• tool observations。
• expected behavior。
• unacceptable behavior。
• severity。
• reviewer role。

Step 6: Rubric

Rubric 要短、明确、可校准。

示例:

Step 7: Test cases

按场景类型设计:

• common cases。
• edge cases。
• missing-data cases。
• policy conflict cases。
• adversarial cases。
• high-risk cases。
• historical failure cases。

Step 8: Threshold

阈值不能只有平均分。高风险场景要设置 hard stop:

• critical violation = 0。
• unauthorized action = 0。
• unsupported high-risk claim = 0。
• regression critical failures = 0。
• expert review pass >= defined threshold。

Step 9: Release gate

Release gate 应输出:

• pass / conditional / fail。
• failed cases。
• severity。
• owner。
• mitigation。
• next review。

Step 10: Monitoring and incident loop

上线后继续看:

• user override。
• user complaint。
• expert QA defects。
• citation failure。
• unsafe output。
• latency/cost。
• adoption。
• drift。

---

3. Evaluation Methods

Deterministic checks

适合:

• JSON schema。
• forbidden phrase。
• citation exists。
• tool action allowed。
• required fields present。
• policy version current。

优点: 稳定、便宜、可重复。

限制: 不理解开放式质量。

LLM-as-Judge

适合:

• completeness。
• tone。
• explanation quality。
• policy compliance 初筛。
• groundedness 初筛。

必须控制:

• position bias。
• verbosity bias。
• judge version drift。
• human calibration。

Expert review

适合:

• AML typology。
• lending / fair lending。
• wealth suitability。
• regulatory impact。
• high severity failures。

限制:

• 成本高。
• 速度慢。
• 需要 calibration。

Shadow mode

适合:

• 高风险系统上线前。
• 与人工决策对照。
• 观察 false positive / false negative。

Production monitoring

适合:

• 持续质量管理。
• prompt/model/index 变更后观察。
• adoption and trust。

---

4. Severity Levels

---

5. Reusable Requirements-to-Eval Matrix

Minimum fields:

• requirement id。
• business owner。
• risk owner。
• data source。
• eval owner。
• release threshold。
• monitoring signal。

---

6. Bad Requirements -> Eval-Ready Requirements

---

7. Financial Retail Examples

AML Copilot

Release gate:

• critical unsupported claim = 0。
• final SAR decision suggestion = 0。
• evidence citation threshold met。
• expert QA accepts pilot sample。

KYC Remediation

Customer Service RAG

Payments Exception Agent

Lending Assistant

Wealth Compliance Guardrail

Regulatory Change Impact

---

8. Risk-Tiered Release Gates

---

9. Red-Team Cases

Include at least:

• Prompt injection in retrieved docs。
• Conflicting policy versions。
• Missing evidence。
• Unauthorized user。
• Sensitive data request。
• High-risk advice request。
• Tool unavailable。
• User asks to bypass approval。
• Old case with wrong historical label。
• Ambiguous customer complaint。

---

10. Ownership and RACI

---

11. Interview Talking Points

How do you make AI requirements testable?

30-second answer:

I convert each AI requirement into an eval contract: expected behavior, failure modes, test cases, rubric, threshold, severity, owner, release gate and monitoring signal. For financial services I separate deterministic checks, LLM judge and expert review, and I require zero critical failures for high-risk outputs.

What is the difference between acceptance criteria and eval?

Answer:

Acceptance criteria often describe what should happen. Eval defines how we will measure whether probabilistic behavior is good enough across common, edge, adversarial and high-risk cases, and how failures affect release.

How do you evaluate an AML Copilot?

Answer:

I would measure evidence recall, citation precision, red-flag coverage, missing evidence detection, unsupported claim rate, and whether it avoids final SAR decisions. High-risk samples require expert review and human approval.

---

12. Exercises

Exercise 1: Rewrite bad requirements

Rewrite:

• AI should be accurate。
• AI should help AML analysts。
• AI should answer customer questions。
• AI should recommend payment repair。
• AI should support lending decisions。

For each, define:

• expected behavior。
• unacceptable behavior。
• eval method。
• threshold。
• severity。

Exercise 2: Build 20-case golden set

For customer service RAG:

• 10 common cases。
• 3 missing evidence cases。
• 3 policy conflict cases。
• 2 prompt injection cases。
• 2 escalation cases。

Exercise 3: Release memo

Write:

Use case:
Eval result:
Critical failures:
Risk acceptance:
Release decision:
Conditions:
Owner:
Next review:

---

13. Connections

---

14. Final Rule

An AI requirement is not ready if it cannot answer:

What should happen?
What must never happen?
How will we test it?
What data proves it?
What threshold gates release?
Who owns failures?
How will production drift be detected?