AI 扩展计划 / Playbooks
AI Architecture Review Gate Checklists
AI 架构评审不能只看模型、API 和一张系统图。它要证明一个概率型能力可以被插入业务流程，并且在数据、知识、权限、工具、评估、人类控制、运维、成本和价值上保持可管理。本文把 AI 架构评审拆成从 intake 到季度复审的门禁体系，适合金融零售企业中的客服、信贷、支付、反欺诈、反洗钱、财富合规和内部知识工作场景。
432 行AI_ARCHITECTURE_REVIEW_GATE_CHECKLISTS.md
AI Architecture Review Gate Checklists
AI 架构评审不能只看模型、API 和一张系统图。它要证明一个概率型能力可以被插入业务流程，并且在数据、知识、权限、工具、评估、人类控制、运维、成本和价值上保持可管理。本文把 AI 架构评审拆成从 intake 到季度复审的门禁体系，适合金融零售企业中的客服、信贷、支付、反欺诈、反洗钱、财富合规和内部知识工作场景。
1. Source Anchors
| Anchor | Link | 在评审中的用法 |
|---|---|---|
| NIST AI RMF | https://www.nist.gov/itl/ai-risk-management-framework | 用 Govern / Map / Measure / Manage 组织风险识别、评估、监控和治理。 |
| NIST AI RMF Generative AI Profile | https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence | 用于 hallucination、data leakage、misuse、synthetic content、evaluation 等 GenAI 风险。 |
| EU AI Act | https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng | 用 risk-based lens 识别高风险场景、透明度、人类监督和技术文档需求。 |
| ISO/IEC 42001 | https://www.iso.org/standard/81230.html | 用 AI management system 思路评审责任、生命周期和持续改进。 |
| OWASP LLM Top 10 | https://owasp.org/www-project-top-10-for-large-language-model-applications/ | 评审提示注入、敏感信息披露、过度代理、不安全输出处理等。 |
| TOGAF | https://www.opengroup.org/togaf | 用架构治理、能力规划、路线图和 architecture board 语言组织评审。 |
| C4 Model | https://c4model.com/ | 用 context、container、component 让系统边界可讨论。 |
| BIAN | https://bian.org/deliverables/service-landscape/ | 用银行服务域评审金融零售能力边界和系统集成。 |
2. 为什么 AI 架构评审必须分 Gate
传统架构评审关注服务边界、数据库、API、安全、部署、可用性和成本。AI 系统还要评审：
| AI-specific concern | 评审问题 |
|---|---|
| Business fit | 业务问题是否真的适合 AI，还是流程、规则、报表或培训更合适？ |
| Workflow insertion | AI 在 read、summarize、recommend、draft、decide、act 哪个环节介入？ |
| Data and knowledge | 模型能否访问正确、合法、当前有效、按权限可见的事实？ |
| Grounding | 输出是否能引用证据，引用是否真的支持结论？ |
| Tool authority | 工具调用是否可能越权，是否有审批、幂等和回滚？ |
| Evaluation | eval 是否覆盖真实失败模式，而不是只看平均分？ |
| Change control | prompt、model、index、policy、tool 变更是否触发回归？ |
| Human oversight | 高风险输出是否进入清晰的人类复核和责任链？ |
| Adoption | 用户是否真的改变工作方式，还是只打开过工具？ |
| Operations | 上线后谁维护知识库、eval、policy、incident 和成本？ |
门禁把风险前移：
Idea
-> Intake Gate
-> Business Fit Gate
-> Data and Knowledge Gate
-> AI Pattern Gate
-> Architecture Gate
-> Eval and Risk Gate
-> Pilot Gate
-> Release Gate
-> Scale Gate
-> Quarterly Review
3. Gate 总览
| Gate | 主要问题 | 决策结果 | 核心证据 |
|---|---|---|---|
| G0 Intake | 这个 use case 是否值得进入 discovery | accept / park / reject | opportunity brief |
| G1 Business Fit | 业务问题、用户、流程、价值是否清楚 | continue / refine / stop | opportunity canvas、workflow、stakeholder map |
| G2 Data and Knowledge | 数据、知识、权限、质量是否足够 | ready / conditional / blocked | source inventory、data readiness pack |
| G3 AI Pattern | 应该用 RAG、workflow、agent、fine-tuning、rules 还是 vendor | architecture direction | ADR、decision matrix |
| G4 Architecture | 系统边界、集成、控制、可观测是否合理 | approve / revise | C4、data flow、sequence、threat model |
| G5 Eval and Risk | 需求是否可测，风险是否可控 | approve pilot / revise | requirements-to-eval、control pack |
| G6 Pilot | 是否可以小范围试点 | launch pilot / no-go | pilot plan、success criteria、rollback |
| G7 Release | 是否可以生产发布 | release / limited release / no-go | eval report、runbook、RACI |
| G8 Scale | 是否可以扩展到更多用户或流程 | scale / hold / rework | adoption dashboard、ROI、risk review |
| G9 Quarterly Review | 这个 AI capability 是否仍有效 | continue / refresh / retire | quality trend、cost、risk、feedback |
4. G0 Intake Gate
Intake gate 用于阻止“领导想看 demo”直接变成项目。
必须回答：
| Question | Evidence |
|---|---|
| 谁提出需求，谁拥有业务结果 | sponsor、business owner |
| 当前痛点是什么 | baseline signal、volume、quality、risk evidence |
| 影响哪些用户和流程 | workflow and user segments |
| 为什么不用规则、报表、流程优化或培训 | no-AI alternative |
| 是否涉及客户权益、信贷、隐私、合规或高风险决策 | initial risk tier |
| 是否有 owner 愿意参与 discovery | named owner |
Decision:
| Decision | 条件 |
|---|---|
| Accept discovery | 有明确业务问题、owner、初步价值和风险边界 |
| Park | 价值不清但方向可能有潜力 |
| Reject | 无 owner、风险过高、明显可用非 AI 方案解决 |
Red flags:
| Red flag | Why it matters |
|---|---|
| 只说“做 AI”，不说业务问题 | 无法定义 outcome 和评估 |
| 涉及高风险流程却先做公开 demo | 控制、权限和客户影响失控 |
| 没有流程 owner | 后续 adoption 和价值无法兑现 |
| 成功标准是“模型能回答” | 不是生产标准 |
5. G1 Business Fit Gate
Business fit gate 评审 AI 是否解决真实流程问题。
Checklist:
| Check | Explanation |
|---|---|
| 有 use case canvas | 定义问题、流程、用户、价值和风险 |
| 有 AS-IS / TO-BE workflow | 明确 AI 进入哪一步以及例外路径 |
| 有 pain metrics | cycle time、touch time、error rate、rework、backlog、cost |
| 有 stakeholder map | 用户、经理、风险、合规、运营、技术、客户影响 |
| 区分 AI fit 与 no-AI boundary | 防止技术冲动 |
| 定义 first insertion point | read、summarize、recommend、draft、decide、act |
| 识别高风险 decision boundary | 明确禁止自动化的决定 |
| 有 business owner 和 operating owner | 价值和运行责任明确 |
金融零售判断：
| Use case | Good fit | Bad fit |
|---|---|---|
| AML copilot | evidence aggregation、red-flag checklist、case narrative draft | automatic filing decision |
| Customer service RAG | approved knowledge retrieval、answer draft、citation | unauthorized fee waiver commitment |
| Lending assistant | memo drafting、policy citation、missing document checklist | model-owned credit decision |
6. G2 Data and Knowledge Gate
AI 系统能否上线，很大程度取决于数据和知识是否可用、合法、可追溯、按权限可见。
Checklist:
| Check | Why |
|---|---|
| source of truth 列清楚 | 防止向量库被误当权威来源 |
| 数据分类完成 | public、internal、confidential、PII、financial、special category |
| data owner 和 knowledge owner 明确 | 支持变更、质量和审批 |
| lineage 和 versioning 明确 | 支持复现和审计 |
| access control 和 entitlement filter | 防止越权检索 |
| retention and logging policy | 控制 prompt、trace、embedding、summary 的记录风险 |
| stale knowledge detection | 防止过期政策输出 |
| prompt/log forbidden data | 防止敏感数据进入不可控日志 |
Required assets:
| Asset | Content |
|---|---|
| Data Readiness Pack | sources、quality、access、lineage、limits |
| Knowledge Source Inventory | owner、version、effective date、authority |
| Permission Model | role、case、source、field-level rules |
| Data Flow Diagram | data movement、storage、vendor boundary |
| Freshness Plan | update cadence、re-index、regression trigger |
Blocked conditions:
| Condition | Gate result |
|---|---|
| 权限不清且可能暴露客户数据 | blocked |
| 来源版本和生效日期不可追踪 | blocked or limited |
| 供应商数据使用条款不清 | blocked until contract review |
| 历史标签存在明显偏差但未评估 | conditional or blocked |
7. G3 AI Pattern Gate
所有需求都用 chatbot、RAG 或 agent 是常见架构失败。Pattern gate 要先选结构，再谈模型。
| Need | Prefer | Avoid |
|---|---|---|
| 最新政策、引用、权限 | RAG + citation + metadata | fine-tuning only |
| 稳定分类、标签、路由 | classifier、rules、small model | free-form generation |
| 多步系统操作 | workflow orchestration + bounded agent | autonomous open agent |
| 高风险建议 | decision support + human review | direct automation |
| 文档抽取 | OCR/document AI + validation | pure chat prompt |
| 风格或格式适配 | prompt baseline then tuning if justified | full training first |
| 严格计算 | deterministic tool or rules | model arithmetic |
| 低风险 FAQ | RAG/cache/fast model | expensive reasoning model |
Required decision records:
| ADR | Must explain |
|---|---|
| RAG vs long context vs tuning | 知识变化、上下文、成本和证据要求 |
| Workflow vs agent | 自由度、工具、风险、可测性 |
| Buy vs build vs hybrid | 供应商边界、数据、控制、成本 |
| Model/provider choice | 质量、延迟、成本、区域、数据处理 |
| Tool access boundary | allowlist、approval、rollback |
| Human review boundary | 哪些场景必须人审 |
| Eval strategy | 数据集、方法、阈值和 owner |
8. G4 Architecture Gate
Architecture gate 要证明系统可落地、可观测、可治理、可回滚。
Required diagrams:
| Diagram | Must answer |
|---|---|
| C4 Context | 系统和外部 actor、数据源、供应商边界 |
| C4 Container | 应用、orchestrator、model gateway、retrieval、tool gateway、audit |
| Data Flow | 数据进入、处理、存储、日志、权限、跨境和保留 |
| Sequence | 单次请求如何检索、调用模型、调用工具、校验、记录 |
| Control Architecture | 高风险控制、人审、stop switch、fallback |
| Eval Architecture | offline eval、online checks、trace sampling、failed case 回流 |
| Cost / Latency Model | 路由、缓存、工具、p95、预算 |
Architecture quality bar:
| Question | Expected answer |
|---|---|
| 数据从哪里来 | source inventory and lineage |
| 证据如何进入 prompt | retrieval filters and citation |
| 模型如何被调用 | model gateway, route, version |
| 工具如何被授权 | tool gateway, policy, approval |
| 输出如何验证 | schema, policy, citation, safety checks |
| 高风险如何升级 | risk trigger and human review |
| 日志和审计在哪里 | audit writer and evidence store |
| 线上如何监控 | telemetry, dashboard, alert |
| 谁能回滚 | release owner and rollback path |
Red flags:
| Red flag | Why |
|---|---|
| UI 直接调用模型 API | 权限、日志和密钥控制不足 |
| 没有 tool permission | agent 越权风险 |
| 没有 audit log | 无法复盘、审计和处理投诉 |
| 没有 eval / monitoring | 质量不可控 |
| 所有错误都靠 prompt 防止 | 控制层过薄 |
| 没有成本和延迟预算 | scale 后不可持续 |
9. G5 Eval and Risk Gate
Eval and risk gate 把需求转成可执行的上线证据。
Checklist:
| Check | Explanation |
|---|---|
| 有 Requirements-to-Eval Matrix | 关键需求都映射到评估 |
| 每条关键需求有 eval method | deterministic、judge、expert、red-team 或 production sample |
| 有 golden dataset | 包含 common、edge、missing-data、high-risk |
| 有 severity levels | critical failure 独立阻断 |
| 有 release thresholds | 不只看平均分 |
| 有 red-team backlog | 覆盖 OWASP LLM 风险 |
| 有 human oversight design | review criteria、日志、override reason |
| 有 incident response path | severity、triage、stop、customer impact |
Risk-tiered gate:
| Risk tier | Example | Gate |
|---|---|---|
| Low | internal product FAQ | automated eval + sample review |
| Medium | customer service draft | automated eval + QA sampling + policy guardrail |
| High | AML, lending, wealth compliance | expert review + human oversight + audit + strict release gate |
| Critical | autonomous customer-impacting decision | generally no-go unless deterministic, authorized, governed |
Minimum release thresholds:
| Gate | Suggested threshold |
|---|---|
| Critical unsafe output | 0 |
| Unauthorized action | 0 |
| Unsupported factual claim in high-risk output | 0 |
| Citation coverage for policy answers | defined by scenario, usually high |
| Expert acceptance | threshold by use case |
| Regression vs previous version | no critical regression |
| Cost / latency | within workflow SLA |
10. G6 Pilot Gate
Pilot gate 确认小范围试点可学习、可停止、可回滚。
| Area | Required decision |
|---|---|
| Scope | 用户、流程、数据、风险等级、流量比例 |
| Metrics | quality、safety、workflow、adoption、cost、risk |
| Stop rules | 触发暂停或回滚的质量、风险、成本、投诉条件 |
| Support | office hour、L1/L2/L3、FAQ、known limitations |
| Training | role-based authorization and scenario training |
| Monitoring | daily/weekly review cadence |
| Feedback | structured reason codes and issue routing |
Pilot 不应只问“用户喜不喜欢”。它要回答：真实流程是否改变，质量是否可控，成本是否可接受，复核负担是否吞掉价值，用户是否知道何时信任和何时升级。
11. G7 Release Gate
Required evidence pack:
| Evidence | Purpose |
|---|---|
| Final architecture diagrams | 系统边界和控制清楚 |
| ADR set | 关键架构选择可追溯 |
| Data readiness sign-off | 数据和知识可用且受控 |
| Eval report | 质量和风险可证明 |
| Risk/control pack | 人审、权限、incident、residual risk |
| Security/privacy review | 技术和数据风险可接受 |
| Operating model / RACI | 上线后责任清楚 |
| Incident runbook | 出错后可处理 |
| Release notes | 变更范围可见 |
| User enablement | 用户知道边界和反馈路径 |
| Monitoring dashboard | 生产行为可观察 |
| Rollback plan | 能停止、降级或回滚 |
Release decision options:
| Decision | Meaning |
|---|---|
| Full release | 符合质量、风险、运营和 adoption 条件 |
| Limited release | 限定用户、流程、风险等级或输出类型 |
| Shadow mode | 只观察，不影响真实流程 |
| Internal-only | 仅内部辅助，不面向客户 |
| No-go | 关键风险或证据不足 |
Final release questions:
- 如果模型今天出错，谁第一个知道？
- 如果供应商明天更新模型，谁跑回归？
- 如果知识库过期，谁负责修？
- 如果用户绕过流程，谁处理？
- 如果输出导致客户投诉，证据链在哪里？
- 如果成本翻倍，谁有权降级模型或关停功能？
12. G8 Scale Gate and G9 Quarterly Review
Scale gate 判断是否值得扩展到更多用户、产品线、区域或风险等级。
| Scale evidence | Question |
|---|---|
| Business metric | pilot 是否达到目标 |
| Quality metric | 输出质量是否稳定 |
| Risk trend | 是否无 critical incident，residual risk 是否可接受 |
| Adoption | 目标用户是否持续使用并形成新工作方式 |
| Operating model | manager、QA、risk、support 是否承受得住 |
| Cost forecast | 单位成本是否可控 |
| Scope change | 新区域、产品和政策差异是否评估 |
Quarterly review 判断 capability 是否仍有效：
| Review area | Evidence |
|---|---|
| Value | business outcome, adoption, finance view |
| Quality | eval regression, QA, failed traces |
| Knowledge | source freshness, policy changes |
| Vendor | model/provider changes |
| Cost | run-rate, unit cost, variance |
| Risk | incidents, complaints, control gaps |
| Lifecycle | continue, refresh, restrict, retire |
13. Financial Retail Gate Profiles
| Use case | Gate emphasis | No-go triggers |
|---|---|---|
| AML copilot | human oversight、evidence citation、audit trail、typology coverage、prompt injection | unsupported claims、final filing suggestion、missing audit log、hidden evidence |
| KYC remediation | data quality、customer communication、jurisdiction policy、source-of-truth update | unauthorized document request、golden source update without approval、misstated legal requirement |
| Customer service RAG | knowledge version、citation、no unauthorized promise、escalation、QA feedback | answers without sources、outdated policy、fee waiver commitment |
| Payments exception agent | tool permission、idempotency、approval before action、reconciliation | repair without approval、unlogged tool action、duplicate or irreversible action |
| Lending assistant | deterministic calculations、fair lending、reason codes、human decision owner | model-owned credit decision、protected/proxy variables unreviewed、unsupported denial explanation |
14. Review Board Pack
Before review, prepare:
| Section | Content |
|---|---|
| Executive summary | decision requested, use case, risk tier |
| Scope and no-AI option | what is in scope, what is excluded |
| Process map | AS-IS, TO-BE, exception, control |
| Architecture | context, container, data flow, sequence |
| Pattern ADR | RAG, workflow, agent, rules, vendor choices |
| Requirements-to-Eval | matrix, dataset, thresholds, severity |
| Data readiness | sources, permissions, freshness |
| Control pack | tool boundary, human oversight, incident |
| Threat model | security and misuse risks |
| Cost / latency model | budget and SLA |
| Operating model | owner, support, runbook |
| Pilot / release plan | scope, stop rule, rollback |
| Open risks | requested decisions and conditions |
Review meeting should end with a decision, not a discussion summary: approve, approve with conditions, revise, park, or reject.
15. Ready and Done Definitions
| Stage | Ready means |
|---|---|
| Discovery | business owner exists, pain has evidence, risk tier roughly known, no-AI option considered |
| Architecture | workflow documented, data sources identified, AI boundaries defined, eval ideas exist |
| Pilot | architecture approved, data readiness acceptable, eval and control drafted, pilot stop rules defined |
| Production | eval gate passed, critical risks controlled, runbook tested, dashboard live, rollback ready |
| Scale | pilot value proven, quality stable, adoption real, cost understood, risk accepted, new scope evaluated |
16. Operating Principle
AI architecture review is the discipline of proving that a probabilistic capability can enter a business workflow without losing control of quality, risk, cost, accountability and user trust.
This is the business decision.
This is the system boundary.
This is the evidence.
This is how we evaluate it.
This is how we control it.
This is who owns it.
This is how we stop it if it fails.