AI 推理预算架构：Test-Time Compute / Verifier Cascade / Reasoning Budget

Date: 2026-06-30
Status: evergreen
Audience: experienced CBAP / financial retail PM / product architect / solution architect / AI governance lead
Output: advanced architecture note, ADR draft, interview answer, 7-day practice plan

---

Why reasoning budget matters for AI product/architecture

AI 产品上线后, 最贵的不是单次模型调用, 而是把所有问题都当成同一种问题处理。简单 FAQ、低风险摘要、高风险授信解释、AML case narrative、支付争议判断和监管投诉根因分析, 不应该共享同一条推理路径、同一延迟预算、同一审计证据、同一人工复核策略。

Reasoning budget 指的是系统在运行时愿意为一次任务投入多少 deliberation resource: tokens, samples, retrieval passes, tool calls, verifier checks, human review time, queue priority, and audit evidence capture。Test-time compute 不是单纯“让模型多想一会儿”, 而是产品和架构层面的动态资源分配机制。

对金融零售 AI 来说, 推理预算的价值有四层:

本笔记不重复 CoT、self-consistency、process supervision、LLM-as-judge 或 prompt optimization 的基础介绍。这里的关注点是: 企业如何把“推理能力”设计成可运营、可审计、可控成本的系统能力。

一句话:

Reasoning budget is the runtime policy that decides when an AI system should answer fast, deliberate deeper, ask for evidence, call tools, run verifiers, abstain, or escalate to a human.

---

Concept diagram

flowchart TB
  A[Business request] --> B[Risk and complexity classifier]
  B --> C{Budget tier}

  C -->|Tier 0 Fast path| F0[Direct response with schema checks]
  C -->|Tier 1 Evidence path| F1[RAG + grounded answer]
  C -->|Tier 2 Deliberation path| F2[Plan -> solve -> check loop]
  C -->|Tier 3 Controlled decision path| F3[Decompose + tools + verifier cascade + human gate]

  F0 --> V0[Safety and format check]
  F1 --> V1[Citation support verifier]
  F2 --> V2[Consistency, policy and calculation verifier]
  F3 --> V3[Independent verifier + rule engine + SME review]

  V0 --> D{Deliver, abstain, or escalate}
  V1 --> D
  V2 --> D
  V3 --> D

  D --> E[Customer or analyst output]
  D --> G[Runtime evidence packet]

  G --> H[Trace spans, source ids, policy versions]
  G --> I[Verifier results, risk score, abstention reason]
  G --> J[Human review action, final disposition]

架构图表达的是控制路径, 不是模型内部实现。企业不需要知道模型内部到底如何推理, 也不应该要求模型暴露 hidden chain-of-thought。企业能控制的是外部运行时: 是否检索证据、是否拆任务、是否调用工具、是否多次生成、是否运行 verifier、是否要求人工审批、是否记录证据。

---

Core architecture model

1. Request intake and risk classification

每次 AI run 先进入 request intake, 识别:

风险分类器不必一开始就复杂。成熟做法是把规则、用例台账和模型分类结合:

budget_tier = f(use_case, action_type, customer_impact, uncertainty, data_sensitivity, user_role, channel_slo)

2. Budget tiering

预算不是“越多越好”。高预算可能带来更长延迟、更高成本、更多表面合理化、更多隐私暴露面和更复杂的证据管理。关键是把预算和风险相称。

3. Decomposition and planner/solver/checker loop

复杂任务不要把所有要求塞进一个 prompt。推荐拆成三个角色, 但不要求一定是三个模型:

示例: 支付争议 reasoning。

Planner:
1. 识别 dispute reason code.
2. 检查交易状态、授权方式、3DS/AVS/CVV 结果、merchant evidence.
3. 检查客户争议窗口和法规/卡组织时限.
4. 判断是否需要临时贷记、补件或人工审查.

Solver:
根据检索到的交易数据和政策版本生成建议处置。

Checker:
验证时限、金额、原因码、客户通知模板、证据引用和禁用话术。

4. Abstention and escalation

推理预算体系必须承认“不能答”和“不能自动决定”是正常产出。

5. Hidden vs exposed rationale

企业系统应区分三类材料:

审计要的是可复现证据, 不是模型的完整思维过程。可复现证据包括: 输入摘要、检索来源、工具结果、规则版本、模型配置、输出哈希、verifier 分数、人工审批记录、最终业务处置。

---

Budgeting policies and gates

Policy 1: Risk-proportional compute

Policy 2: Budget cannot override authority

更多推理 token 不等于更多权限。系统即使很“自信”, 也不能绕过:

• 信贷拒绝、降额、冻结账户等 customer-impacting action 的授权流程。
• AML/SAR 的法定流程和合规复核。
• 支付争议的时限、通知和客户权利要求。
• 投诉处置的监管分类、根因编码和整改流程。

Policy 3: Budget escalation requires evidence

从 Tier 1 升到 Tier 2 或 Tier 3, 必须记录触发原因:

Policy 4: Stop conditions

系统需要明确何时停止继续消耗 test-time compute:

Policy 5: Budget ownership

---

Verifier cascade and evidence design

Verifier cascade 是把多个低耦合校验器按成本、确定性和风险影响排序。目标不是让某个 judge “裁判一切”, 而是用便宜、确定、可解释的检查先拦截明显问题, 再把复杂判断交给更昂贵的模型、规则或人工。

Cascade pattern

Evidence object design

建议每次 high-impact run 生成 reasoning_budget_evidence 对象:

不要把 hidden chain-of-thought 放进这个对象。需要记录的是控制结果和证据链接。

---

Financial retail scenarios

1. AML investigations

AML copilot 的高价值不是“自动判定可疑”, 而是减少 analyst 搜集材料和起草 narrative 的时间。

关键控制: AI 不应把“看起来可疑”的语言变成事实断言。叙述必须区分 observed facts, policy indicators, analyst judgment。

2. Credit policy reasoning

信贷场景的 reasoning budget 要保护公平性、可解释性和授权边界。

关键控制: AI 不能生成未经验证的不利行动原因, 不能使用禁止变量或 proxy reasoning, 不能把内部推理作为客户解释。

3. Payment dispute reasoning

支付争议需要在客户体验、卡组织规则、监管时限和损失控制之间平衡。

关键控制: 若证据不足, 系统应要求补件或升级, 而不是为了给出答案而猜测。

4. Complaints root-cause analysis

投诉 RCA 适合 test-time compute, 因为它通常需要跨渠道、跨系统、跨政策的证据拼接。

关键控制: 区分 customer allegation, confirmed fact, business error, systemic root cause, remediation action。

5. Contact center policy QA

Contact center 需要低延迟, 但不能牺牲政策准确性。

关键控制: 对话中不要展示内部推理。坐席需要的是可读话术、来源、限制和下一步。

---

Metrics/control/evidence model

Product and operations metrics

Control metrics

Evidence model aligned to observability

OpenTelemetry 的 trace/span 思路可以映射到 AI reasoning workflow:

root span: ai.reasoning.run
  span: ai.budget.classify
  span: ai.context.retrieve
  span: ai.plan.create
  span: ai.solve.generate
  span: ai.tool.calculate
  span: ai.verify.citation
  span: ai.verify.policy
  span: ai.escalate.human
  span: ai.output.deliver

关键属性:

---

Anti-patterns and failure modes

---

Architecture mapping to RAG / Agent / Copilot / Eval / Governance

---

ADR draft

Title

Adopt risk-proportional reasoning budget and verifier cascade for high-impact financial retail AI workflows.

Status

Proposed.

Context

Current AI workflows often use a uniform generation path. This creates three problems: low-risk tasks overconsume compute, high-risk tasks lack formal verification and escalation, and audit evidence is inconsistent across use cases. Financial retail workflows such as AML investigation support, credit policy reasoning, payment disputes, complaints RCA, and contact center policy QA require different latency, cost, evidence, and governance profiles.

Decision

We will introduce a reasoning budget policy with four tiers: Fast path, Evidence path, Deliberation path, and Controlled decision path. Budget tier assignment will be based on use case, customer impact, uncertainty, data sensitivity, user role, and channel SLO. High-impact workflows will use decomposition, deterministic tools where available, verifier cascade, abstention/escalation rules, and human approval gates. Runtime evidence will record inputs, source references, tool outputs, policy versions, verifier results, output hashes, and human actions. Hidden model reasoning will not be used as audit evidence or exposed to customers.

Consequences

Positive consequences:

• Better alignment between cost, latency, risk, and business value.
• Clearer control design for high-impact AI use cases.
• Stronger auditability without leaking chain-of-thought.
• Better operational capacity planning through tier mix and escalation metrics.

Tradeoffs:

• More orchestration complexity than a single-call model design.
• Requires ownership of budget policy, verifier maintenance, and evidence retention.
• High-risk workflows may have higher p95 latency and require human queue capacity.

Alternatives considered

Decision criteria

---

Interview answer: 30秒, 2分钟, CTO版本

30秒版本

AI reasoning budget 是把 test-time compute 当成产品和架构策略来管理。低风险问题走 fast path, 需要政策证据的问题走 RAG 和引用校验, 高影响场景走拆解、工具、verifier cascade 和人工复核。关键不是展示模型的 chain-of-thought, 而是保存可审计证据: 输入、来源、工具结果、政策版本、校验结果和人工决策。金融零售里, 这能同时控制成本、延迟、客户影响和监管风险。

2分钟版本

我会把推理预算设计成四层运行时策略。第一层是低风险 fast path, 适合简单 FAQ 和文本改写。第二层是 evidence path, 用在客服政策 QA 这类必须引用来源的任务。第三层是 deliberation path, 用在支付争议初判、投诉根因分析等需要拆解和检查的任务。第四层是 controlled decision path, 用在 AML investigation、信贷政策边界和客户不利影响解释等高风险场景。

架构上先做 risk and complexity classification, 再选择预算层。高预算不是简单增加 token, 而是增加证据检索、确定性工具、planner/solver/checker loop、verifier cascade、abstention 和人工复核。Verifier 应从便宜确定的检查开始, 包括权限、字段完整性、引用支撑、计算工具、政策规则, 最后才到模型评估或人工专家。

治理上, 我不会保存 hidden chain-of-thought 作为审计证据。审计证据应该是 source ids、policy versions、tool outputs、verifier results、model config hash、output hash、human approval and final disposition。这样既能证明系统做了合理控制, 又不会泄露内部草稿或把模型推测误当事实。

CTO版本

我会把 reasoning budget 作为 AI platform 的 runtime policy layer, 而不是每个应用团队自己写 prompt retry。平台提供统一的 budget classifier、orchestration contract、verifier registry、tool permission model、OpenTelemetry instrumentation 和 evidence store。每个 use case 注册 risk tier、SLO、allowed tools、verifier cascade、human gate 和 retention policy。

技术决策上, 我会优先把确定性校验和业务规则前置, 把 LLM 放在擅长的 decomposition、summarization、evidence synthesis 和 language generation 位置。高影响任务用 planner/solver/checker loop, 但 checker 不依赖同一个上下文自证正确, 而是结合 citation verifier、rule engine、calculation tool、policy version check 和 human review。成本方面, 我会建立 per-use-case compute budget、tier mix dashboard、p95 latency、override rate、unsupported claim rate 和 cost per resolved case, 用真实业务结果验证高预算路径是否值得。

治理上, 我会把 NIST AI RMF 的 Govern/Map/Measure/Manage 思路落到 evidence plane, 并对齐 ISO/IEC 42001 的 AI management system 责任。上线门禁不问“模型聪不聪明”, 而问“哪些请求会进入高预算路径, 哪些结论会被阻断或升级, 证据能否复盘, 模型或政策变更会不会破坏这些控制”。

---

7-day practice plan

---

Source anchors