AI 底层逻辑 / 经典论文
AI Agentic Process Audit：流程审计与重放保证架构
Agentic process audit 不是给 AI agent 加一层普通日志, 而是把意图、计划、工具动作、策略决策、人工批准、例外处理、输出和学习信号组织成可审阅、可重放、可质询的 evidence architecture。它的核心问题不是“模型回答得好不好”, 而是“当 agent 成为流程参与者时, 组织能否证明关键行为受到边界、权限、证据和控制约束”。
568 行ai-foundations/papers/161-ai-agentic-process-audit-workflow-replay-assurance-architecture.md
AI Agentic Process Audit / Workflow Replay / Assurance Architecture 解读
配对阅读：本篇的操作手册版（模板/RACI/门禁/runbook）是
docs/AI_AGENTIC_PROCESS_AUDIT_WORKFLOW_REPLAY_ASSURANCE_PLAYBOOK.md。第一遍读本篇建立原理与架构判断；第二遍做案例时再用 playbook 查表落地，两者不需要重复精读。
重要说明: 本文是学习、产品架构和内部 assurance 训练材料, 不构成法律意见、监管解释、审计意见、模型验证结论、内控有效性结论、风险接受决定或生产上线批准。文中的 audit、assurance、evidence、control effectiveness 和 process owner review, 均指支持内部审计、风险、合规、业务 Owner 和管理层进行独立审阅与挑战的证据架构, 不代表任何正式审计签署或监管认可。访问日期按 2026-06-30 记录。
Source Anchors
以下来源用于组织 AI 风险管理、AI 管理体系、架构描述、需求工程、可观测性、provenance 和金融机构 IT 审查语言。本文只将它们作为产品、架构和内部 assurance 的设计锚点, 不声称任何 schema、trace 或 replay workbench 自动满足审计、监管或模型风险批准。
| Source | Official link | 本文采用的思想 |
|---|---|---|
| NIST AI Risk Management Framework | https://www.nist.gov/itl/ai-risk-management-framework | 用 Govern / Map / Measure / Manage 组织 agentic workflow 风险识别、度量、控制、运行监测和改进证据。 |
| ISO/IEC 42001 AI management system | https://www.iso.org/standard/81230.html | 用 AI management system 的 scope、policy、operation、performance evaluation、management review 和 improvement 设计持续 assurance operating model。 |
| ISO/IEC/IEEE 42010 Architecture Description | https://www.iso.org/standard/74393.html | 用 stakeholder、concern、viewpoint、architecture view、correspondence 和 rationale 组织 replay architecture 的多视图描述。 |
| ISO/IEC/IEEE 29148 Requirements Engineering | https://www.iso.org/standard/72089.html | 用 stakeholder need、requirement、verification、validation、traceability 和 information item 设计 process claim 与 evidence contract。 |
| OpenTelemetry Documentation | https://opentelemetry.io/docs/ | 用 traces、metrics、logs、context propagation 和 semantic attributes 设计 runtime observability 与 replay trace。 |
| W3C PROV Overview | https://www.w3.org/TR/prov-overview/ | 用 Entity、Activity、Agent 的 provenance 思维表达一次 agentic 行为由谁、基于什么、通过什么活动生成。 |
| FFIEC IT Handbook | https://ithandbook.ffiec.gov/ | 用金融机构 IT 风险、审查、治理、外包、业务连续性和控制评估语言校准金融零售场景。 |
核心导读
Agentic process audit 不是给 AI agent 加一层普通日志, 而是把意图、计划、工具动作、策略决策、人工批准、例外处理、输出和学习信号组织成可审阅、可重放、可质询的 evidence architecture。它的核心问题不是“模型回答得好不好”, 而是“当 agent 成为流程参与者时, 组织能否证明关键行为受到边界、权限、证据和控制约束”。
User intent
-> intent interpretation
-> plan generation
-> policy decision
-> tool selection
-> tool call / observation
-> human approval
-> exception handling
-> output / side effect
-> feedback / incident learning
成熟的 agentic assurance 关注三类可证明性:
| 可证明性 | 需要回答的问题 | 典型证据 |
|---|---|---|
| 行为可证明 | Agent 在什么流程状态下做了什么, 是否越权 | trace、event、tool input hash、side effect id |
| 控制可证明 | 策略、审批、职责分离和例外管理是否真实运行 | policy decision、approval scope、SoD result、exception record |
| 学习可证明 | 缺陷、事故、人工编辑和 QA 发现是否进入改进闭环 | incident replay、eval case、remediation action、process update |
Replayable 不等于 bit-for-bit deterministic。对 LLM、RAG 和第三方模型来说, 更现实的承诺是:
在授权、隐私和保留边界内, 能重建批准版本集、关键输入、策略决策、工具动作、人工审批、输出哈希、业务状态和已知复现限制。
1. 问题定义
普通 AI 系统的审阅重点通常集中在模型输出、RAG 引用、安全过滤和线上监控。Agentic workflow 的风险更复杂, 因为 agent 会跨越流程步骤、调用工具、触发副作用、请求人工批准并处理例外。只保存最终答案或 API 日志, 无法支持金融零售场景中的独立挑战。
| 审阅问题 | 普通日志的缺口 |
|---|---|
| 用户真正要求了什么, agent 如何解释意图 | 只有 request text, 没有 intent classification、clarification 和 confidence |
| Agent 为什么选择这条计划 | 没有 plan version、candidate plan、rejected plan 和 rationale |
| 哪个工具动作影响了客户、资金、案件或监管材料 | 只有 API 成功/失败, 没有 side effect、idempotency、approval 和 rollback |
| 策略为什么允许、拦截或升级 | 没有 policy version、decision reason 和 obligations |
| 人工审批者看到了什么证据 | 只记录 approved, 没有 visible evidence set |
| 例外是合理偏差还是控制失败 | 没有 exception reason、owner、expiry 和 compensating control |
| 事故发生后能否重放流程 | 时间线断裂, 版本集合不完整, 敏感内容无法安全访问 |
| 控制是否有效 | 没有 sampling universe、population definition、control test result 和 exception aging |
1.1 Process Claim
Process claim 是团队对 agentic workflow 的可验证主张。它把业务结果、流程边界、agent 责任、人类责任、策略边界、证据要求和控制要求绑在一起。
| Weak claim | Strong process claim |
|---|---|
| AML copilot helps analysts work faster | For AML alert type A and B, the copilot generates a sourced case timeline and draft narrative, but final disposition remains analyst-owned and every saved narrative has source-span evidence. |
| KYC agent automates onboarding | The KYC onboarding agent classifies documents, identifies missing evidence and drafts customer follow-up, but cannot reject an applicant without human review and appeal path. |
| Payment repair agent fixes exceptions | The payment repair queue agent suggests repair actions and executes only low-risk reversible updates after policy decision and dual-control approval. |
Process claim 的结构:
Business outcome
+ workflow scope
+ agent responsibility
+ human responsibility
+ policy boundary
+ evidence requirement
+ control / exception handling
+ outcome measurement
1.2 Audit Trail、Observability 与 Provenance
Audit trail 和 observability 必须连接, 但不能混为一谈。
| Dimension | Audit trail | Observability |
|---|---|---|
| Primary question | 能否证明谁在什么权限和证据下做了什么 | 能否理解系统行为、性能、依赖和故障模式 |
| Main users | 内审、风险、合规、流程 Owner、法律、监管沟通 | SRE、平台、工程、产品、运营 |
| Data shape | events、approvals、version records、evidence objects、control results | traces、metrics、logs、exemplars、dashboards |
| Quality bar | completeness、integrity、non-repudiation、access control、chain of evidence | coverage、latency、sampling、diagnostic usefulness |
| Failure mode | 无法证明控制发生或输出被授权 | 无法诊断 outage、drift、cost spike 或依赖故障 |
OpenTelemetry span 可以说明 tool.invoke 在 250 ms 内完成; audit evidence 还必须说明工具 contract 版本、调用权限、policy decision、审批是否适用、side effect 是什么、是否可逆以及执行输入是否与批准输入一致。W3C PROV 的 Entity / Activity / Agent 视角则补足“由谁、基于什么、通过什么活动生成”的 lineage。
2. 架构模型
Agentic process audit architecture 应该作为 workflow runtime 的原生能力, 而不是上线后补一个日志导出。逻辑上可以拆为 orchestration、policy、tool gateway、HITL、event store、trace store、evidence lake、provenance graph、replay workbench 和 audit query layer。
Business workflow UI / queue / API
|
AI agent orchestrator
intent classifier | planner | policy engine | tool gateway | HITL workflow
|
OpenTelemetry traces + agentic process events
|
Evidence collection layer
schema validation | redaction | hashing | retention tagging | integrity checks
|
Event store + trace store + evidence lake
|
Provenance graph
entities | activities | agents | causal edges | version set
|
Replay workbench
timeline view | causal graph | evidence chain | policy decision view | approval view
|
Audit query and assurance reporting
process conformance | sampling | incident replay | control testing | process owner review
2.1 Audit Layers
| Layer | 核心问题 | Evidence objects |
|---|---|---|
| Intent layer | 用户、系统或队列触发的业务意图是什么 | request record、intent classification、clarification prompt、user role |
| Process layer | 该意图对应哪个批准流程、状态机和业务规则 | workflow definition、state transition、process version、BPMN / state model |
| Planning layer | Agent 如何把意图变成计划, 是否跨越风险边界 | plan candidate、plan rationale、risk tier、approval requirement |
| Control layer | 哪些 policy、entitlement、SoD、risk 和 compliance controls 被执行 | policy decision、permission result、SoD check、obligations |
| Action layer | Agent 读取、生成、写入或提交了什么 | tool contract、tool input hash、tool output pointer、side effect id |
| Human layer | 人如何审阅、批准、覆盖、拒绝或补充事实 | visible evidence set、decision reason、reviewer role、edit diff |
| Output layer | 哪些输出进入业务记录、客户沟通或监管材料 | output hash、citation map、delivery state、record id |
| Learning layer | 缺陷、反馈、事故和抽样结果如何改变未来行为 | QA finding、incident replay report、eval case、control action |
2.2 Core Components
| Component | Responsibility |
|---|---|
| Agent orchestrator | 为 intent、plan、tool call、observation、approval、output 发出 trace context 和 process events。 |
| Policy decision point | 产生 versioned allow、block、escalate、approval-required 和 obligation decisions。 |
| Tool gateway | 执行 authorization、idempotency、dry-run、side-effect capture、approval binding 和 kill switch。 |
| HITL workflow service | 记录 visible evidence、reviewer decision、reason code、edit diff、expiry 和 SoD result。 |
| Evidence collector | 校验 event schema, 应用 redaction profile, 计算 hashes, 指派 retention class。 |
| Event store | 保存 append-only domain events, 支持 workflow replay 和 conformance analysis。 |
| Trace store | 保存 spans、timing、cost、latency 和 dependency context, 支持 observability drilldown。 |
| Provenance graph | 连接 prompt、context、RAG、tool、approval 和 output objects 的实体、活动、参与者和因果边。 |
| Replay workbench | 让授权审阅者重建 timeline、causal chain、version set 和 evidence gap。 |
| Audit query layer | 提供内审、风险、流程 Owner 和事故复盘使用的参数化查询。 |
| Redaction / access layer | 控制 raw content、redacted content、hashes、pointers 和 aggregate metrics 的可见性。 |
| Retention / legal hold layer | 应用保留策略、删除处理和事故/审阅场景下的证据保全。 |
2.3 Event-Sourced Agent Workflow
Event-sourced agent workflow 把 agent 行为记录为不可随意改写的业务事件序列。当前 workflow state 可以由事件投影而来, 事故复盘和过程审阅可以从事件重新构造。
| Event family | 典型事件 | 审阅价值 |
|---|---|---|
| Intent | intent.received、intent.classified、intent.clarified | 证明用户需求和 agent 解释之间的关系 |
| Plan | plan.generated、plan.reviewed、plan.revised、plan.approved | 证明 agent 为什么按某个路径行动 |
| Action | action.proposed、tool.invoked、tool.completed、tool.failed | 证明工具动作、参数、结果和副作用 |
| Observation | observation.received、context.loaded、retrieval.completed | 证明 agent 看到的事实和证据 |
| Policy | policy.evaluated、policy.blocked、policy.escalated | 证明控制点真实运行 |
| Approval | approval.requested、approval.decided、approval.expired | 证明 HITL 和 dual control |
| Exception | exception.detected、override.recorded、compensation.executed | 证明例外管理和补救 |
| Output | output.drafted、output.finalized、output.delivered、output.discarded | 证明客户、监管或业务记录中的输出来源 |
| Feedback | feedback.captured、qa.sampled、user.edited | 证明 adoption、quality 和 learning signals |
| Incident | incident.signal.detected、incident.replay.started、learning.action.created | 证明事故复盘和改进闭环 |
2.4 Common Event Envelope
Agentic process events 应共享稳定 envelope, 再承载 domain-specific payload。
| Field | Purpose |
|---|---|
event_id | globally unique event identifier |
event_type | stable event name such as ai.plan.generated |
schema_version | payload contract version |
occurred_at / recorded_at | event time and collection time in UTC |
trace_id | end-to-end workflow trace |
workflow_id | business workflow instance |
case_id_hash | privacy-preserving case reference |
use_case_id | AI use case registry id |
risk_tier | workflow risk classification |
producer | service, gateway or application emitting event |
actor_type / actor_id_hash | user, agent, service, reviewer, policy engine |
data_class | classification for privacy, security and retention |
retention_policy_id | retention rule applied to event |
redaction_profile | how sensitive content was minimized |
prev_event_ids | causal predecessors, not only previous time event |
evidence_refs | pointers to controlled evidence objects |
integrity_hash | tamper-evidence or content hash |
2.5 Plan / Action / Observation / Approval / Output Schema
| Schema | Required fields | Assurance use |
|---|---|---|
| Plan event | plan_id、plan_version、goal、steps、risk_assessment、requires_approval、rationale_hash、rejected_options | Shows why the agent intended to act and whether the plan crossed a risk boundary. |
| Action event | action_id、tool_name、tool_schema_version、action_type、input_hash、dry_run_result、policy_decision_id、approval_id、side_effect_id、idempotency_key | Shows tool use, write boundaries, approvals and recoverability. |
| Observation event | observation_id、source_system、source_version、fact_type、fact_hash、retrieval_refs、confidence_band、freshness | Shows what facts the agent used and whether they were current and authorized. |
| Policy event | policy_id、policy_version、decision、reason_code、obligations、input_attribute_hash、evaluation_mode | Shows allow, block, escalate, redact, approval required or restrict decisions. |
| Approval event | approval_id、request_reason、approver_role、decision、reason_code、visible_evidence_hash、approval_scope、expiry | Shows HITL, maker-checker and SoD evidence. |
| Exception event | exception_id、exception_type、detected_by、severity、justification、owner、compensating_control、expiry、closure_evidence | Shows whether exceptions are controlled or unmanaged drift. |
| Output event | output_id、output_type、output_hash、citation_map、safety_label、record_system、delivery_channel、final_status | Shows what was finalized, stored, delivered or suppressed. |
| Feedback event | feedback_id、feedback_type、accept_edit_reject、edit_diff_hash、reason_code、business_outcome_ref | Shows adoption, quality signals and business outcome evidence. |
示例事件:
{
"event_id": "evt_20260630_kyc_000184",
"event_type": "ai.approval.decided",
"schema_version": "1.0",
"occurred_at": "2026-06-30T18:42:10Z",
"recorded_at": "2026-06-30T18:42:11Z",
"trace_id": "trc_kyc_onboarding_72f1",
"workflow_id": "wf_kyc_case_review_9482",
"case_id_hash": "sha256:1d4f...",
"use_case_id": "kyc_onboarding_agent",
"risk_tier": "high",
"producer": "kyc-review-workbench",
"actor_type": "reviewer",
"actor_id_hash": "hash:user:9a27",
"data_class": "restricted",
"retention_policy_id": "ret_ai_high_business_record_7y",
"redaction_profile": "pii-minimized-v3",
"prev_event_ids": ["evt_20260630_kyc_000181", "evt_20260630_kyc_000183"],
"evidence_refs": [
"evidence://visible-set/vis_kyc_5521",
"evidence://policy-decision/pol_kyc_8910",
"evidence://tool-input/tool_kyc_followup_2322"
],
"integrity_hash": "sha256:ab77...",
"data": {
"approval_id": "appr_kyc_7721",
"request_reason": "customer_followup_message",
"approver_role": "KYC_Senior_Reviewer",
"decision": "approved_with_edit",
"reason_code": "missing_ubo_evidence_clearer_language",
"visible_evidence_hash": "sha256:392e...",
"approval_scope": "draft_customer_followup_only",
"expiry": "2026-07-01T18:42:10Z"
}
}
3. 关键机制与生命周期
3.1 Process Claim to Evidence Chain
一个 process claim 必须能一路追到 evidence chain, 否则它只是产品愿景。
Process claim
-> requirement
-> control objective
-> workflow event
-> evidence object
-> audit query
-> sampling approach
-> control result
-> management action
Payment dispute assistant 示例:
| Chain element | Example |
|---|---|
| Process claim | Assistant drafts dispute evidence packet but does not submit chargeback without maker-checker approval. |
| Requirement | All customer-impacting dispute submissions require reviewer approval based on visible evidence set. |
| Control objective | Prevent unsupported or unauthorized chargeback submissions. |
| Workflow event | approval.decided before tool.invoked for chargeback_submit. |
| Evidence object | approval record, visible evidence hash, tool input hash, policy decision id, side effect id. |
| Audit query | Show all chargeback submissions where approval is missing, expired or not tied to exact execution input. |
| Sampling approach | 100% automated exception query plus monthly sample of approved submissions. |
| Control result | Exceptions by reason, reviewer quality, stale approval count, remediation status. |
| Management action | Update tool gateway to block mismatched approval input hashes. |
3.2 Chronology vs Causality
Workflow replay 必须区分“时间上发生过”和“因果上被授权、被约束、被证据支持”。
| Chronology asks | Causality asks |
|---|---|
| What happened before what? | Which event caused, enabled, constrained or justified another event? |
| What was the next timestamp? | What policy, approval, observation or plan step was a required predecessor? |
| Did the tool call occur after approval? | Was this approval scoped to this exact tool input and still valid at execution time? |
| Did the output follow retrieval? | Did the output's material claims rely on retrieved evidence or unsupported model generation? |
Approval A happened before Tool Call B.
That is chronology.
Approval A authorized input hash H1.
Tool Call B executed input hash H2.
H1 != H2.
That is causal control failure.
因此 event schema 必须记录 prev_event_ids、causal_refs、policy_decision_id、approval_scope、input_hash 和 side_effect_id。没有这些连接, timeline 只能帮助讲故事, 不能支持控制测试。
3.3 Replay Modes
| Replay mode | Purpose | Limits |
|---|---|---|
| Technical replay | Reconstruct model, prompt, RAG, policy, tool and runtime version set. | May not reproduce identical LLM output if provider behavior changed. |
| Business replay | Reconstruct workflow states, user actions, approvals, outputs and business record updates. | Requires business systems to retain record ids and state transitions. |
| Control replay | Re-evaluate whether required controls fired and were bound to correct inputs. | Does not itself prove control design is sufficient. |
| Incident replay | Reconstruct events in an incident window and connect root cause, impact and remediation. | Sensitive content may require restricted access and legal guidance. |
| Learning replay | Turn failures, edits, overrides and QA findings into eval cases and process improvements. | Must avoid using feedback data outside purpose and consent boundaries. |
Reproducibility limits 要成为 replay packet 的一部分:
| Limit | Mitigation |
|---|---|
| LLM nondeterminism | Preserve prompt/config/model route/output hash and use replay to compare behavior, not guarantee exact output. |
| Third-party model version opacity | Capture vendor metadata, response headers, model alias resolution and contractually available version info. |
| External tool state changes | Store tool request/response hashes, side-effect ids, business record state and compensating action records. |
| Knowledge base drift | Preserve KB version, index version, retrieved chunk ids, effective dates and document lifecycle state. |
| Privacy redaction | Preserve enough hashed/pointer evidence to reconstruct under authorized conditions without broad raw content exposure. |
3.4 Exception、Override 与职责分离
真实流程一定会有例外。assurance 关注的不是消灭所有偏差, 而是区分合理业务例外、文档缺陷、控制失败、流程漂移和流程模型错误。
| Type | Example | Required evidence |
|---|---|---|
| Business exception | KYC case requires non-standard document due to jurisdiction rule | policy citation, reason code, reviewer approval, customer communication record |
| Policy override | Payment dispute response needs supervisor approval despite low automated risk score | override reason, approver role, scope, expiry, evidence set |
| Tool exception | Payment repair API unavailable, manual repair queue used | incident link, manual action record, reconciliation evidence |
| Data exception | RAG source stale for one policy section | affected scope, compensating manual source check, expiry |
| Workflow exception | HITL reviewer queue exceeds SLA, case routed to backup team | capacity signal, escalation decision, customer impact analysis |
| Model exception | Agent confidence below threshold but reviewer proceeds after independent evidence check | confidence band, reviewer rationale, QA sample inclusion |
Good override record 应包含 baseline rule、business reason、risk impact、approver role、visible evidence、compensating control、expiry / closure trigger 和 learning path。
Weak:
Supervisor approved exception.
Strong:
Supervisor approved hardship option deviation for case class H2 because customer submitted verified disaster impact documentation not covered by standard script. AI recommendation was restricted to draft language. Final treatment selected by hardship specialist. Case added to monthly vulnerable-customer QA sample and policy team review.
Agentic AI 也会模糊职责, 因为同一平台可能同时 propose、execute、document 和 monitor。关键 SoD evidence 包括 requester_role、approver_role、relationship_check_result、same_user_blocked、delegated_authority_source、approval_scope、independent_challenge_required 和 break_glass_reason。
3.5 Incident Replay Lifecycle
Incident replay reconstructs what happened, why it happened, who or what allowed it, what impact occurred and what changed afterward.
| Section | Contents |
|---|---|
| Incident scope | incident id, use case, workflow, time window, affected cases, severity |
| Version set | model, prompt, RAG index, policy, tool schema, release bundle, feature flags |
| Timeline | chronological events and spans |
| Causal graph | required predecessors, policy decisions, approvals, tool actions, outputs |
| Evidence gaps | missing spans, missing event fields, inaccessible source records, redaction limits |
| Customer or business impact | affected customers, cases, funds, reports, timelines, operational backlog |
| Control analysis | which controls worked, failed, were bypassed or were absent |
| Exception analysis | whether deviations were justified, expired or unmanaged |
| Remediation | rollback, compensation, customer action, policy update, prompt update, tool restriction |
| Learning loop | eval cases, regression tests, training, process model update, control improvement |
Payment repair incident 示例:
| Finding | Evidence |
|---|---|
| Duplicate side effects occurred in 18 cases | tool side_effect_id and payment system state transitions |
| Idempotency key was generated from case id only, not repair action id | tool gateway event schema and code release notes |
| Approval existed but was scoped to first repair attempt | approval scope and expiry |
| Retry path bypassed dry-run after timeout | trace timeline and causal graph |
| Customer-visible balances were corrected through compensating entries | compensating action records and reconciliation report |
| Regression test added | eval/control test case for retry idempotency |
4. 证据与控制
4.1 Evidence Chain Model
Agentic process evidence 应组织成链, 从业务结果倒推到事件、来源系统、版本和控制。
Business outcome evidence
<- process conformance evidence
<- workflow replay evidence
<- event evidence
<- source system evidence
<- version and control evidence
| Evidence type | Example | Assurance use |
|---|---|---|
| Business outcome evidence | alert aging reduction, dispute cycle time, onboarding completion, hardship treatment quality | Shows whether workflow produced intended value without unacceptable harm. |
| Process conformance evidence | state transitions, required approvals, SoD results, exception reasons | Shows whether workflow followed approved process or justified exception. |
| Workflow replay evidence | trace, event sequence, causal graph, version set | Supports reconstruction and independent challenge. |
| Event evidence | plan, action, observation, policy, approval, output, feedback events | Provides granular proof of actions and decisions. |
| Source system evidence | case management records, payment system state, KYC document store, regulatory report data | Anchors AI evidence to systems of record. |
| Version and control evidence | prompt, model, KB, policy, tool schema, release bundle | Shows which approved artifact set governed behavior. |
业务价值指标必须配控制 counterweight:
| Use case | Outcome evidence | Control counterweight |
|---|---|---|
| AML investigation copilot | reduced alert aging, better narrative completeness, lower reopen rate | no unsupported SAR implication, analyst final disposition retained |
| Payment dispute assistant | faster evidence packet creation, fewer missing-document reworks | chargeback submission requires maker-checker approval |
| KYC onboarding agent | faster first-pass completion, lower document chase volume | no automated rejection, appeal route preserved |
| Collections hardship case agent | better hardship option matching, improved follow-up timeliness | vulnerable customer and fair treatment checks |
| Regulatory reporting narrative drafter | shorter variance explanation cycle, fewer reviewer corrections | no unsupported metric cause, source lineage visible |
| Payment operations repair queue agent | lower repair backlog and fewer duplicate repairs | dual control for irreversible or customer-impacting updates |
4.2 Sampling and Testing
Sampling begins with a population definition. Without it, teams cherry-pick good examples.
| Population | Example |
|---|---|
| All high-risk workflow instances | All KYC onboarding agent cases where customer follow-up was drafted in June 2026. |
| All customer-impacting tool actions | All payment repair queue updates that changed customer-visible status. |
| All overrides | All collections hardship cases where AI recommendation was overridden. |
| All policy blocks | All regulatory narrative drafts blocked for unsupported claim. |
| All incidents or near misses | All AML copilot summaries flagged by QA as material evidence omission. |
| Test type | Purpose | Example |
|---|---|---|
| Design effectiveness test | Determine whether the control, if operated, would address the risk. | Does tool gateway approval binding prevent mismatched execution input? |
| Operating effectiveness test | Determine whether the control actually operated across samples. | Sample approved chargeback submissions and verify approval hash equals tool input hash. |
| Automated exception query | 100% scan for impossible or prohibited patterns. | Find tool write actions with no policy decision or expired approval. |
| Process conformance test | Compare actual event sequence to approved workflow model. | KYC follow-up must have intent, document observation, policy check, draft, review and output. |
| Outcome reasonableness test | Compare process result with business outcome and control counterweight. | Dispute cycle time improved without higher rework or complaint rate. |
| Replay drill | Reconstruct one case end-to-end under access controls. | Rebuild AML case timeline from trace, events, source records and approvals. |
Risk-based sampling should combine 100% automated queries for mandatory events, attribute sampling for required control evidence, judgmental samples for high-risk or complaint-linked cases, stratified samples by channel/segment/language/geography/model route/reviewer, incident-driven samples from near misses, and negative samples where agent refused, escalated or blocked action.
Sampling record should include population_id、sample_method、sample_period、sample_size、test_objective、pass_criteria、exception_classification 和 remediation_owner。
4.3 Process Conformance and Audit Queries
Process conformance asks whether actual workflow execution matches the approved process model.
Approved model:
intent -> plan -> policy -> tool dry-run -> approval -> tool execution -> output -> feedback
Actual trace:
intent -> plan -> tool execution -> output
Conformance result:
non-conformant because policy and approval events are missing before customer-impacting tool execution.
| Category | Meaning | Example |
|---|---|---|
| Conformant | Actual event path follows approved process. | KYC agent drafted follow-up only after document evidence and reviewer approval. |
| Justified exception | Process deviated but with authorized reason, owner and compensating control. | Backup reviewer approved due to outage under documented continuity procedure. |
| Control failure | Required control absent, expired, mismatched or bypassed. | Payment repair tool executed without valid approval. |
| Process drift | Repeated deviations show the real process has changed without approval. | Reviewers routinely skip citation check because UI makes it difficult. |
| Model of process wrong | Approved process model omits legitimate operational path. | AML escalation path for multi-jurisdiction cases not modeled. |
Audit query catalog should be designed during requirements and architecture:
| Query | Purpose |
|---|---|
| Show all tool write actions without preceding policy decision. | Detect bypassed control. |
| Show all approvals where visible evidence hash is missing. | Detect weak HITL evidence. |
| Show all output deliveries where output hash differs from approved draft hash. | Detect post-approval mutation. |
| Show all overrides by reviewer, reason and case type. | Detect concentration, training need or process ambiguity. |
| Show all workflows where exception expiry passed without closure. | Detect unmanaged residual risk. |
| Show all regulatory narrative drafts with unsupported material claims. | Detect output evidence failure. |
4.4 Operating Cadence
| Forum | Cadence | Main question | Output |
|---|---|---|---|
| Workflow evidence design review | Before pilot and major release | Are process claims, events, controls and replay needs defined? | evidence contract and release gate |
| Process conformance review | Monthly or risk-based | Are actual traces matching approved process? | conformance report and action log |
| Exception and override review | Weekly for high-risk workflows | Are exceptions justified, aging and closing? | exception register update |
| Incident replay review | Triggered by incident or near miss | What happened, why, impact and learning? | replay packet and remediation |
| Assurance management review | Quarterly | Are controls, outcomes and evidence architecture improving? | management action and roadmap |
5. 金融零售场景
5.1 Scenario Map
| Use case | Audit focus | Replay evidence |
|---|---|---|
| AML Investigation Copilot | Analyst final accountability | final disposition event, analyst approval, no auto-SAR submission |
| AML Investigation Copilot | Evidence completeness | transaction refs, KYC refs, adverse media refs, source-span map |
| Payment Dispute Assistant | Chargeback submission authority | maker-checker approval, tool input hash, policy decision |
| Payment Dispute Assistant | Customer communication | approved letter output hash, delivery channel, complaint link |
| KYC Onboarding Agent | No automated rejection | output status, reviewer decision, appeal route evidence |
| KYC Onboarding Agent | Missing evidence detection | document observation events, confidence band, source pointer |
| Collections Hardship Case Agent | Fair treatment | vulnerability flag handling, policy decision, human specialist approval |
| Collections Hardship Case Agent | Override quality | override reason codes, supervisor sample, customer outcome |
| Regulatory Reporting Narrative Drafter | Source lineage | metric id, data source, report period, transformation refs |
| Regulatory Reporting Narrative Drafter | Attestation boundary | AI draft marked as draft, authorized signer retained |
| Payment Operations Repair Queue Agent | Repair action authorization | tool risk tier, policy decision, dual control |
| Payment Operations Repair Queue Agent | Reversibility | side effect id, idempotency key, compensating action |
5.2 Design Patterns
| Pattern | 适用场景 | 架构要点 |
|---|---|---|
| Draft-only agent | AML narrative、regulatory variance、customer letter | output hash、citation map、reviewer edit diff、final human disposition |
| Approval-bound tool action | chargeback submission、payment repair、account update | tool input hash 与 approval scope 绑定, 过期或变更即重新审批 |
| Policy-first RAG | KYC explanation、collections hardship、wealth suitability | source freshness、jurisdiction filter、policy precedence、unsupported claim blocker |
| Exception-managed automation | outage fallback、manual repair、backup reviewer | structured exception、owner、expiry、compensating control、closure evidence |
| Learning replay loop | QA defect、complaint、near miss、incident | failed trace -> eval case -> regression test -> policy/process update |
6. 反模式
| Anti-pattern | Why it fails | Mature replacement |
|---|---|---|
| Final answer as audit evidence | It hides intent, plan, tool calls, approvals and policy decisions. | Replayable trace with event-sourced workflow evidence. |
| Logging everything raw | Creates privacy, security and retention risk without better assurance. | Minimum sufficient evidence with redaction, hash, pointer and controlled raw access. |
| Chronology treated as causality | "Approval happened before action" does not prove action was approved. | Causal links through input hash, approval scope and policy decision. |
| HITL recorded as yes/no | Review cannot determine what evidence human saw. | Visible evidence set, decision reason, edit diff and expiry. |
| Exceptions hidden in comments | Cannot distinguish justified business exception from control failure. | Structured exception record with owner, expiry and compensating control. |
| Audit query afterthought | Evidence exists but cannot answer real review questions. | Audit query catalog designed during requirements and architecture. |
| Same team self-certifies all controls | Lack of independent challenge and SoD. | Separate author, approver, reviewer and process owner roles based on risk. |
| Replay promises exact LLM reproduction | Overclaims determinism and ignores vendor/model drift. | Document reproducibility limits and preserve version set plus output hashes. |
| Sampling only successful cases | Misses near misses, blocks, overrides and failures. | Risk-based samples covering negative paths and exceptions. |
| Outcome metrics without control counterweights | Speed gains may hide customer harm or control erosion. | Pair business outcome evidence with quality, risk and conformance evidence. |
7. 最终心智模型
Agentic workflow assurance 要让五件事变得可见:
The final answer is not the process.
The timeline is not the cause.
The approval is not valid unless bound to exact evidence and action.
The exception is not acceptable unless owned, justified, expiring and monitored.
The replay is not audit sign-off, but it is the evidence architecture that makes serious review possible.
高级产品与架构判断不是把 agent 做得更像“自动化黑箱”, 而是把它设计成可重放的流程参与者: 每个高影响动作都有边界, 每个边界都有证据, 每个证据都能被查询, 每次偏差都能被分类, 每次事故都能进入可验证的学习闭环。
SOTA 状态标注 (2026-07-01)
本篇属于第二、三遍深读池（参考架构/深读笔记），未列入 12 周主线必读。时效基线为写作时点；引用前请按 CLAUDE.md 全局时效性硬规则复查最新进展。模块级 SOTA 对照见 docs/AI_SYSTEMATIC_LEARNING_ROADMAP_2026.md 各周「2026 SOTA 对照」行与文末「SOTA 检查」。