AI Document Intelligence / Unstructured Data / Evidence Quality Architecture 解读

面向对象: CBAP+ Senior BA / Advanced AI PM / Product Architect / Enterprise Architect / Operations Architect / Model Risk / Records Management / Fraud Risk / KYC-KYB Operations / Claims and Disputes Lead / Loan and Insurance Servicing Product Owner。 核心问题: 金融零售 AI 系统如何把 bank statement、paystub、claim package、dispute evidence、KYC/KYB 文件、insurance / loan servicing 文档和运营来信, 从 unstructured documents 转成 evidence-grade, auditable, reviewable, workflow-ready 的事实, 同时控制 OCR/layout/multimodal extraction、classification、entity extraction、summarization、confidence scoring、human review、document provenance、records retention、legal hold、fraud/tamper checks 和 model risk? 学习目标: 建立 document intelligence reference architecture、evidence quality model、document provenance and chain-of-custody、confidence and review design、records/legal hold integration、workflow automation controls、fraud/tamper detection、model risk governance 和 senior PM/architect decision framework。

0. Disclaimer

本文是学习、架构训练和作品集材料, 不构成法律意见、监管意见、记录保留结论、e-discovery 建议、KYC/KYB 充分性判断、贷款或保险承保结论、消费者争议处置意见、欺诈处置指令、模型验证报告或供应商推荐。

正式项目必须由 Legal、Compliance、Privacy、Records Management、Information Governance、Model Risk、Fraud Risk、Financial Crime、Operations、Product、Architecture、Information Security、Data Governance、Vendor Management、Internal Audit 和相关业务 owner 共同判断。记录、证据、法律保留、客户通知、KYC/KYB、信贷、保险、投诉、争议、索赔、跨境数据和 e-discovery 的具体适用性, 取决于 product、record type、jurisdiction、retention schedule、legal hold status、customer segment、channel、policy、contract 和 Legal / Compliance / Records interpretation。

本文不把 document intelligence 简化成 OCR 教程。OCR 只是把图像转成文本的一个能力。金融零售场景真正需要的是 evidence-grade extraction architecture: 能说明文档从哪里来、是否完整、字段从哪一页哪一区域抽取、置信度如何校准、何时需要人工复核、如何进入工作流、如何保留记录、如何处理 legal hold、如何检测篡改和欺诈, 以及事后如何重放决策证据。

---

Source Anchors

一句话:

Document intelligence is not "OCR + LLM summary". In financial operations, it is an evidence system that converts unstructured documents into policy-bound, source-linked, confidence-calibrated, human-reviewable and records-aware decision inputs.

---

1. Thesis

金融零售文档智能的核心目标不是“更快读文档”, 而是把文档变成可依赖的 operational evidence。成熟架构应实现下面的转换:

from: uploaded PDF / scanned image / email attachment / photo / fax
to: evidence envelope
    + document provenance
    + page/layout map
    + extracted entities with source coordinates
    + normalized facts
    + confidence and validation results
    + fraud/tamper signals
    + human review decisions
    + workflow action
    + records retention / legal hold metadata
    + replayable audit trail

核心判断:

Text recognized does not mean fact established.
A model summary does not mean evidence accepted.
Confidence score does not mean business risk resolved.
Human review does not mean control effectiveness unless review is designed, sampled and evidenced.
Document storage does not mean records compliance.

高级 PM / Architect 要把 document AI 设计成四层系统:

最重要的架构边界是:

extraction result
  != verified fact
  != policy-accepted evidence
  != legal sufficiency
  != final business decision

---

2. Why It Matters

金融零售运营高度依赖非结构化文档:

AI 放大三类风险:

1. 规模风险: 一个 extraction defect 会批量影响成千上万个 case。
2. 语义风险: 模型把“看起来像工资单”误写成“收入已验证”。
3. 证据风险: 业务行动发生后无法证明字段来自哪份文档、哪个模型、哪个版本、哪个 reviewer。

Senior PM / Architect 的目标不是“自动化率最高”, 而是:

Use automation where evidence quality is sufficient,
route ambiguity to the right human queue,
preserve proof of what was seen and decided,
and keep records, privacy, fraud and model risk controls in the same workflow.

---

3. Evidence Object Taxonomy

document AI 中至少要区分七类对象。混用这些对象会导致不可审计的自动化。

字段级 evidence metadata 建议:

field_name
document_id
document_version
page_number
bounding_box_or_anchor
raw_text
normalized_value
extraction_method
model_or_rule_version
confidence_score
calibration_bucket
validation_results
cross_document_match
fraud_or_tamper_signals
human_review_status
policy_acceptance_status
evidence_retention_rule
legal_hold_flag

---

4. Reference Architecture Model

参考架构:

intake channels
  -> document capture and provenance service
  -> file normalization / rendering / virus and content safety scan
  -> document classification and package splitting
  -> OCR + layout understanding + table extraction
  -> multimodal extraction / entity extraction / relationship mapping
  -> normalization and business validation
  -> confidence calibration and quality scoring
  -> fraud / tamper / duplicate / synthetic-document checks
  -> evidence policy engine
  -> human review and exception queues
  -> workflow integration: KYC, claims, disputes, servicing, complaints
  -> records retention / legal hold / disposition integration
  -> evidence ledger, monitoring, QA and model governance

关键组件:

---

5. Document Classes and Evidence Risk

不是所有文档都应该用同样的 automation threshold。按 business impact、field criticality、fraud exposure 和 records sensitivity 分层。

高级设计原则:

• 用 document class 决定 extraction schema、review threshold、retention class、fraud checks 和 workflow route。
• 对 high-impact fields 使用 field-level controls, 不只看 document-level confidence。
• 对 summaries 使用 source-linked citations inside the case tool, 不能让摘要成为唯一证据。
• 对 customer-provided documents 和 institution-generated records 分开治理。

---

6. Evidence-Grade Extraction Pipeline

6.1 Intake and Capture

6.2 Classification and Package Splitting

分类不是 UI 标签, 而是 workflow/risk/records decision 的入口。

6.3 Layout, OCR and Multimodal Understanding

架构关注点不是 OCR 算法细节, 而是 evidence recoverability:

6.4 Entity Extraction and Normalization

Entity extraction 必须按 field criticality 分级:

---

7. Confidence Architecture

置信度不是一个漂亮分数。它是 routing, review, policy acceptance and monitoring 的控制输入。

弱模式:

if model_confidence > 0.85 then auto-approve

强模式:

if field is low impact and confidence calibrated and validations pass
  then auto-populate with audit trace
if field is high impact or conflicts with another source
  then route to human review
if document class has fraud/tamper signal or legal/authority implication
  then require specialized queue

置信度设计要点:

• threshold 按 field criticality、journey risk、customer harm、fraud exposure 定义。
• 不能用 document-level average 掩盖关键字段错误。
• 置信度必须校准, 并用 reviewer outcomes 监控 drift。
• 对 high-impact decisions 使用 validation + review + policy, 不只用 score。
• reviewer override 进入 feedback loop, 但不自动训练模型, 除非数据治理和模型治理已批准。

---

8. Human Review Design

人工复核不是“自动化失败后的人工兜底”, 而是 evidence architecture 的组成部分。

Reviewer UI 应具备:

• 左侧原文/页面/图像, 右侧结构化字段, 字段点击可定位 source。
• 显示 model output、confidence、validation failures、previous reviewer actions。
• 不显示会诱导 rubber-stamping 的“AI says approve”。
• 对 fraud/risk signal 做最小必要解释, 避免泄露敏感规则。
• reviewer 必须选择 structured reason code and free-text rationale where needed。
• 每次 override 记录 old value、new value、reason、reviewer、timestamp、policy version。

---

9. Document Provenance and Chain of Custody

Evidence-grade document AI 必须能回答:

Who or what submitted the document?
When was it received?
Which exact file and page were processed?
Which model/rule/version extracted the field?
Was the document changed, re-rendered, split, redacted or reprocessed?
Who reviewed or overrode the result?
Which workflow decision used the evidence?
Which final customer or counterparty communication referenced it?

Provenance controls:

LLM output 必须 grounded:

summary_claim:
  text: "The claimant submitted two repair invoices totaling 3,420.00."
  sources:
    - document_id: D123
      page: 2
      field: invoice_total
      value: 1,850.00
    - document_id: D124
      page: 1
      field: invoice_total
      value: 1,570.00
  model_run_id: MR789
  reviewer_status: reviewed

---

10. Records Retention and Legal Hold Architecture

Document intelligence 经常会生成多种 derived artifacts: OCR text、layout JSON、extracted fields、summary、review notes、decision logs、redacted copies、exports。它们是否构成 records、保留多久、是否进入 legal hold, 不能由 AI 团队自行判断。

架构应提供可配置机制:

边界原则:

• 不在产品文案或架构文档中断言某类文档的法定保留期限。
• retention schedule、legal hold status、e-discovery、regulatory response 由 Legal / Compliance / Records 确认。
• AI summaries 不能替代原始记录, 除非 Records/Legal 明确认可该 artifact 的用途和保留方式。
• legal hold 下, 自动删除、模型训练清理、vendor purge、data minimization job 都必须检查 hold state。

---

11. Fraud, Tamper and Authenticity Checks

document AI 必须假设输入可能被操纵。特别是 bank statements、paystubs、invoices、receipts、screenshots、photos 和 identity/KYB documents。

重要边界:

• fraud/tamper model output 是 risk signal, 不是最终欺诈结论。
• 客户沟通应解释需要补充或复核的业务原因, 不暴露内部检测规则。
• 对高风险文档, 控制组合通常比单一模型更重要: metadata + layout + arithmetic + external validation + human review + monitoring。

---

12. Workflow Integration

Document intelligence 的价值来自进入业务流程, 不是停留在 extraction dashboard。

Workflow contract should include:

input artifact type
required extracted fields
confidence thresholds by field
validation rules
human review triggers
fraud/tamper triggers
policy decision states
records metadata
case update payload
customer communication constraints
fallback and exception path
monitoring metrics

---

13. Model Risk and AI Governance

Document AI 可以同时使用 OCR engine、layout model、classification model、multimodal LLM、entity extraction model、rules engine、fraud model、summarizer。每个模型/规则的风险不同。

Governance design:

• Map use cases and harms using NIST AI RMF categories: validity, reliability, safety, security, accountability, transparency, privacy, fairness。
• Maintain model inventory with purpose, owner, vendor, version, data classes, decision impact and allowed uses。
• Define eval sets by document class, language, channel, quality, customer segment, fraud pattern and workflow outcome。
• Monitor extraction accuracy by field criticality, not only aggregate F1。
• Track reviewer overturns, complaint defects, downstream rework and customer harm indicators。
• Use ISO/IEC 42001-style AI management system controls: role accountability, operational procedures, performance evaluation, internal audit and continual improvement。
• Treat prompt/template/ruleset changes as governed artifacts when they affect extraction or workflow decisions。

---

14. Product / Architecture Decisions

---

15. Control Matrix

---

16. Metrics

Balanced executive dashboard:

Speed: cycle time and review productivity improve.
Quality: critical fields are accurate and calibrated.
Risk: fraud, tamper, legal hold and records controls work.
Fairness: errors are monitored across document quality, language and channel.
Trust: every automated or reviewed decision is replayable.

---

17. Failure Modes

---

18. Interview-Ready Takeaways

Q1: 为什么 document intelligence 不是 OCR 项目?

OCR 只解决“看见文字”。金融零售真正需要的是 evidence-grade extraction: 文档 provenance、layout/source coordinates、字段置信度、业务验证、fraud/tamper checks、人工复核、workflow action、records retention、legal hold 和 audit replay。否则只是把人工读错文档变成机器批量读错文档。

Q2: 如何判断某个抽取字段可以自动进入业务流程?

不能只看模型 confidence。要看 field criticality、document class、confidence calibration、source linkage、validation result、cross-document consistency、fraud/tamper signals、policy gate 和 human review threshold。高影响字段例如 income、authority、claim amount、beneficial owner 通常需要更强验证或复核。

Q3: AI summary 在 claims/disputes/KYC 中如何安全使用?

Summary 应该作为 reviewer productivity tool, 不是 evidence replacement。每个关键陈述要 source-linked, 不能给出 unsupported eligibility、KYC/KYB、fraud 或 legal conclusion。最终业务决定应引用 structured evidence、policy reason 和 reviewer action。

Q4: records retention 和 legal hold 为什么要进入 document AI 架构?

因为 document AI 会产生 raw docs、OCR text、layout JSON、extracted fields、summaries、review notes、exports 等 derived artifacts。哪些是 records、保留多久、是否受 legal hold 影响, 取决于 product、record type、jurisdiction、retention schedule、hold status 和 Legal/Compliance/Records interpretation。架构必须能传播 metadata and hold state, 而不是事后人工查找。

Q5: 高级 PM 如何衡量 document AI 成功?

不只看自动化率或处理时间。要看 field-level accuracy、confidence calibration、review overturn、evidence completeness、records/hold metadata completeness、fraud/tamper yield、complaint defects、downstream rework、customer harm and audit replay success。速度必须和证据质量一起看。

---

19. Practical Templates

19.1 Document Evidence Envelope

Document ID:
Case ID:
Customer / business reference:
Source channel:
Received timestamp:
Submitter / system reference:
Raw file hash:
File type and size:
Page count:
Document class / subtype:
Classification confidence:
Language / locale:
Quality score:
Processing lineage:
  renderer version:
  OCR/layout version:
  extraction model version:
  ruleset version:
Fraud/tamper signals:
Record class:
Retention rule:
Legal hold flag:
Access restrictions:
Derived artifacts:
Workflow decisions:
Complaint / audit links:

19.2 Field Extraction Spec

19.3 Confidence and Review Policy

Document class:
Workflow:
Field criticality:
Customer impact:
Fraud exposure:
Auto-populate allowed when:
  field confidence:
  classification confidence:
  validations:
  tamper signals:
  cross-document consistency:
Human review required when:
  low confidence:
  conflict:
  authority/legal implication:
  high amount:
  vulnerable customer / complaint sensitivity:
Sampling rule:
Reviewer queue:
QA metric:

19.4 Human Review Record

Review task ID:
Reviewer role:
Document ID:
Field(s) reviewed:
Model suggestion:
Source location:
Validation failures:
Fraud/tamper signals:
Reviewer decision:
Corrected value:
Reason code:
Free-text rationale:
Second approval:
Workflow action:
Customer communication reference:
Timestamp:

19.5 Records / Legal Hold Integration Card

Artifact type:
Raw document:
Rendered page:
OCR text:
Layout JSON:
Extracted fields:
AI summary:
Review notes:
Workflow decision:
Exported package:
Record class owner:
Retention schedule reference:
Legal hold propagation rule:
Disposition approval:
Vendor copy:
Search / retrieval metadata:
Access restrictions:
Audit evidence:

---

20. Final Operating Principle

成熟的 AI document intelligence architecture 可以用一个问题检验:

Can the institution prove that every automated or human-assisted document decision
was based on the right document,
the right source-linked fields,
the right confidence and validation controls,
the right human review boundary,
the right fraud and records treatment,
and the right workflow policy at that point in time?

如果答案不清楚, 团队缺的不是更强 OCR。缺的是 document provenance、evidence quality、confidence calibration、human review design、records/legal hold integration、fraud controls、workflow contracts 和 AI governance 组成的一套 evidence operating architecture。