AI Uncertainty UX / Abstention / Confidence / Escalation Architecture Playbook

定位: 面向 CBAP+ Senior BA、金融零售 AI PM、Product Architect、Solution Architect、AI Governance Lead、Model Risk、Compliance Technology、Contact Center、Complaint Operations 和 Financial Crime Ops 的可执行 playbook。 核心目标: 把 AI 不确定性从 "confidence score" 升级为可运行的产品控制、体验语言、人工升级、证据包、监控和治理体系。 适用范围: credit eligibility assistant、wealth education / advice boundary、payment dispute claim assistant、AML analyst assistant、KYC document extraction、complaint intelligence、contact center copilot、RAG customer service、agentic workflow 和 employee copilot。

重要说明: 本 playbook 是学习、作品集、架构训练和内部治理讨论材料, 不构成法律意见、合规结论、投资建议、信贷审批建议、AML/SAR 决策、投诉处理意见、模型验证报告或审计结论。正式项目必须由 Legal、Compliance、Risk、Model Risk、Privacy、Security、Operations、Business Owner、Technology Owner 和管理层结合机构类型、司法辖区、产品、客户群、渠道和内部政策确认。

---

Source Anchors

---

1. Purpose And When To Use

Use this playbook when an AI system may influence:

• customer understanding of eligibility, fees, disputes, complaints, account status or next steps;
• employee handling of regulated, sensitive or customer-impacting workflows;
• extraction, classification, summarization or recommendation used in operations;
• routing to human review, specialist queue, licensed advisor, AML investigator or complaint team;
• customer trust, harm prevention, support handoff, appeal or regulatory evidence.

Do not use uncertainty UX as a cosmetic disclaimer. Use it when the product must decide:

Can AI answer?
Can AI answer only part of the question?
Should AI ask for more information?
Should AI refuse safely?
Should AI route to a human?
Should AI block the action and trigger control review?
What should the customer or employee see?
What evidence proves the decision was appropriate?

1.1 Outcome

By the end of design, each use case should have:

---

2. Operating Model

2.1 Roles

2.2 Decision Cadence

2.3 Runtime Flow

intent detected
  -> impact and sensitivity classified
  -> evidence state assessed
  -> policy boundary checked
  -> model / retrieval / tool confidence aggregated
  -> action class selected
  -> approved language applied
  -> handoff or answer delivered
  -> evidence packet recorded
  -> monitoring and learning loop updated

2.4 Minimum Viable Control Plane

---

3. Template: Uncertainty Taxonomy

Design rule:

Every uncertainty class must map to:
trigger -> action class -> message -> owner -> evidence -> metric.

---

4. Template: Abstention Rule

Use this table as a design artifact. Rows below are concrete examples ready to adapt into a control library.

4.1 Rule Design Pattern

rule_id: UXR-003
scenario: payment_dispute
trigger:
  intent: dispute_outcome_prediction
  evidence_state: merchant_evidence_missing
  customer_impact: high
decision:
  action_class: partial_answer_ask_more
  prohibited_outputs:
    - promise_success
    - assign_fault_without_investigation
  allowed_outputs:
    - explain_process
    - request_missing_evidence
    - create_case
message:
  customer_language_id: dispute_missing_evidence_v4
owner:
  business: dispute_operations
  control: payments_compliance
evidence:
  required:
    - transaction_ref
    - dispute_policy_version
    - missing_evidence_list
monitoring:
  tags:
    - high_impact
    - partial_answer
    - evidence_missing

---

5. Template: Escalation Matrix

Escalation quality test:

• The customer is not forced to restart the story.
• The employee receives evidence, not just a risk label.
• The receiving team has authority to act.
• SLA starts and closes in a system of record.
• The AI trace links to the case without overexposing sensitive data.

---

6. Template: Confidence Language Guide

6.1 Customer-Facing

6.2 Employee-Facing

6.3 Governance-Facing

---

7. Template: Customer Communication Test

Run every customer-facing uncertainty message through this test before release.

Example revised message:

I can explain the dispute process and record the facts you provide. I cannot promise the outcome because the merchant evidence and network review are not complete. Please upload the cancellation confirmation or tell me the date you contacted the merchant. If this transaction is fraud or is causing immediate hardship, I can route it to a specialist now.

Why it passes:

• It gives scope.
• It refuses outcome promise.
• It asks for specific evidence.
• It provides escalation triggers.
• It avoids raw confidence and vague disclaimer language.

---

8. Template: Evidence Packet

Each uncertainty decision should create a compact event that can be joined to conversation, case, model, policy and workflow records.

Concrete JSON-style example:

{
  "uncertainty_event_id": "uxr-2026-06-30-169-001",
  "use_case_id": "kyc_document_extraction",
  "intent": "extract_identity_fields",
  "impact_tier": "high",
  "abstention_class": "evidence_conflict",
  "action_class": "field_level_review",
  "evidence_state": {
    "source_refs": ["doc_front_image_ref", "application_address_ref"],
    "contradictions": ["address_mismatch"],
    "missing_fields": []
  },
  "confidence": {
    "name_field": "high",
    "dob_field": "high",
    "address_field": "low"
  },
  "policy_decision_id": "kyc_field_abstention_v3",
  "approved_language_ids": ["kyc_address_review_message_v2"],
  "handoff": {
    "escalation_required": true,
    "queue": "kyc_review",
    "handoff_reason": "document_application_address_mismatch"
  },
  "monitoring_tags": ["high_impact", "field_level_abstention", "manual_review"]
}

---

9. PM / BA / Architecture Questions

9.1 PM Questions

9.2 BA Questions

9.3 Architecture Questions

---

10. Release Checklist

10.1 Product And Policy

10.2 Architecture And Operations

10.3 Eval And Monitoring

10.4 Governance

---

11. Scenario Design Packs

11.1 Credit Eligibility

11.2 Wealth Advice Boundary

11.3 Payment Dispute Claims

11.4 AML Analyst Assistant

11.5 KYC Document Extraction

11.6 Complaints

11.7 Contact Center

---

12. Executive Narrative

Executive message:

We are not launching AI that simply says "I may be wrong."
We are launching a decision-control architecture that knows when to answer,
when to ask for more evidence, when to refuse, when to route to specialists,
and how to prove the decision later.

Why it matters:

• Customer trust improves when uncertainty is specific and paired with a next step.
• Customer harm decreases when high-impact, low-evidence and regulated-boundary cases are not guessed.
• Operations improve when handoff includes evidence, owner, reason and SLA.
• Compliance improves when approved language and policy ids are enforced at runtime.
• Model risk improves when uncertainty behavior is measured, not hidden.
• Executive governance improves when incidents, complaints and overrides feed back into CAPA.

Board-level framing:

---

13. Interview Drills

Drill 1: Explain The Difference Between Confidence And Evidence

Strong answer:

Confidence is a signal about model or retrieval behavior. Evidence is what supports a specific claim in a specific customer context. In finance, a high model confidence answer can still be unsafe if the source is stale, the policy boundary forbids the claim, the user lacks authorization, or the action affects customer rights. I separate model confidence, evidence confidence, policy certainty and impact severity, then use those signals to choose answer, partial answer, ask-more, refusal or escalation.

Drill 2: Design Uncertainty UX For Payment Disputes

Strong answer:

I would not show a probability that the customer will win. The AI can explain the dispute process, collect facts, identify missing evidence and create a case. It should abstain from outcome promises and escalate fraud, hardship, high-value, repeat or complaint cases. The handoff packet should include transaction refs, customer narrative, evidence gaps, policy version and AI uncertainty reason.

Drill 3: Handle Wealth Advice Boundary

Strong answer:

If the user asks whether to buy or sell a product, uncertainty is not just technical. It is a licensing, suitability and conduct boundary. The AI should provide neutral education, disclose that personal recommendations require an authorized channel, and offer a licensed advisor handoff. It should log the intent, channel, profile freshness and boundary policy.

Drill 4: CTO Architecture Question

Strong answer:

I would create a policy decision service that receives intent, impact tier, evidence state, authorization, confidence signals, tool status and reversibility. It returns an action class and copy constraints. The LLM does not independently decide regulated refusal or escalation. Each event writes an evidence packet with model, RAG source, policy id, output hash and handoff status. Monitoring then tracks abstention class, escalation precision, override, complaint and segment friction.

Drill 5: Governance Lead Question

Strong answer:

I would require every customer-impacting uncertainty event to be replayable. That means source refs, policy versions, approved language ids, model/tool versions, action class, handoff route, human override and downstream complaint or appeal link. Governance reviews should look at not only model accuracy, but also unsupported claims, wrong abstentions, missed escalations, customer harm and CAPA closure.

---

14. Portfolio Summary

AI uncertainty architecture is the difference between a fluent assistant and a governed service system.

The mature design question is not:

How confident is the model?

The mature design question is:

Given this user, intent, evidence, policy boundary, impact level and operational capacity,
what is the safest useful next action, what should we say, who owns it,
and what evidence proves we handled it correctly?

That question is where advanced PM, BA, architecture and AI governance work meet.