AI Human Factors Operations / Cognitive Load / Automation Bias Playbook

定位: 面向高级 AI PM / AI BA / Product Architect / Enterprise Architect / Operations Lead / Model Risk / Compliance / Internal Audit, 把 AI 人因风险从“培训和提醒”升级为可设计、可路由、可度量、可审计、可持续改进的运营架构。 适用范围: AML investigator copilot、credit underwriter assist、contact center agent assist、complaints copilot、fraud intervention、collections hardship、KYC review、payment dispute、financial retail internal knowledge assistant。 重要说明: 本文是学习、作品集和内部治理训练材料, 不是法律意见、合规结论、审计意见、模型验证报告、监管解释或生产批准。正式项目必须由 Legal、Compliance、Risk、Model Risk、Internal Audit、Security、Privacy、Business Owner、Operations、Workforce Planning 和管理层结合机构类型、司法辖区、客户影响和内部政策确认。

---

1. Purpose And When To Use

1.1 Purpose

This playbook helps teams design AI operations where human attention, judgment and authority are treated as production architecture. It gives practical templates for operator load, automation bias, trust calibration, QA sampling, escalation and evidence packets.

Use it when the AI system:

• assists regulated or customer-impacting decisions;
• drafts customer-visible messages;
• recommends case closure, escalation, approval, denial, freeze, refund or outreach;
• changes workload shape for frontline or specialist operators;
• relies on human review as a risk control;
• creates risk of over-trust, under-trust, alert fatigue, review fatigue or decision anchoring;
• must produce control evidence for audit, model risk, compliance or management review.

1.2 When To Use In Delivery

1.3 Core Principle

Do not ask humans to be the control unless the system gives them time, skill,
evidence, independence, authority, escalation paths and proof that their review
changed risk outcomes.

---

2. Operating Model

2.1 End-To-End Flow

AI-assisted work item
  -> risk, impact and reversibility classification
  -> operator load estimate
  -> review policy decision
  -> skill, authority and capacity routing
  -> evidence-first workspace
  -> automation bias controls
  -> human decision
  -> escalation, override or downstream action
  -> evidence packet
  -> QA sampling and calibration
  -> training, workflow, RAG, prompt, model and policy improvement

2.2 Operating Roles

2.3 Operating Cadence

---

3. Template: Operator Load Map

Use this template before sizing AI benefits. It prevents teams from transferring effort from one role to a more expensive or fragile human control without recognizing it.

Load calculation for a release gate:

required_review_hours =
  AI_assisted_items
* review_rate
* average_handling_minutes
* complexity_multiplier
* double_review_multiplier
* rework_multiplier
/ 60

Capacity sanity check:

available_review_hours =
  reviewers
* productive_hours_per_shift
* skill_match_percentage
* attendance_factor

Release rule:

available_review_hours must exceed required_review_hours
with surge reserve for incidents, policy changes and volume spikes.

---

4. Template: Automation Bias Control Matrix

---

5. Template: Trust Calibration Script

Use this script in training, supervisor coaching and in-product microcopy. It teaches operators how to rely on AI neither too much nor too little.

---

6. Template: QA Sampling Plan

6.1 Sample Design

6.2 Sampling Formula

daily_QA_sample =
  mandatory_P0_items
+ max(risk_based_minimum, ceil(P1_volume * P1_sample_rate))
+ stratified_segment_minimums
+ sentinel_case_count
+ incident_surge_addon

Example baseline for pilot:

6.3 QA Defect Taxonomy

---

7. Template: Escalation Design

Escalation design rules:

• Escalation is not a button unless it has a destination, SLA, receiving role, decision right and evidence requirement.
• High-risk uncertainty should stop or narrow automation, not simply add a note.
• Escalation volume is a signal. A sudden drop can be as concerning as a spike.
• Escalation outcomes must feed training, eval cases, knowledge updates and release gates.

---

8. Template: Evidence Packet

Evidence packet acceptance standard:

A qualified reviewer or auditor can reconstruct the decision without interviewing
the original operator, reading private chat, or relying on memory.

---

9. PM / BA / Architecture Questions

9.1 PM Questions

9.2 BA Questions

9.3 Architecture Questions

---

10. Release Checklist

10.1 Product And Workflow

• Review unit is defined for every AI-assisted workflow.
• Risk tiers include customer impact, financial impact, regulatory sensitivity and reversibility.
• AI role is explicit: summarize, retrieve, draft, recommend, classify or propose action.
• Human role is explicit: accept, edit, reject, override, approve, escalate or safe-stop.
• User-facing and employee-facing trust messages are approved for each risk tier.
• Recovery path exists for wrong answer, wrong action, missing evidence and customer complaint.

10.2 Operations And Capacity

• Operator load map includes volume, AHT, skill, complexity, fatigue and surge assumptions.
• Capacity exceeds required review hours with incident reserve.
• Skill routing covers domain, product, language, risk, authority and independence.
• Reviewer training includes AI failure modes, not only screen usage.
• Calibration uses gold cases and affects eligibility for high-risk queues.
• Queue dashboard includes backlog, SLA, complexity, fatigue and quality metrics.

10.3 Bias And Trust Controls

• P0/P1 workflows remove default accept.
• P0/P1 workflows use evidence-first or blind first-pass where anchoring risk is high.
• Confidence is decomposed into model, retrieval, policy and data-completeness signals.
• Reason code and evidence reference are required for high-impact accept, edit, reject, override or escalation.
• Management scorecards balance productivity with quality, escalation and customer impact.
• Sentinel cases and blind second reviews are active before expansion.

10.4 Architecture And Evidence

• AI gateway captures model, prompt, input, output, confidence components and policy decision.
• RAG layer captures source id, version, freshness, authority and citation support.
• Tool gateway captures parameters, permissions, result and downstream side effect.
• Reviewer workspace captures evidence opened, action, edit diff, reason and time on task.
• Shared trace id connects AI output, human action and downstream system.
• Evidence packet can be replayed by QA or audit without relying on memory.

10.5 Governance

• Risk, compliance, model risk, operations and business owner reviewed the control design.
• QA sampling plan covers risk-based, stratified, sentinel and incident surge samples.
• Escalation paths have destination, SLA, authority and communication rule.
• Safe-stop criteria are documented and tested.
• Residual risk and expansion criteria are approved by accountable owners.
• Post-release review date and dashboard owner are assigned.

---

11. Executive Narrative

11.1 One-Minute Executive Version

We should not describe this release as simply adding a human in the loop. The real control is whether qualified people can challenge AI under production workload. This playbook designs the human side as an operating architecture: risk-based routing, workload capacity, evidence-first review, automation bias controls, second-line QA, escalation rights and audit evidence.

The business value is sustainable automation. We can reduce manual effort where AI is reliable, but we avoid shifting hidden work to specialists or creating a rubber-stamp review queue. The release gate should ask three questions:

1. Can operators handle the expected volume without fatigue-driven quality loss?
2. Can they see enough evidence and authority boundaries to challenge AI?
3. Can we prove through trace and QA that human review actually reduced risk?

11.2 Board / Audit Committee Version

The AI system relies on human oversight for customer-impacting workflows. Management has designed oversight as a measurable production control, not a general assurance statement. Controls include risk-tiered review, skill and authority routing, evidence-first workspaces, explicit override and escalation rights, QA sampling, calibration, training and traceable evidence packets.

Management will monitor operator load, automation reliance, evidence use, QA defects, customer impact and escalation performance. Expansion decisions will depend on control performance, not only efficiency or adoption metrics.

11.3 Product Portfolio Version

For the portfolio, this pattern becomes reusable across AML, credit, fraud, complaints, collections and contact center AI. Each use case configures its own risk tiers, skill matrix, evidence packet and QA sample, while the platform provides common trace, routing, reviewer actions, feedback taxonomy and governance dashboards.

---

12. Interview Drills

Drill 1: "Isn't human review enough to control AI risk?"

Strong answer:

No. Human review is only effective if the human has capacity, skill, evidence,
independence, authority and escalation rights. Otherwise it becomes a bottleneck
or rubber stamp. I would design review as an operating architecture with risk-tiered
routing, evidence-first workspace, automation bias controls, QA sampling and audit trace.

Drill 2: "How would you detect automation bias in production?"

Strong answer:

I would monitor accept rate, edit depth, evidence-open rate, override validity,
blind-pass delta, escalation trend and QA defects. A very high accept rate with low
evidence interaction is not automatically good. It may indicate anchoring or pressure
to accept AI. I would add sentinel cases and blind second reviews to validate.

Drill 3: "How do you reduce cognitive load for an AML investigator copilot?"

Strong answer:

I would not just shorten the summary. I would define the review unit, rank evidence
by authority, expose missing and conflicting evidence, group related transactions,
show typology support, require close reason codes and route high-risk cases to
senior investigators. The goal is to reduce navigation and narrative burden while
preserving independent investigation.

Drill 4: "What is the difference between confidence and calibrated trust?"

Strong answer:

Confidence is a system signal. Calibrated trust is a human behavior outcome.
In financial retail I would separate model confidence, retrieval support, policy
certainty and data completeness, then observe whether operators rely more when
evidence is strong and escalate when evidence is weak or risk is high.

Drill 5: "What would you tell a CTO before scaling a copilot?"

Strong answer:

I would ask for proof that controls work under load: queue capacity, skill routing,
no-default-accept for high-risk work, evidence trace, second-line QA, safe-stop,
and production telemetry connecting AI output to human action and downstream impact.
Scaling without that proof turns human review into control theater.

---

13. Reference Anchors