AI 底层逻辑 / 经典论文

AI Human Factors Operations：认知负载与自动化偏差架构

435 行ai-foundations/papers/165-ai-human-factors-operations-cognitive-load-automation-bias-architecture.md

AI 人因运营架构：Cognitive Load / Automation Bias / Calibrated Trust Architecture

Date: 2026-06-30 Status: evergreen Audience: experienced CBAP / financial retail PM / AI product architect / enterprise architect / operations lead / model risk partner Output: advanced architecture note, operating model, control design, ADR draft, interview-ready narrative

1. Why Human Factors Are Architecture, Not Just UX

AI human factors often get reduced to "make the interface clearer" or "add a human review step." That framing is too shallow for financial retail operations. In AML, credit, fraud, complaints, collections and contact centers, the human is not a decorative fallback. The human is a scarce decision resource inside a production control system.

Human factors become architecture because they shape:

Architecture concern	Human factor	Why it matters in production
Throughput	case volume, handling time, fatigue, task switching	A review queue that exceeds cognitive capacity becomes a rubber stamp or backlog.
Risk control	automation bias, alert fatigue, anchoring, over-trust	A reviewer who accepts AI output without challenge is not an effective control.
Decision rights	who may accept, edit, override, escalate, stop or approve	Regulated decisions require accountable authority, not just a button in the UI.
Evidence	what the operator saw, changed, ignored and relied on	Audit needs reconstructable control evidence, not only model logs.
Quality	reviewer calibration, second-line QA, disagreement handling	Human judgment quality drifts just like model quality.
Workload routing	skill, risk, language, customer vulnerability, deadline	The right work must reach the right human at the right time.
Trust	calibrated reliance, skepticism, confidence and recoverability	Trust must match evidence strength, task risk and action reversibility.

In a financial retail AI system, the architecture question is not:

Can we put a human in the loop?

The senior question is:

Can the operating architecture preserve independent human judgment under real workload,
while proving that the human control reduced risk instead of becoming control theater?

This note deliberately avoids repeating generic Human-AI Interaction principles or Team Topologies cognitive load language. The focus is operational architecture: operator burden, review fatigue, automation bias, calibrated trust, escalation design, second-line QA, sampling, error cost, workload routing, skill matrix, training loops, decision rights and control evidence.

2. Concept Diagram

flowchart TB
  Intake[Case intake<br/>customer, alert, application, complaint, call] --> Classifier[Task, risk, impact and reversibility classifier]
  Classifier --> Workload[Operator load estimator<br/>volume, AHT, skill, fatigue, SLA]
  Classifier --> Assist[AI assistance layer<br/>RAG, copilot, agent, model score]
  Assist --> Evidence[Evidence bundle<br/>sources, tool trace, policy, confidence limits]
  Workload --> Route[Workload and skill router]
  Evidence --> BiasCtl[Automation bias controls<br/>blind pass, reason codes, friction, challenge prompts]
  BiasCtl --> Route
  Route --> Workspace[Operator workspace<br/>task, evidence, AI output, controls]
  Workspace --> Decision{Human decision}
  Decision -->|Accept or edit| Action[Downstream action<br/>reply, approve, freeze, close, escalate]
  Decision -->|Override| Override[Override governance<br/>reason, authority, evidence]
  Decision -->|Escalate| Escalation[Escalation path<br/>SME, compliance, second line, supervisor]
  Decision -->|Stop route| SafeStop[Safe stop<br/>pause automation or queue]
  Action --> Ledger[Evidence ledger<br/>trace, version, operator action, timing]
  Override --> Ledger
  Escalation --> Ledger
  SafeStop --> Ledger
  Ledger --> QA[Second-line QA and calibration]
  QA --> Metrics[Metrics and control dashboard]
  Metrics --> Improve[Training, prompt, RAG, workflow and policy improvement]
  Improve --> Assist
  Improve --> Route

Architecture interpretation:

The AI layer is only one part of the operating architecture.
Bias control sits before and inside the reviewer workspace, not only in training material.
Queue routing must consider cognitive load and skill, not just FIFO order.
QA and calibration are production feedback loops, not one-time launch activities.
Evidence ledger is the connective tissue across product, model risk, compliance, audit and operations.

3. Operating Architecture Model

3.1 Architecture Layers

Layer	Core responsibility	Design decision
Work intake	Convert events into reviewable work units	Define whether the unit is a claim, draft, recommendation, tool action, case, alert or sampled outcome.
Risk and impact classifier	Determine customer impact, financial impact, regulatory sensitivity and reversibility	Use risk tier and error cost to drive routing, review depth and escalation.
Cognitive load manager	Estimate operator burden and fatigue risk	Track queue size, average handling time, interruption rate, context switching, active hours and case complexity.
AI assistance layer	Generate summaries, recommendations, drafts, evidence retrieval, next-best action or tool proposals	Separate AI evidence, AI reasoning summary, model score, system-of-record facts and policy constraints.
Bias and trust control plane	Reduce over-reliance and under-reliance	Design blind review, challenge prompts, no default accept, confidence explanation and required evidence checks.
Workload router	Match work to skill, authority, independence and capacity	Route by domain, product, language, risk, customer vulnerability, deadline and conflict-of-interest rules.
Operator workspace	Give the human enough context and control to make a defensible decision	Show evidence first, expose missing data, structure actions and capture reason codes.
Decision rights layer	Decide who may accept, edit, override, approve, escalate or stop	Tie permissions to role, skill certification, risk tier and policy authority.
QA and calibration	Detect drift in human judgment and AI reliance	Use gold cases, blind samples, second review, adjudication and reviewer coaching.
Evidence and observability	Record what happened and why	Capture input, output, evidence version, model/prompt/version, action, reason, trace, timings and reviewer identity.
Governance loop	Convert operational signals into control improvements	Review trends, incidents, audit findings, training gaps, policy changes and release gates.

3.2 Review Unit Taxonomy

Review unit	Financial retail example	Human factor risk	Architecture implication
Claim	"This fee can be waived under policy X."	Operator may trust fluent unsupported claim.	Require source-linked claim verification.
Draft	Complaint response letter or hardship script	Reviewer skims language and misses commitment risk.	Highlight obligations, promises, policy citations and prohibited phrases.
Recommendation	AML alert close, credit approve, fraud block	AI recommendation anchors the human.	Use blind first-pass or evidence-first design for high-impact cases.
Tool action	Refund, freeze, close case, send notice	One click changes customer state.	Require authority, preview, confirmation, trace and reversible design where possible.
Sampled outcome	Auto-handled contact center answer	Sample misses minority error segments.	Use stratified and sentinel sampling, not only random QA.
Escalation	PEP match, legal threat, vulnerable customer	Human may route late due to workload pressure.	Define escalation triggers, SLA, stop rules and destination ownership.

3.3 Decision Rights Matrix

Risk tier	AI role	Human role	Required control
P0: irreversible, legal, regulatory, material customer harm	Summarize evidence and propose options only	Authorized human decides; second-line or specialist may approve	Mandatory review, no default accept, explicit reason, escalation trace
P1: high impact but controllable	Recommend with evidence and uncertainty	Skilled operator accepts, edits, rejects or escalates	Evidence checklist, reason code, QA sample and override monitoring
P2: customer-visible but reversible	Draft or answer with source support	Frontline reviews or risk-based samples	Citation support, feedback, random QA and recovery path
P3: internal productivity	Assist task completion	Operator accountable for use	Periodic QA, training loop and telemetry
P4: learning and experimentation	Shadow output	No production decision	Offline eval, calibration cases and model comparison evidence

3.4 Skill Matrix

Skill dimension	Why it matters	Evidence of readiness
Domain knowledge	AML, credit, fraud, complaints and collections have different risk rules.	Certification record, gold case score, supervisor sign-off
Policy interpretation	Operators must identify policy conflict and exception boundaries.	Scenario test, policy quiz, audit sample
AI literacy	Operators must recognize hallucination, retrieval gaps, false confidence and tool misuse.	Training completion plus challenge-case performance
Evidence handling	Reviewers need to know which sources are authoritative and which are weak.	Evidence rubric score and citation sufficiency rate
Customer impact judgment	Same error has different cost for hardship, vulnerable customer or adverse action.	Escalation accuracy and severity calibration
Decision authority	Some actions need senior approval or independent review.	Role-based entitlement and authority matrix
Communication quality	Customer-visible outputs require clear, compliant, empathetic language.	QA language sample and complaint trend

4. Cognitive Load And Automation Bias Controls

4.1 Cognitive Load Control Model

Operator load is not only the number of cases. It is the mental work required to understand evidence, challenge AI, decide under uncertainty, document the decision and recover from exceptions.

operator_load =
  case_complexity
+ evidence_volume
+ evidence_conflict
+ policy_ambiguity
+ customer_impact
+ interruption_rate
+ context_switching
+ time_pressure
+ UI_navigation_cost
+ documentation_burden
- task_chunking
- evidence_prioritization
- skill_match
- workflow automation
- escalation clarity

Load driver	Architecture control	Financial retail example
Evidence sprawl	Evidence bundle with ranked sources, freshness, authority and missing fields	AML investigator sees transaction cluster, SAR history, customer profile and typology in one bundle.
Policy ambiguity	Policy conflict detector and escalation rule	Collections hardship script flags conflict between temporary forbearance policy and state-specific rule.
High context switching	Queue batching by domain, product and case type	Contact center QA reviewers handle complaint drafts in blocks instead of mixing fraud, credit and servicing.
Time pressure	SLA-aware routing with surge mode	Fraud interventions route real-time blocks separately from batch post-event QA.
Documentation burden	Structured reason codes plus short free-text rationale	Credit underwriter records adverse action rationale without retyping the whole memo.
Fatigue	shift limits, complexity caps, break triggers and reviewer rotation	High-risk AML queue limits consecutive complex cases and rotates to calibration work.
Alert overload	risk-based triage and sampled low-risk verification	Fraud false-positive stream gets stratified QA while high-risk account takeover gets immediate review.

4.2 Automation Bias Controls

Automation bias means operators give excessive weight to AI output because it is fluent, confident, convenient, faster or socially endorsed by management. In production it shows up as one-click accept, low override rates, shallow evidence review, declining escalation and reduced error detection.

Bias pattern	Control	Architecture implementation	Evidence
Anchoring on AI recommendation	Evidence-first or blind first-pass review for P0/P1	Hide AI recommendation until reviewer marks evidence sufficiency or preliminary risk tier.	UI event sequence, preliminary decision, final decision delta
Default acceptance	No preselected accept action	Require active accept, edit, reject or escalate selection with reason code.	Action log and reason-code distribution
Confidence theater	Explain confidence source and limit	Separate model confidence, retrieval support, policy certainty and data completeness.	Confidence component telemetry and QA findings
Speed pressure	Balanced metrics	Score throughput together with quality, override validity, escalation accuracy and missed-risk rate.	Operations dashboard and performance scorecard
Reviewer fatigue	Load-aware routing	Throttle queue, cap complex cases, route surge and flag fatigue risk.	Workload trace and shift-level quality trend
AI social proof	Independent challenge prompt	Ask "What evidence would make this recommendation wrong?" before final accept.	Challenge response captured in high-risk cases
Shallow review	Mandatory evidence checklist	Require source opening, key field confirmation or missing-evidence acknowledgement for high-impact tasks.	Evidence interaction log
Over-correction or under-trust	Calibration and gold cases	Train reviewers on cases where AI is right, wrong, partially right and unsupported.	Calibration score and drift trend
Blind spots in sampling	Sentinel and stratified QA	Include known tricky cases, edge segments, languages, products and customer vulnerability markers.	QA sample design and hit rate

4.3 Calibrated Trust Design

Trust calibration is the match between reliance and actual capability under the current task, evidence and risk condition.

Trust state	Symptom	Control
Over-trust	Accept rate rises while evidence-open rate drops.	Evidence-first review, reason-code friction, second-line QA and management metric reset.
Under-trust	Operators ignore useful AI and duplicate all work manually.	Training with model strengths, clear source support, workflow integration and feedback response.
Mis-trust	Operators trust AI for the wrong tasks, such as policy exceptions or adverse action language.	Scope boundaries, task-specific affordances and prohibited-use controls.
Calibrated trust	Reliance varies by evidence strength, risk tier and reversibility.	Confidence decomposition, risk-tiered workflow, QA sampling and continuous calibration.

5. Financial Retail Scenarios

5.1 AML Investigator Copilot

Dimension	Architecture design
AI assistance	Summarizes alerts, clusters transactions, retrieves prior SAR narratives, identifies typology matches and drafts investigation notes.
Operator burden	High evidence volume, fragmented systems, deadline pressure and repetitive narrative writing.
Automation bias risk	Investigator accepts AI "close as false positive" recommendation because the summary looks complete.
Controls	Evidence-first review, mandatory typology evidence, reason code for close, senior review for high-risk customer, sentinel QA for false-negative patterns.
Metrics	evidence-open rate, close override rate, SAR escalation precision, QA miss rate, backlog age, alert fatigue index.
Control evidence	alert trace, sources used, model version, investigator action, close rationale, second-line sample and calibration outcome.

5.2 Credit Underwriter Assist

Dimension	Architecture design
AI assistance	Builds credit memo, flags missing documents, summarizes income, debt, collateral and policy exceptions.
Operator burden	Policy interpretation, fair lending sensitivity, adverse action reason quality and exception handling.
Automation bias risk	Underwriter over-relies on model score and misses contradictory evidence or prohibited variable proxy.
Controls	Independent policy checklist, adverse action reason validation, feature/proxy warning, second review for exceptions, fair lending sample.
Metrics	policy exception accuracy, adverse action defect rate, override validity, protected-class proxy investigation rate, QA disagreement.
Control evidence	memo version, evidence references, reason code, reviewer authority, override rationale, second-line QA and model risk sign-off sample.

5.3 Contact Center Agent Assist

Dimension	Architecture design
AI assistance	Real-time answer suggestions, account status lookup, call summary, next best action and knowledge retrieval.
Operator burden	Simultaneous listening, reading, compliance scripting, empathy and system navigation.
Automation bias risk	Agent reads a suggested answer without checking source or customer context.
Controls	concise evidence cards, prohibited phrase detection, customer vulnerability escalation, real-time fallback, sampled call QA.
Metrics	suggestion acceptance with source-open rate, handle time, transfer rate, complaint after contact, script compliance, correction rate.
Control evidence	call segment, suggestion, source link, agent edit, customer-facing text, transcript marker and QA result.

5.4 Complaints Copilot

Dimension	Architecture design
AI assistance	Classifies complaint type, extracts allegations, drafts acknowledgement and response, tracks deadlines.
Operator burden	Regulatory deadlines, legal language, emotional context, root-cause analysis and remediation tracking.
Automation bias risk	Reviewer accepts a polished draft that under-admits issue severity or misses required rights language.
Controls	deadline-first queue, complaint severity checklist, legal/compliance escalation trigger, evidence sufficiency gate, final response QA.
Metrics	response defect rate, missed allegation rate, deadline breach, escalation accuracy, customer reopen rate.
Control evidence	complaint taxonomy, allegation map, evidence bundle, draft edits, approval chain, customer communication version.

5.5 Fraud Intervention

Dimension	Architecture design
AI assistance	Scores account takeover risk, recommends block or step-up, drafts customer outreach, explains signals.
Operator burden	Time-critical decision, false-positive customer friction, fraud loss exposure and live-channel pressure.
Automation bias risk	Operator accepts high fraud score without considering customer travel or recent verified behavior.
Controls	reversible action preference, signal decomposition, customer contact path, real-time supervisor escalation for high-loss cases.
Metrics	false positive rate, fraud loss prevented, customer friction rate, block reversal rate, decision latency.
Control evidence	model score components, tool action request, human approval, customer verification status, downstream account action.

5.6 Collections Hardship

Dimension	Architecture design
AI assistance	Identifies hardship indicators, suggests available programs, drafts empathetic scripts and repayment options.
Operator burden	Emotional labor, policy exceptions, vulnerability signals and jurisdictional constraints.
Automation bias risk	Agent follows a repayment recommendation that is inappropriate for the customer's hardship status.
Controls	vulnerability-first routing, affordability evidence checklist, prohibited pressure language detection, supervisor escalation for complex hardship.
Metrics	hardship identification rate, complaint rate, script compliance, repayment plan suitability, customer repeat contact.
Control evidence	hardship signal, program eligibility facts, script version, agent edits, customer consent and supervisor review where applicable.

6. Metrics, Control And Evidence Model

6.1 Metric Families

Metric family	Example metrics	What it detects
Operator load	cases per hour, average handling time, queue age, interruption rate, complex-case streak, after-hours work	overload, fatigue, unsustainable control design
Evidence behavior	source-open rate, missing-evidence acknowledgement, citation support, policy conflict review	shallow review and weak evidence use
Automation reliance	accept rate, edit depth, override rate, blind-pass delta, AI-human disagreement	over-trust, under-trust and anchoring
Quality	QA defect rate, second-review disagreement, missed escalation, gold-case score, calibration drift	human judgment drift and training gaps
Customer impact	complaint reopen, adverse outcome defect, fraud false positive, collections complaint, contact center repeat contact	harm and recovery quality
Control operation	mandatory review completion, SLA breach, escalation timeliness, authority violations, safe-stop activation	whether controls actually operated
Improvement loop	defect closure time, knowledge update cycle time, eval case creation, retraining trigger response	whether learning loops are real

6.2 Control Evidence Packet

Evidence object	Required content	Audit question it answers
Work item record	case id, risk tier, source channel, customer impact, SLA	Why did this item enter this workflow?
AI trace	model, prompt, RAG query, retrieved sources, tool calls, confidence components	What did the AI use and produce?
Evidence bundle	authoritative sources, timestamps, policy versions, missing evidence flags	What evidence was available to the human?
Operator interaction log	evidence opened, AI output viewed, edits, time on task, action selected	Did the human perform meaningful review?
Decision record	final action, reason code, authority, override or escalation rationale	Who decided what and why?
QA sample	sample frame, reviewer independence, result, defect class, adjudication	How was control quality tested?
Training record	role, skill certification, calibration score, retraining completion	Was the human qualified for this task?
Governance record	issue owner, remediation, release gate, residual risk acceptance	Was the operating risk managed?

6.3 Sampling Model

Sampling must align with error cost, not just volume.

Sampling type	Use	Example
100 percent review	irreversible or high-impact actions	credit adverse action reason, account freeze, formal complaint final response
Risk-based sample	model score or workflow signal is reliable but not complete	fraud interventions below threshold but with unusual geography
Stratified sample	risk varies by product, channel, language, region or customer segment	contact center agent assist answers across English, Spanish and vulnerable-customer signals
Sentinel sample	known difficult or high-risk cases	AML typology edge cases, policy conflicts, tricky hardship conversations
Blind second review	detect anchoring and groupthink	credit memo recommendations and AML alert closure decisions
Incident surge sample	production defect, policy change or model drift	stale RAG policy discovered in complaints response drafting

6.4 Control Threshold Examples

Signal	Threshold	Action
Evidence-open rate for P1 cases below 90 percent	two consecutive business days	supervisor review, targeted coaching and UI friction increase
AI accept rate above 95 percent with low edit depth	weekly trend	automation bias investigation and blind sample expansion
QA defect rate above 3 percent for customer-visible drafts	rolling two-week sample	release rollback review and knowledge/prompt correction
Escalation rate drops by 50 percent after AI launch	monthly comparison	check for suppressed escalations and revise incentives
Queue age breaches SLA for high-risk work	same day	surge staffing, intake throttling or safe-stop rule
Gold-case calibration below 85 percent	per reviewer or team	restrict high-risk queue access until recalibrated

7. Anti-Patterns And Failure Modes

Anti-pattern	Why it looks attractive	Failure mode
"Human in the loop" as a single approval step	Easy to explain to executives and auditors	Human lacks time, skill, evidence or authority to challenge AI.
All high-risk cases to one queue	Appears conservative	Queue overload causes delay, missed deadlines and superficial review.
Default accept button	Improves handling time	Creates anchoring and rubber-stamping.
One confidence badge	Simple UI	Hides whether confidence comes from model score, retrieval support, policy certainty or data completeness.
Throughput-only productivity target	Shows ROI quickly	Encourages shallow review and suppressed escalation.
Low override rate celebrated	Looks like AI quality	May indicate automation bias or fear of challenging the system.
Training sign-off only	Easy compliance artifact	Does not prove operators can detect difficult failures.
Random QA only	Statistically neat	Misses rare high-cost cases and minority segment harms.
AI output above evidence	Feels efficient	Reviewers read conclusion first and search for confirming evidence.
No safe-stop authority	Keeps automation running	Operators cannot stop a harmful route during incident conditions.
Hidden AI assist	Avoids customer concern	Makes responsibility, disclosure, audit and root cause unclear.
Control evidence scattered across tools	Avoids integration cost	Audit cannot reconstruct why a decision happened.

8. Architecture Mapping To RAG / Agent / Copilot / Eval / Governance

Architecture pattern	Human factors risk	Required architecture move	Evidence
RAG	Unsupported or stale retrieved content becomes fluent advice.	Source authority ranking, evidence sufficiency gate, citation support, policy freshness monitoring.	retrieval trace, source version, citation QA, stale-source incident log
Agent	Tool actions bypass careful human decision or increase time pressure.	Action policy engine, pre-action review, reversible action preference, authority check and safe-stop.	tool call proposal, approval trace, action result, rollback record
Copilot	Operator accepts drafts without reading or editing.	Evidence-first layout, no default accept, edit tracking, challenge prompt and QA sampling.	accept/edit metrics, source-open rate, draft diff, QA defect class
Eval	Offline accuracy hides workload and automation bias.	Add human factors evals: review time, disagreement, evidence use, missed escalation and calibration.	eval set, reviewer protocol, inter-rater agreement, production comparison
Governance	Policies exist but do not operate at runtime.	Connect risk tier, decision rights, training, QA, sampling and observability to release gates.	RACI, control matrix, release approval, dashboard, management review
Observability	Model logs do not show human burden or review quality.	Instrument workflow traces, operator events, queue metrics and evidence ledger.	OpenTelemetry trace ids, queue dashboard, audit packet
Data product	Human corrections do not improve knowledge or model behavior.	Structured feedback taxonomy, defect owner, knowledge update SLA and eval case creation.	feedback log, correction ticket, updated corpus, regression eval

9. ADR Draft

ADR-165: Adopt A Human Factors Operations Control Plane For High-Impact AI Workflows

Field	Decision
Status	Proposed for portfolio architecture review
Context	Financial retail AI use cases rely on human operators to review AML narratives, credit memos, fraud interventions, complaint responses, contact center suggestions and hardship scripts. Existing HITL patterns do not adequately manage cognitive load, automation bias, calibrated trust, decision rights, QA sampling and audit evidence.
Decision	Implement a human factors operations control plane across high-impact AI workflows. The control plane includes risk-tiered work intake, cognitive load estimation, skill-based routing, evidence-first reviewer workspace, automation bias controls, decision-right enforcement, second-line QA, sampling, calibration, training loops and evidence ledger.
Drivers	Customer harm prevention, regulatory defensibility, operational resilience, reviewer capacity, model risk management, audit replay, production incident response and executive accountability.
Selected option	Central control-plane pattern integrated with workflow orchestration, AI gateway, reviewer workspace, QA tooling and observability.
Alternatives considered	Local UI-only warnings; generic human approval step; post-hoc QA without runtime routing; full automation with exception review only.
Why selected	The selected option treats human judgment as a managed production capacity and provides runtime controls plus evidence. It reduces the risk that human review becomes a bottleneck or rubber stamp.
Consequences	Requires instrumentation, reviewer training, authority matrix, workflow changes, QA operations and management reporting. It may reduce short-term automation ROI but improves sustainable adoption and defensibility.
Scope	AML, credit, fraud, complaints, collections hardship, contact center agent assist and any AI workflow with customer-visible or customer-impacting output.
Non-goals	This ADR does not approve a specific model, vendor, regulatory interpretation or legal position. It defines the architecture pattern for human factors operations.

Acceptance Criteria

Criterion	Evidence
Every high-impact AI workflow has a review unit definition and risk tier.	workflow catalog and risk-tier map
Review routing uses skill, authority, capacity and independence rules.	routing configuration and role matrix
Reviewer workspace exposes evidence, uncertainty, missing data and allowed actions.	UI review checklist and trace sample
Automation bias controls are implemented for P0/P1 tasks.	blind pass logs, no-default-accept proof, reason-code logs
QA sampling covers risk, volume, segments and sentinel cases.	sampling plan and QA report
Training and calibration affect queue eligibility.	certification records and access control linkage
Control evidence can be replayed for a sample decision.	evidence packet and audit replay script

10. Interview Answer

30秒版本

AI human factors 不是 UI 问题，而是生产控制架构问题。金融零售里，人类审核承担的是风险吸收和最终判断，但人的注意力、技能、疲劳和自动化偏差都是有限资源。我会把它设计成 human factors operations control plane：按风险分层、估算负荷、技能路由、证据优先、去默认采纳、设置升级和二线 QA，并记录完整 evidence。这样才能证明人不是橡皮图章，而是真正降低客户伤害和模型风险的控制。

2分钟版本

我会先定义 review unit，比如 AML alert close、credit memo、complaint response draft、fraud block request 或 contact center answer。然后按客户影响、监管敏感性、可逆性和错误成本分层。不同层级对应不同的 AI 角色和人类决策权：低风险可以抽样 QA，高风险需要证据优先、强制 reason code、无默认 accept、必要时 blind review 或 second review。

架构上我会建立四类能力。第一是 workload routing，按照技能、容量、语言、产品、风险和 deadline 路由，避免把所有高风险 case 堆到一个队列。第二是 automation bias control，比如先看证据再看 AI 建议、要求 reviewer 标记证据是否充分、记录 override 和 edit depth。第三是 calibrated trust，用 model confidence、retrieval support、policy certainty 和 data completeness 分开展示，不做一个虚假的信心徽章。第四是 evidence ledger，把模型版本、prompt、检索来源、工具调用、人类动作、reason code、QA 结果和下游影响串起来。

在面试里我会强调：human-in-the-loop 不等于控制有效。控制是否有效，要看 reviewer 是否有时间、技能、证据、独立性、升级权和可审计记录。否则它只是合规幻觉。

CTO版本

我会把 human factors 作为 AI platform control plane 的一部分，而不是每个产品团队自己加提示语。平台层提供 risk-tier classifier、review policy engine、skill/capacity router、evidence bundle service、decision-right enforcement、QA sampling service 和 trace/evidence ledger。业务线配置任务、风险、权限和 SLA。

技术上要把 AI gateway、RAG provenance、agent tool policy、workflow engine、IAM、observability 和 QA 数据模型打通。OpenTelemetry-style trace id 贯穿 case、retrieval、model invocation、tool proposal、human action 和 downstream system update。治理上用 NIST AI RMF 的 Govern / Map / Measure / Manage 组织风险闭环，用 ISO/IEC 42001 的管理体系语言定义责任、能力、运营控制、绩效评价和持续改进。

我不会只承诺 "we have human review." 我会要求能回答三个 CTO 级问题：生产高峰时 review control 是否还能运行？发生客户伤害时能否重放证据链？AI 提升效率是否以削弱人工判断为代价？这三个问题答不上来，AI 系统就还没有准备好扩大自动化范围。

11. 7-Day Practice Plan

Day	Practice	Output
1	Pick one workflow: AML copilot, credit assist, contact center assist, complaints, fraud or collections hardship. Define review unit, risk tiers and error-cost ladder.	one-page review unit map
2	Build an operator load map with volume, average handling time, evidence volume, policy ambiguity, interruption rate and fatigue triggers.	workload and capacity table
3	Design automation bias controls for P0/P1/P2 tasks, including blind pass, no default accept, reason codes and challenge prompts.	automation bias control matrix
4	Create a skill and decision-right matrix for frontline, specialist, supervisor, compliance and second-line QA roles.	authority and routing matrix
5	Draft a QA sampling plan with 100 percent review, risk-based sample, stratified sample, sentinel cases and incident surge sampling.	QA sampling plan
6	Define evidence packet fields and observability trace across AI output, retrieved sources, human action and downstream result.	evidence ledger schema
7	Prepare interview narrative and ADR summary. Practice answering as PM, architect and CTO.	30-second, 2-minute and CTO answer

12. Source Anchors

These anchors are used as architecture and operating model references. They are not legal, compliance, audit or model validation advice. Access date: 2026-06-30.

Anchor	Link	How this note uses it
NIST AI Risk Management Framework	https://www.nist.gov/itl/ai-risk-management-framework	Uses Govern, Map, Measure and Manage as the lifecycle for human factors risk identification, monitoring, treatment and improvement.
NIST bias publication	https://www.nist.gov/blogs/taking-measure/powerful-ai-already-here-use-it-responsibly-we-need-mitigate-bias	Anchors the need to mitigate bias beyond the model, including use context, human decision processes and deployment controls.
Microsoft Guidelines for Human-AI Interaction	https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/	Provides human-AI interaction principles that this note translates into operational controls for review, trust, escalation and recovery.
ISO/IEC 42001 AI management system	https://www.iso.org/standard/81230.html	Connects human factors controls to management system concepts: responsibility, operation, performance evaluation and continual improvement.
ISO/IEC/IEEE 42010 architecture description	https://www.iso.org/standard/74393.html	Supports treating human factors as architecture views, stakeholders, concerns, decisions and evidence, not isolated UI guidance.
OpenTelemetry Documentation	https://opentelemetry.io/docs/	Supports runtime traces, metrics and logs that connect AI output, human action, queue state and downstream impact.