AI 扩展计划 / Playbooks
AI Adoption Analytics / Behavior Change / Value Realization Playbook
AI 系统上线后，真实问题不是“有多少人点开过”，而是 AI 是否改变了受治理的工作行为，并且在质量、风险、成本和组织承载能力都可控的前提下产生了可持续净价值。Adoption analytics 不是使用量报表，而是 work-as-done baseline、行为事件、telemetry、outcome attribution、value leakage 和 scale/stop decisi
611 行AI_ADOPTION_ANALYTICS_BEHAVIOR_CHANGE_VALUE_REALIZATION_PLAYBOOK.md
AI Adoption Analytics / Behavior Change / Value Realization Playbook
配对阅读：本手册的原理/架构解读版是
docs/ai-foundations/papers/151-ai-adoption-analytics-behavior-change-value-realization-architecture.md。先读 paper 建立机制与取舍，再用本手册落地为模板、RACI 与门禁，两者不需要重复精读。
AI 系统上线后，真实问题不是“有多少人点开过”，而是 AI 是否改变了受治理的工作行为，并且在质量、风险、成本和组织承载能力都可控的前提下产生了可持续净价值。Adoption analytics 不是使用量报表，而是 work-as-done baseline、行为事件、telemetry、outcome attribution、value leakage 和 scale/stop decision 的证据体系。
1. System Boundary
| System function | 形成的证据 |
|---|---|
| Adoption governance | adoption success criteria、behavior funnel、scale/stop gates、improvement loop |
| Telemetry architecture | event contract、trace、schema、identity、workflow outcome join、evidence store |
| Process analysis | work-as-done baseline、change impact map、resistance taxonomy、process outcome model |
| AI Value Office | 将 adoption 证据纳入 portfolio value realization、funding gate、finance sign-off |
| Operations leadership | 用 adoption evidence 管理 coaching、SOP 调整、队列负荷和服务质量 |
| Risk / control review | 观察 over-reliance、control override、human review load、complaint 和 exception evidence |
Adoption analytics 明确排除三种常见误读：
| Misread | Why weak |
|---|---|
| 培训完成率等于采用 | 培训证明接触，不证明真实工作改变 |
| 登录数或 prompt 数等于价值 | 使用频繁可能代表输出不稳定或流程摩擦 |
| 用户满意度等于收益 | 满意度可能与质量、风险和净价值脱节 |
2. Executive Summary
AI 项目上线后，常见报告是：
licenses activated: 1,200
weekly active users: 870
prompts submitted: 42,000
average satisfaction: 4.2/5
这些数字只能说明有人接触工具。成熟 adoption analytics 必须证明：
eligible workflow population
-> real exposure
-> qualified task use
-> trust-calibrated human action
-> changed work artifact or decision
-> improved process flow / quality / control
-> realized net value
-> reinforced behavior over time
核心资产：
| Asset | 用途 |
|---|---|
| Work-as-done baseline | 捕捉真实流程和当前价值基线 |
| Adoption event taxonomy | 定义什么算真实 adoption |
| Telemetry schema | 让 adoption 可测、可追溯、可审查 |
| Metrics hierarchy | 防止 usage 指标冒充业务价值 |
| Behavior funnel | 定位 adoption drop-off |
| Cohort analysis | 识别角色、经理、case type 和风险等级差异 |
| Resistance signal map | 解释用户不用、误用或绕用的原因 |
| Change saturation review | 判断组织是否有容量吸收变化 |
| Outcome attribution model | 解释结果变化与 AI 的关系 |
| Value leakage model | 从 gross benefit 到 net realized value |
| Risk/control pack | 监控 over-reliance、override、review load 和客户影响 |
| Operating review loop | 把证据变成产品、流程、控制和管理动作 |
3. Source Anchors
| Source | Link | 用法 |
|---|---|---|
| NIST AI RMF | https://www.nist.gov/itl/ai-risk-management-framework | 用 Govern / Map / Measure / Manage 组织 adoption risk and evidence。 |
| NIST AI RMF Playbook | https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook | 用 action-oriented review pattern 设计 evidence and monitoring routine。 |
| ISO/IEC 42001 | https://www.iso.org/standard/81230.html | 用 AI management system 的 objective、operation、performance evaluation、improvement 语言。 |
| Prosci ADKAR | https://www.prosci.com/blog/adkar-model | 用 Awareness、Desire、Knowledge、Ability、Reinforcement 诊断行为改变。 |
| OpenTelemetry | https://opentelemetry.io/docs/ | 用 traces、metrics、logs 思路设计 adoption observability。 |
| DORA | https://dora.dev/ | 用 engineering performance and reliability thinking 连接 release、learning and restore。 |
| FFIEC Management IT Handbook | https://ithandbook.ffiec.gov/it-booklets/management.aspx | 对接金融机构治理、风险识别、监控和报告证据。 |
4. Conceptual Model
Adoption-to-value chain:
problem and baseline
-> AI intervention
-> exposure
-> qualified use
-> human trust action
-> behavior change
-> process quality change
-> business outcome
-> net value
-> reinforcement
Definitions:
| Term | Definition |
|---|---|
| Exposure | 目标用户在真实工作步骤中有机会看到或调用 AI |
| Qualified use | 用户在目标任务、目标 case type、目标流程阶段使用 AI |
| Trust-calibrated action | 用户能正确接受、编辑、拒绝、升级或覆盖 AI 输出 |
| Behavior change | 工作顺序、工件、决策、handoff 或控制执行发生可观察变化 |
| Workflow outcome | 周期、质量、返工、队列、客户体验、风险控制等流程结果 |
| Net realized value | 扣除运行、复核、返工、支持、风险和变更成本后的收益 |
| Durability | adoption 和 outcome 在 novelty effect 后仍然持续 |
Scale 前必须能回答：
- 当前 work-as-done baseline 是什么？
- 什么事件证明用户在目标流程中真实采用？
- 采用行为改变了哪个工件、判断、handoff 或控制？
- 哪些 leading 和 lagging indicators 证明流程改善？
- human review load、override、rework 和 cost-to-serve 是否吞掉价值？
- 组织如何通过经理节奏、SOP、培训和产品改进强化新行为？
5. Architecture Components
| Component | Owner | Execution details |
|---|---|---|
| Workflow map | Process analysis / Ops | AS-IS、work-as-done、exception path、control point、artifact map |
| Event taxonomy | Adoption governance / Process analysis | exposure、intent、output、response、influence、control、outcome events |
| Instrumentation SDK | Telemetry architecture / Engineering | emit events with workflow context、model version、user action |
| Identity and cohort layer | Analytics | role、team、manager、training wave、region、risk entitlement |
| Model and prompt registry | Platform | model_id、prompt_version、tool version、policy pack |
| Outcome connector | Data / Analytics | join events to handling time、cycle time、quality、rework、complaint、loss |
| Control evidence store | Risk / Telemetry architecture | overrides、escalations、QA defects、dual review、policy boundary hits |
| Adoption mart | Analytics | curated tables for funnel、cohort、attribution、value leakage |
| Dashboard and evidence pack | Adoption governance / Value Office | monthly operating pack and scale/stop memo |
| Operational learning loop | Adoption governance / Ops | backlog、SOP update、coaching、training、control tuning |
Reference data flow:
AI surface
-> adoption event stream
-> workflow context resolver
-> event validation and privacy filtering
-> adoption mart
-> outcome and control joins
-> behavior funnel / cohort / value analytics
-> operating review and action backlog
Architecture principles:
| Principle | Design implication |
|---|---|
| Context first | every event carries workflow_id、stage、role、case type、risk tier |
| Version everything | model_id、prompt_version、policy_pack_version、SOP_version、feature_flag |
| Capture human judgment | accept、edit、reject、ignore、regenerate、override、escalate are first-class |
| Do not over-collect payload | store event facts and references, not unnecessary customer content |
| Link to outcomes | adoption metrics without outcome join are not value evidence |
| Preserve negative evidence | rejection、complaint、defect、bypass drive learning |
| Reviewability | metric definitions, lineage and sample drilldown must be inspectable |
6. Work-as-Done Baseline
Instrumentation 前先建立 work-as-done baseline。不要从 AI 工具开始，要从今天工作如何发生开始。
| Field | Questions | Example: KYC onboarding |
|---|---|---|
| Workflow | Which end-to-end process? | New SMB account onboarding |
| Trigger | What starts the work? | Application submitted with documents |
| Actor | Who does the work? | KYC analyst, RM, onboarding ops, QA |
| Case mix | What types and complexity? | Sole proprietor, LLC, high-risk country exposure |
| Systems | Which systems are used? | CRM, document store, screening, core banking |
| Artifacts | What records are created? | deficiency notice, review note, approval record |
| Controls | Which control points matter? | sanctions, beneficial ownership, risk rating |
| Pain points | Where is work slow or poor? | repeated customer document chase |
| Informal work | What unofficial workarounds exist? | analyst checklist spreadsheet |
| Current metrics | What is baseline? | cycle time, first-pass completeness, rework |
| Failure modes | What causes defects? | outdated policy, missing doc, unclear ownership |
| Change capacity | What else is changing? | new onboarding policy and CRM migration |
Baseline evidence:
| Source | What it proves |
|---|---|
| SME observation | Actual sequence, friction and judgment |
| Process logs | Timing, queue, handoff and rework |
| Case notes | Artifact quality and evidence gaps |
| QA samples | Defect type and severity |
| Manager coaching logs | Behavioral patterns and recurring issues |
| Complaint records | Customer harm or confusion |
| Policy and SOP | Expected controls and business rules |
| Informal tools review | Workarounds not visible in system logs |
7. Adoption Event Taxonomy
| Class | Required events | Why it matters |
|---|---|---|
| Exposure | AI panel shown、suggestion presented、feature available in eligible case | proves opportunity to use |
| Intent | user opens assistant、asks task-specific question、requests summary | proves user pull |
| Output | summary、classification、recommendation、draft、next action generated | proves AI response existed |
| Human response | accept、edit、reject、ignore、regenerate | proves trust and fit |
| Decision influence | used in note、customer response、disposition、package、handoff | proves workflow impact |
| Control action | override、escalation、dual review、policy boundary hit | proves governed use |
| Learning signal | feedback reason、defect report、manager comment | proves improvement signal |
| Outcome | case closed、call completed、application approved、package passed QA | proves process link |
| Reinforcement | manager coaching、SOP update、training wave、team review | proves behavior support |
Event naming convention:
<workflow>.<stage>.<ai_surface>.<event_action>
Examples:
| Event name | Meaning |
|---|---|
aml.triage.case_summary.generated | AML summary produced during triage |
aml.investigation.narrative.accepted_with_edit | investigator used AI narrative with edits |
contact_center.customer_response.suggestion.rejected | agent rejected suggested response |
kyc.document_review.completeness_flag.overridden | analyst overrode AI document flag |
credit_ops.package_review.condition.extracted | credit condition extracted into review package |
branch.rm_prep.next_action.escalated | relationship manager escalated AI next action due to policy boundary |
Qualified adoption:
A qualified adoption event occurs when an eligible user, in an eligible workflow stage and case type,
uses an AI output to influence a governed work artifact, decision, handoff or customer interaction,
with human action and control outcome captured.
8. Data and Telemetry Schema
Canonical event contract:
| Field | Required | Description |
|---|---|---|
| event_id | Yes | Unique event id |
| event_time | Yes | Event timestamp |
| event_name | Yes | Taxonomy event name |
| event_class | Yes | Exposure, intent, output, response, influence, control, learning, outcome, reinforcement |
| user_id_hash | Yes | Pseudonymous worker id |
| role | Yes | Agent, investigator, analyst, manager, QA |
| team_id | Yes | Team, branch, region or operations unit |
| manager_id_hash | Recommended | Enables manager effect analysis |
| cohort_id | Yes | Pilot wave, training wave or feature flag cohort |
| workflow_id | Yes | AML, KYC, contact center, credit ops, branch |
| workflow_stage | Yes | Triage, document review, customer response, QA, decision |
| case_id_hash | Yes | Pseudonymous case id |
| case_type | Yes | Alert type, call reason, product, onboarding type |
| case_complexity | Recommended | Low, medium, high or scoring band |
| risk_tier | Yes | Business risk tier |
| ai_surface | Yes | Panel, inline suggestion, draft generator, policy search |
| model_id | Yes | Model registry id |
| prompt_version | Yes | Prompt or policy pack version |
| tool_ids | Recommended | Tools or connectors invoked |
| output_type | Yes | Summary, recommendation, draft, classification, next action |
| user_action | Yes | Accept, edit, reject, ignore, regenerate, override, escalate |
| edit_distance_band | Recommended | None, light, material, rewrite |
| reason_code | Recommended | Useful, inaccurate, incomplete, unsafe, policy unclear, slow, irrelevant |
| control_point_id | Recommended | Link to control or policy boundary |
| override_reason | Conditional | Required when override occurs |
| human_review_required | Yes | True or false |
| human_review_minutes | Recommended | Review load |
| downstream_artifact_id | Recommended | Note, letter, case record or decision package |
| outcome_event_id | Recommended | Link to process outcome |
| latency_ms | Recommended | Response latency |
| cost_estimate | Recommended | Unit cost estimate |
| privacy_class | Yes | Event-only, sensitive-reference, restricted |
| retention_class | Yes | Analytics, business-record-link, control-evidence |
OpenTelemetry mapping:
| Adoption concept | Observability mapping |
|---|---|
| Case journey | Trace |
| Workflow step | Span |
| AI call | Span with model and prompt attributes |
| User action | Event on span |
| Control override | Event with control attributes |
| Outcome | Linked span or downstream event |
| Aggregate adoption | Metric |
| Defect or complaint | Log/event with trace link |
9. Metrics Hierarchy
| Layer | Metrics | Owner |
|---|---|---|
| Telemetry quality | event completeness、missing context、join rate、schema drift | telemetry / analytics |
| Exposure | eligible users exposed、eligible case exposure、workflow placement coverage | adoption governance |
| Qualified adoption | qualified use rate、returning qualified use、case penetration | adoption governance |
| Trust and behavior | accept/edit/reject mix、edit distance、override、escalation、artifact reuse | process analysis |
| Flow and quality | cycle time、handling time、queue aging、first-pass quality、rework、QA defects | operations |
| Risk and control | over-reliance、under-reliance、policy boundary hits、complaint linkage | risk / QA |
| Value | net hours released、cost-to-serve、loss reduction、conversion、complaint reduction | Value Office / finance |
| Durability | 4/8/12-week retention、manager variance、post-release stability | operations |
Metric guardrails:
| Metric | Must not be interpreted alone |
|---|---|
| High prompt count | Could mean confusion or poor output |
| High accept rate | Could mean automation bias |
| Low override rate | Could mean users do not understand controls |
| Handling time reduction | Could hide repeat contact or QA rework |
| Time saved survey | Could ignore review load and support cost |
| High user satisfaction | Could coexist with customer harm |
10. Behavior Change Model
ADKAR-to-evidence:
| ADKAR stage | Execution evidence | Analytics signal |
|---|---|---|
| Awareness | Managers communicate why workflow changes | awareness pulse、team briefing completion |
| Desire | Users believe AI helps and does not punish them | opt-in demand、low resistance、champion pull |
| Knowledge | Users know when to use, avoid, escalate and override | correct reason codes、policy quiz、guidance views |
| Ability | Users perform the new workflow in real cases | qualified completion、light-edit acceptance、reduced rework |
| Reinforcement | Managers, SOP and metrics reinforce behavior | returning use、coaching logs、SOP_version adoption |
Resistance taxonomy:
| Signal | Diagnostic question | Response |
|---|---|---|
| Ignore | Is AI shown at the wrong time? | Move trigger closer to decision point |
| Reject | Is output inaccurate, irrelevant or untrusted? | Improve retrieval, prompt, source evidence |
| Regenerate | Is user trying to force a better answer? | Add structured task templates |
| Heavy edit | Is output format mismatched to artifact? | Redesign output contract |
| Override | Is user bypassing control or correcting AI? | Require reason and review patterns |
| Shadow AI | Is sanctioned tool missing a real need? | Bring unmet need into roadmap |
| Low returning use | Was initial experience poor or reinforcement absent? | Fix first-run quality and manager coaching |
| Team variance | Is adoption manager-led? | Add manager enablement and peer learning |
| Complaint rise | Is AI improving internal speed at customer expense? | Stop or restrict affected scenario |
Change saturation should be reviewed before scale: if the same team is already absorbing policy change, platform migration and performance target changes, AI rollout may fail even with good product quality.
11. Outcome Attribution
可归因收益需要合理实验设计，而不是把上线前后差值全部归因于 AI。
| Method | Use | Limits |
|---|---|---|
| Phased rollout | 分批上线，比较波次 | 需要控制 case mix 和管理差异 |
| Matched cohort | 匹配团队、case type、风险等级 | 匹配质量决定可信度 |
| Difference-in-differences | 比较试点组和对照组前后变化 | 需要平行趋势假设 |
| Interrupted time series | 观察上线前后趋势变化 | 容易受其他变更干扰 |
| Shadow comparison | AI 建议与人工结果对照 | 不证明真实 adoption |
Attribution report 必须说明：
| Field | Explanation |
|---|---|
| baseline period | 使用哪个时间段作为基线 |
| eligible population | 哪些用户和 case 可被纳入 |
| exposure | 哪些 case 真正有机会用到 AI |
| case mix controls | 风险等级、产品、地区、团队 |
| concurrent changes | 同期政策、系统、人员、流程变化 |
| model and prompt version | 哪个版本产生效果 |
| confidence | high / medium / low and why |
12. Value Leakage Model
Gross benefit 必须扣除 leakage 才能进入 scale decision。
net realized value =
gross process benefit
- AI run cost
- human review load
- rework cost
- support cost
- training and change cost
- control and audit cost
- incident and customer harm adjustment
| Leakage | Signal |
|---|---|
| Review load | review_minutes_per_case、review_queue_depth |
| Rework | correction rate、reopen、after-call work |
| Support | tickets、office hour volume、FAQ updates |
| Latency | abandonment、fallback、manual path |
| Control overhead | sampling rate、dual review、approval backlog |
| Customer harm | complaint、repeat contact、remediation |
| Adoption decay | 4/8/12-week retention decline |
13. Risk and Control Analytics
| Risk | Signal | Control response |
|---|---|---|
| Over-reliance | high accept、low edit、rising defects | QA sampling、rationale check、high-risk friction |
| Under-reliance | high reject despite good quality | trust evidence、workflow placement、coaching |
| Control bypass | override without reason | mandatory reason、manager review |
| Hidden review burden | review queue grows | end-to-end capacity dashboard |
| Policy boundary drift | answers outside allowed domain | policy engine、refusal、escalation |
| Customer harm | complaint、repeat contact、correction | scenario restriction、content review |
| Uneven access | low exposure in certain branches | entitlement and training remediation |
| Version trust decay | adoption drops after release | rollback and communication |
Override classification:
| Classification | Meaning | Review action |
|---|---|---|
| Corrective override | User corrected AI error | Feed defect into model/product backlog |
| Risk override | User bypassed control | Manager/risk review |
| Policy ambiguity | User could not determine boundary | Clarify SOP and policy evidence |
| Workflow mismatch | AI suggestion did not fit actual process | Redesign output or trigger |
| Emergency override | Used due to service or customer urgency | Review exception governance |
Human review load:
| Metric | Why |
|---|---|
| review_minutes_per_case | Measures hidden labor |
| reviewer_queue_depth | Detects backlog transfer |
| review_defect_yield | Shows whether review finds real issues |
| review_sampling_rate | Controls auditability and cost |
| review_escalation_rate | Shows uncertainty and boundary issues |
| review_reversal_rate | Shows AI or user judgment quality |
14. Operating Model
| Cadence | Forum | Decision |
|---|---|---|
| Daily | Ops pulse | blockers、latency、incidents、support questions |
| Weekly | Adoption working session | funnel drop-off、resistance、product fixes |
| Biweekly | Risk/control review | overrides、defects、complaints、review load |
| Monthly | Value realization review | benefit、leakage、finance challenge、scale/stop |
| Quarterly | Architecture and portfolio review | platform reuse、telemetry maturity、lifecycle |
RACI:
| Activity | Adoption governance | Process analysis | Telemetry architecture | Ops | Risk | Analytics | Finance |
|---|---|---|---|---|---|---|---|
| Define adoption taxonomy | A/R | R | C | C | C | C | I |
| Build work-as-done baseline | C | A/R | I | R | C | C | I |
| Implement telemetry | C | C | A/R | I | C | R | I |
| Validate data quality | C | C | R | I | C | A/R | I |
| Run behavior funnel review | A/R | R | I | R | C | R | I |
| Manage resistance actions | A/R | R | I | R | I | C | I |
| Review risk/control | C | C | C | R | A/R | C | I |
| Calculate value | R | C | I | C | C | C | A/R |
| Decide scale/stop | A/R | C | C | C | C | C | C |
Operational learning loop:
Observe -> Diagnose -> Decide -> Change -> Reinforce -> Measure -> Record
15. Evidence Pack
| Section | Content |
|---|---|
| Executive summary | adoption、behavior、risk、value、decision |
| Problem and baseline | work-as-done、pain points、baseline metrics |
| Intervention | AI capability、workflow integration、model/prompt version |
| Event taxonomy | qualified adoption definition and events |
| Telemetry quality | completeness、join rate、known limitations |
| Behavior funnel | step conversion and drop-off |
| Cohort analysis | role、manager、team、case type、risk tier |
| Outcome attribution | method、baseline、confounders、confidence |
| Value realization | gross benefit、leakage、net value |
| Risk/control | over-reliance、override、review load、defects、complaints |
| User trust | reason codes、qualitative themes、sentiment |
| Operating actions | backlog、SOP、training、manager coaching |
| Decision | scale、continue pilot、redesign、restrict or stop |
Evidence quality:
| Level | Meaning |
|---|---|
| Weak | usage-only、no baseline、no outcome join |
| Developing | baseline and adoption funnel exist, limited cohort analysis |
| Strong | cohort、outcome、risk、review load and value leakage included |
| Executive-ready | finance-challenged value、risk-reviewed controls、clear scale/stop action |
16. Execution Roadmap
| Day range | Work |
|---|---|
| 1-3 | Select one high-value workflow and define business owner |
| 4-6 | Build work-as-done baseline from observation, logs, QA and SME review |
| 7-9 | Define adoption event taxonomy and qualified adoption event |
| 10-12 | Define metrics hierarchy and risk/control signals |
| 13-15 | Review with ops, risk, architecture and finance |
| 16-20 | Implement event contract with workflow context and model version |
| 21-24 | Connect identity, cohort, feature flag and training wave |
| 25-28 | Join process outcome and QA/control data |
| 29-32 | Build behavior funnel and cohort dashboard |
| 33-35 | Validate telemetry completeness and metric definitions |
| 36-42 | Run pilot with manager reinforcement and support path |
| 43-48 | Analyze resistance signals, edit/reject/override reasons |
| 49-53 | Calculate review load, rework, cost and early value leakage |
| 54-57 | Run risk/control review |
| 58-60 | Produce pilot evidence pack and decision recommendation |
| 61-90 | Expand only if risk and value evidence meet gate; add attribution, durability and finance challenge |
17. Financial Retail Examples
| Use case | Qualified adoption | Key outcomes | Risk and leakage |
|---|---|---|---|
| AML investigator copilot | Investigation summary used in narrative or evidence review for eligible alert | alert aging、QA correction、narrative quality、re-open rate | senior reviewer time、false comfort、incomplete narrative |
| Contact-center agent assist | Suggested response used in target call reason with policy citation | handling time、first contact resolution、QA score、complaint | repeat contact、QA burden、unsafe answer |
| KYC onboarding assistant | AI document completeness flag used before customer chase | first-pass completeness、cycle time、customer chase count | false deficiency notice、analyst re-review |
| Credit ops reviewer | AI extraction used in package review, not final judgment | first-pass package quality、condition miss rate | downstream correction、risk escalation |
| Branch relationship copilot | permitted insight used to prepare client follow-up | follow-up completion、retention、qualified referral | compliance review、unsuitable suggestion correction |
18. Evidence Artifact Structures
18.1 Adoption Event Card
| Field | Fill with concrete value |
|---|---|
| Event name | workflow.stage.surface.action |
| Event class | exposure / intent / output / response / influence / control / learning / outcome |
| Workflow | Named workflow |
| Stage | Exact process step |
| Eligible users | Roles and cohorts |
| Eligible cases | Case types and risk tiers |
| Human action | accept、edit、reject、ignore、regenerate、override、escalate |
| Business artifact | note、decision、response、package、handoff |
| Control point | policy or control id |
| Outcome link | downstream result |
| Misinterpretation risk | how this event could be over-read |
18.2 Monthly Operating Review Agenda
| Agenda item | Decision |
|---|---|
| Telemetry quality | Can we trust the data? |
| Funnel drop-off | What is the biggest adoption bottleneck? |
| Cohort variance | Which manager, team or case type needs action? |
| Resistance signals | Product, workflow, trust, control or incentive issue? |
| Risk/control | Any over-reliance, override or complaint trend? |
| Value leakage | Is review load or rework consuming benefit? |
| Product backlog | What changes ship next? |
| Ops and manager actions | What coaching, SOP or process changes happen? |
| Scale/stop | Continue, scale, redesign, restrict or stop? |
18.3 Scale / Stop Memo Structure
| Section | Evidence expectation |
|---|---|
| Decision requested | 明确是 scale、continue pilot、redesign、restrict 还是 stop |
| Workflow and target population | 命名流程、目标用户、case type、risk tier 和 rollout cohort |
| Baseline | work-as-done summary、baseline metrics、case mix 和当前控制点 |
| Adoption evidence | qualified adoption、behavior funnel、cohort findings、durability |
| Outcome evidence | flow、quality、customer result、risk/control outcome |
| Value evidence | gross benefit、AI run cost、human review load、rework、support、risk adjustment、net realized value |
| Risk/control evidence | override、escalation、defects、complaints、over-reliance、under-reliance |
| Recommendation | 决策、理由、约束、下一次复盘时间 |
19. Anti-Patterns
| Anti-pattern | Consequence | Replacement |
|---|---|---|
| Reporting MAU as adoption | Hides whether work changed | qualified adoption event |
| Counting prompts as value | Rewards friction | outcome-linked behavior metrics |
| Treating training as adoption | Ignores real workflow | work-as-done and behavior funnel |
| Celebrating high accept rate | Encourages automation bias | accept/edit/reject with quality and defects |
| Ignoring rejection reasons | Misses product and trust issues | structured reason codes |
| Using averages only | Hides manager and case mix effects | cohort analysis |
| Not measuring review load | Overstates benefit | human review load and value leakage |
| No override taxonomy | Confuses healthy challenge with bypass | override classification |
| No change saturation view | Overloads teams | rollout capacity review |
| Dashboard without action | Creates reporting theater | operating learning loop and decision log |
20. Operating Principle
Do not ask whether users used AI.
Ask whether AI changed governed work in a way that improved durable net outcomes.