AI 扩展计划 / Playbooks
AI Workforce Capability Academy / Role-Skill Transformation Playbook
当企业出现以下信号时, 使用本 playbook:
783 行AI_WORKFORCE_CAPABILITY_ACADEMY_ROLE_SKILL_TRANSFORMATION_PLAYBOOK.md
AI Workforce Capability Academy / Role-Skill Transformation Playbook
定位: 面向 Senior AI PM、AI Solution Architect、Enterprise Architect、CBAP-level BA、AI Transformation Lead、RiskOps / EvalOps Lead、HR/L&D Partner 和金融零售运营负责人。 目标: 把 AI 能力建设从课程交付升级为 workforce capability product platform, 用角色、技能、证据、情景评估、作品集、adoption telemetry 和治理节奏持续提升组织能力。 核心观点: AI academy 的产品结果不是培训完成率, 而是可证明的岗位胜任、可复用的能力资产、可度量的 adoption 改变和可管理的 workforce risk。
1. 使用场景
当企业出现以下信号时, 使用本 playbook:
| Signal | Meaning |
|---|---|
| AI 培训很多, 但生产 use case 质量参差不齐 | 学习活动没有转成能力证据和 release gate。 |
| PM、BA、Architect、Risk、Ops 对 AI 责任边界争议大 | 缺少 role taxonomy、RACI 和 proficiency target。 |
| AI POC 多, 能规模化的人少 | skills debt 和关键岗位瓶颈正在阻塞 portfolio。 |
| 业务部门只要求"教大家用 AI" | 需要转成 role-based learning path and evidence-of-competence。 |
| 风控担心员工误用 AI, 技术担心治理拖慢速度 | 需要分层 literacy、授权门槛和场景化评估。 |
| 高级人才面试时难证明 AI PM / AI Architect / AI BA 能力 | 需要 portfolio artifacts 和 scenario defense。 |
一句话操作原则:
Build the academy like a product platform: role demand in, verified capability and adoption outcomes out.
2. Source Anchors
这些来源作为官方学习锚点和治理语言来源, 不替代机构内部政策、法律、人力资源、合规或审计判断。
| Anchor | Official link | Playbook 用法 |
|---|---|---|
| SFIA AI skills resources | https://sfia-online.org/en/tools-and-resources/ai-skills-framework | 作为 AI 技能分类、岗位能力和组织技能管理的参考。 |
| SFIA 9 Skills A-Z | https://sfia-online.org/en/sfia-9/skills/all-skills-a-z | 用技能目录和等级化责任语言表达 role proficiency。 |
| NIST NICE Framework | https://www.nist.gov/itl/applied-cybersecurity/nice/nice-framework-resource-center | 借鉴 work role、task、knowledge、skill、ability 的结构化表达。 |
| NIST AI RMF | https://www.nist.gov/itl/ai-risk-management-framework | 用 Govern / Map / Measure / Manage 设计 AI workforce risk 和持续改进。 |
| ISO/IEC 42001 | https://www.iso.org/standard/81230.html | 用 AI management system 的责任、运行控制、绩效评价和改进语言组织 academy governance。 |
| ISO 30414 | https://www.iso.org/standard/69338.html | 作为 human capital reporting 参考, 帮助组织技能、领导力、能力投资和 workforce risk reporting。 |
3. Operating Principles
| Principle | Operational rule |
|---|---|
| Role before course | 先定义角色要承担的 AI 工作, 再定义学习内容。 |
| Evidence before badge | 高风险岗位不能只看课程完成, 必须看情景评估、作品集和生产证据。 |
| Scenario before theory | 高级岗位通过金融零售场景证明能力, 例如 KYC、AML、客服、信贷、数据产品。 |
| Adoption before vanity metrics | 学院指标必须连接真实 workflow adoption、quality、benefit、risk。 |
| Community before one-off training | 用 communities of practice 持续复盘案例、沉淀模式和校准标准。 |
| Skills debt is risk | 关键岗位能力缺口进入 AI portfolio 和 operational risk review。 |
| Academy as platform | 角色、技能、路径、评估、证据、分析和治理都要产品化、版本化、可迭代。 |
4. 12-Step Build Method
Step 1: Define business capability demand
输入 AI strategy、portfolio roadmap 和业务痛点, 先回答:
| Question | Output |
|---|---|
| 未来 6-12 个月哪些 AI capabilities 最关键? | priority capability list。 |
| 哪些业务线最需要 role transformation? | target domains: KYC, AML, contact center, lending, data product。 |
| 哪些角色阻塞 scale? | bottleneck role list。 |
| 哪些风险需要 workforce control? | high-risk responsibility list。 |
产物:
AI Workforce Capability Demand Brief
最小字段:
| Field | Example |
|---|---|
| capability | AI-assisted KYC onboarding review。 |
| business outcome | reduce manual review cycle time while preserving customer protection。 |
| required roles | AI PM, AI BA, Solution Architect, EvalOps, RiskOps, Ops Lead。 |
| target proficiency | PM Level 4, BA Level 4, Architect Level 4, Ops Lead Level 3。 |
| risk if absent | weak release gate, inconsistent review, unsupported automation。 |
Step 2: Create role taxonomy
不要从 HR 现有职位名直接复制。先按 AI 工作责任定义角色。
| Role family | Role profile | Key accountabilities |
|---|---|---|
| Product | Senior AI PM | value thesis, roadmap, adoption, eval gate, scale/stop。 |
| Business analysis | AI BA / CBAP-level Transformation BA | process evidence, requirements, role redesign, scenario pack。 |
| Architecture | AI Solution Architect | AI control plane, integration, security, observability, rollback。 |
| Eval and quality | EvalOps Lead | golden set, rubric, regression, production sampling。 |
| Risk and controls | RiskOps Lead | risk tiering, control evidence, incident taxonomy。 |
| Data and knowledge | Data Product Manager / Knowledge Owner | source authority, lineage, freshness, permission, retention。 |
| Operations | AML/KYC Ops Lead / Contact Center Lead | adoption, QA, manager cadence, support model。 |
| Enablement | AI Academy Product Owner / COP Lead | path design, evidence platform, community learning, analytics。 |
产物:
Role Profile Registry
Template:
| Field | Definition |
|---|---|
| role_id | Stable role identifier。 |
| role_purpose | Why this role exists in the AI operating model。 |
| critical tasks | 5-10 tasks tied to real AI work。 |
| decision authority | What the role can approve, recommend or execute。 |
| risk-sensitive activities | Tasks requiring higher proficiency or independent review。 |
| target proficiency | Required level by skill domain。 |
| evidence required | Artifact, scenario and production evidence。 |
| recertification trigger | Policy, model, tool, incident or role change。 |
Step 3: Build skill ontology
Use a skill object, not a course list.
| Skill domain | Skills to define |
|---|---|
| Responsible AI literacy | data boundaries, hallucination recognition, human oversight, escalation。 |
| Product and value | use case framing, value hypothesis, adoption metrics, AI economics。 |
| Requirements and process | AI-assisted discovery, process mining, exception path, role redesign。 |
| Architecture and integration | RAG, model gateway, tool gateway, IAM, logging, fallback。 |
| EvalOps | golden sets, rubrics, regression, production sampling, critical failures。 |
| Risk and governance | AI risk tiering, control mapping, incident response, audit evidence。 |
| Data and knowledge | source authority, metadata, lineage, permissions, freshness, retention。 |
| Operations adoption | coaching, QA calibration, support tiers, feedback loop, COP facilitation。 |
Skill template:
| Field | Example |
|---|---|
| skill_id | EVAL-SCENARIO-DESIGN。 |
| definition | Design scenario-based evaluations for AI workflows。 |
| related roles | AI PM, AI BA, EvalOps, RiskOps。 |
| tasks | define critical failures, build golden journeys, set thresholds。 |
| knowledge | AI RMF concepts, UAT, business acceptance, model/prompt/RAG versioning。 |
| evidence | eval pack, release gate memo, reviewer calibration record。 |
| risk if absent | weak release decisions and undetected customer harm。 |
Step 4: Define proficiency levels
Use one enterprise ladder, then specialize by role.
| Level | Name | Authorization meaning |
|---|---|---|
| 1 | Awareness | Can use AI under broad policy and identify obvious risks。 |
| 2 | Guided practitioner | Can perform defined tasks under SOP and supervision。 |
| 3 | Independent practitioner | Can perform role tasks, create artifacts and handle standard exceptions。 |
| 4 | Lead / reviewer | Can review others, set local standards and sign evidence within authority。 |
| 5 | System owner / strategist | Can design organizational capability, governance and investment approach。 |
Evidence rule:
No high-risk responsibility is granted without current Level 3+ evidence.
No reviewer authority is granted without Level 4 evidence and calibration.
Step 5: Map roles to capabilities
Build a role-to-capability table for each priority domain.
Example: AI-assisted KYC onboarding
| Capability | AI PM | AI BA | Architect | EvalOps | RiskOps | Data Product | Ops Lead |
|---|---|---|---|---|---|---|---|
| Use case thesis | A/R | R | C | C | C | C | C |
| KYC process redesign | C | A/R | C | C | C | I | R |
| Document AI architecture | C | C | A/R | C | C | C | I |
| Policy and source authority | C | R | C | C | A/R | R | C |
| Eval and UAT gate | A | R | C | A/R | C | C | R |
| Human review operations | C | R | I | C | C | I | A/R |
| Adoption and benefit tracking | A/R | R | I | C | C | C | A/R |
Use this table to decide who needs which learning path and what assessment evidence is required.
Step 6: Assess current evidence and skill gaps
Do not ask managers only "who is strong at AI". Inventory evidence.
| Evidence source | What to collect |
|---|---|
| Existing project artifacts | PRDs, BRDs, architecture diagrams, ADRs, eval reports, risk memos。 |
| Production telemetry | usage, override, defect, incident, adoption, benefit metrics。 |
| Manager observations | coaching notes, quality review, support issues。 |
| Peer review | artifact critique, COP contribution, reviewer calibration。 |
| Scenario assessment | case performance against rubric。 |
Skill gap output:
| Gap | Impact | Treatment |
|---|---|---|
| Few PMs can define eval release gates | AI roadmap slows or ships with weak evidence | Senior AI PM path with eval artifact defense。 |
| Architects know LLM APIs but not entitlement-aware retrieval | privacy and source leakage risk | Architecture path with RAG governance lab。 |
| Ops managers do not understand override analytics | adoption quality drifts | Manager path with QA calibration and dashboard review。 |
Step 7: Design role-based learning paths
Path template:
| Section | Content |
|---|---|
| Entry criteria | Current role, prerequisite literacy, business domain。 |
| Target proficiency | Skill levels by domain。 |
| Modules | Short conceptual modules tied to role tasks。 |
| Labs | Hands-on artifacts and scenario exercises。 |
| Assessment | Scenario-based exam and portfolio defense。 |
| Supervised practice | Real project or pilot assignment with reviewer。 |
| Exit evidence | Artifact set and adoption or operating evidence。 |
| Validity | Expiry and recertification trigger。 |
Recommended paths:
| Path | Modules | Required portfolio artifacts |
|---|---|---|
| AI BA Role Transformation | process mining, requirements-to-eval, role redesign, human oversight, UAT evidence | TO-BE workflow, scenario pack, acceptance criteria, role impact memo。 |
| Senior AI PM | value thesis, AI economics, eval gates, adoption telemetry, platform reuse, scale/stop | AI PRD, benefit register, eval strategy, adoption dashboard, scale memo。 |
| AI Solution Architect | model gateway, RAG governance, tool authorization, observability, rollback, security | reference architecture, ADR, control map, rollback runbook。 |
| EvalOps / RiskOps | risk tiering, golden sets, rubric design, regression, production sampling, incident learning | eval pack, control evidence, incident taxonomy, monitoring review。 |
| Data Product Manager | AI data contracts, source authority, consent, lineage, freshness, data product metrics | data contract, source inventory, lineage map, freshness dashboard。 |
| Operations Transformation Lead | frontline adoption, QA calibration, manager cadence, support model, COP | adoption plan, QA calibration pack, support runbook, coaching dashboard。 |
Step 8: Design scenario-based assessments
Scenario assessment should include messy artifacts and conflicting constraints.
Assessment template:
| Field | Definition |
|---|---|
| scenario_id | Stable identifier。 |
| business context | Domain, product, channel, customer/employee impact。 |
| role objective | What candidate must decide or produce。 |
| input artifacts | PRD excerpt, SOP, policy, logs, sample output, incident, metrics。 |
| expected output | Artifact or decision memo。 |
| scoring rubric | Dimensions and score anchors。 |
| critical failures | Errors that fail regardless of overall score。 |
| reviewer roles | Business, risk, architecture, EvalOps or Ops。 |
| feedback | Strengths, gaps and required practice。 |
Example scenario:
scenario_id: KYC-AI-DOC-REVIEW-001
context: retail account opening team wants an AI assistant to review documents
role objective for AI PM: define release gate and adoption metrics
role objective for AI BA: design TO-BE workflow and exception handling
role objective for Architect: design source authority, logging, fallback and access control
critical failures:
- AI can reject customer without human review
- no policy version or source authority
- no appeal or escalation path
- no monitoring for unsupported rejection recommendations
Step 9: Build portfolio evidence registry
Artifacts should be reusable for interviews, performance review and internal governance.
| Artifact | Owner role | Evidence of competence |
|---|---|---|
| AI PRD | AI PM | problem framing, metrics, eval and adoption design。 |
| Process and role redesign pack | AI BA | workflow evidence, human oversight, stakeholder alignment。 |
| AI architecture decision record | Architect | trade-offs, controls, rollback, observability。 |
| Eval release gate | EvalOps / PM | quality threshold, critical failures, regression coverage。 |
| Risk and control memo | RiskOps | risk tier, controls, monitoring and residual risk。 |
| Data product contract | Data Product Manager | source, lineage, permissions, freshness, ownership。 |
| Adoption dashboard | Ops Lead / PM | usage, quality, benefit, friction, coaching actions。 |
Registry fields:
| Field | Purpose |
|---|---|
| artifact_id | Stable evidence identity。 |
| owner | Person or team。 |
| role and skill mapping | Which capability this proves。 |
| reviewer | Who validated it。 |
| result | Pass, partial, needs supervised practice, not accepted。 |
| validity | Expiry or trigger。 |
| production link | Use case, release, adoption or incident reference。 |
Step 10: Launch communities of practice
Community of practice is not a casual chat channel. It should produce reusable standards.
| COP | Cadence | Outputs |
|---|---|---|
| AI PM product clinic | Biweekly | PRD critiques, value metrics, adoption patterns, scale/stop examples。 |
| AI BA process and evidence guild | Biweekly | scenario packs, role redesign patterns, acceptance criteria examples。 |
| AI architecture review circle | Biweekly | ADR examples, RAG patterns, tool gateway patterns, rollback lessons。 |
| EvalOps calibration board | Monthly | rubric updates, reviewer variance review, critical failure library。 |
| Ops adoption forum | Weekly during rollout, monthly later | frontline feedback, support patterns, manager coaching actions。 |
COP artifact rule:
Every session should produce or improve one reusable artifact:
pattern, rubric, anti-pattern, scenario, case study, checklist or evidence example.
Step 11: Connect telemetry
Create an academy analytics view that connects learning evidence to outcomes.
| Signal | Source | Why it matters |
|---|---|---|
| Role readiness | assessment and evidence registry | shows who can own AI work。 |
| Adoption | AI application logs and workflow systems | shows whether trained roles changed behavior。 |
| Quality | eval, QA, override, defect data | shows whether capability improved work outcomes。 |
| Risk | incidents, control exceptions, audit issues | shows whether skills are preventing harm。 |
| Benefit | cycle time, cost per case, STP, complaint, rework | shows business value。 |
| Skills debt | gap inventory, aged gaps, bottleneck roles | shows workforce risk。 |
Minimum dashboard:
| View | Questions answered |
|---|---|
| Executive readiness | Which critical AI capabilities lack verified roles? |
| Domain heatmap | Which business domains have skills debt? |
| Path health | Which paths have weak pass rates or low artifact quality? |
| Adoption impact | Which trained cohorts show production behavior change? |
| Risk watchlist | Which gaps correlate with incidents, defects or delayed releases? |
Step 12: Run governance loop
| Forum | Cadence | Decisions |
|---|---|---|
| Academy product review | Monthly | path backlog, learner friction, platform improvements。 |
| Assessment board | Monthly | scenario quality, rubric changes, reviewer calibration。 |
| Workforce capability review | Quarterly | skills debt treatment, investment priority, hiring vs training。 |
| AI portfolio review | Quarterly | link capability readiness to scale/stop decisions。 |
| Risk and audit evidence review | Quarterly | high-risk role evidence, control gaps, incident learning。 |
5. Operating Model and RACI
| Activity | Academy PO | HR/L&D | Business Owner | AI PM Lead | BA Lead | Architect Lead | Risk/EvalOps | Ops Lead |
|---|---|---|---|---|---|---|---|---|
| Define role taxonomy | A/R | C | C | R | R | R | C | C |
| Define skill ontology | A/R | C | C | R | R | R | R | C |
| Set target proficiency | C | C | A/R | R | R | R | R | R |
| Design learning paths | A/R | R | C | R | R | R | C | C |
| Build scenario packs | A | C | C | R | R | R | R | R |
| Calibrate reviewers | A | C | C | C | C | C | A/R | C |
| Approve high-risk evidence | C | I | A | C | C | C | A/R | C |
| Run COPs | A/R | C | C | R | R | R | R | R |
| Track adoption outcomes | A | C | A/R | R | R | C | R | R |
| Report skills debt | A/R | R | A | C | C | C | C | C |
6. Financial Retail Role Packages
6.1 AI BA Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for process and evidence architecture, Level 3 for EvalOps literacy。 |
| Core skills | process mining, requirements-to-eval, role redesign, human oversight, UAT evidence。 |
| Scenario | Contact center complaint triage and KYC exception handling。 |
| Artifacts | AS-IS/TO-BE, exception path, acceptance criteria, scenario pack, role impact memo。 |
| Adoption signal | fewer ambiguous requirements, stronger release evidence, lower rework。 |
6.2 Senior AI PM Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for AI product strategy and adoption, Level 3 for risk and architecture literacy。 |
| Core skills | use case thesis, value metrics, eval gate, AI economics, scale/stop decision。 |
| Scenario | KYC document review copilot product launch。 |
| Artifacts | AI PRD, benefit register, eval strategy, adoption dashboard, launch decision memo。 |
| Adoption signal | target users repeatedly use workflow, quality remains within thresholds, benefits are finance-reviewable。 |
6.3 AI Solution Architect Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for architecture controls, Level 3 for product and adoption context。 |
| Core skills | model gateway, RAG governance, entitlement, tool authorization, observability, rollback。 |
| Scenario | Loan policy assistant with RAG and workflow integration。 |
| Artifacts | architecture view set, ADR, control map, data flow, fallback and rollback runbook。 |
| Adoption signal | safe reuse by multiple use cases, stable monitoring, reduced architecture review defects。 |
6.4 RiskOps / EvalOps Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for eval and controls, Level 3 for product context。 |
| Core skills | risk tiering, golden set, rubric, critical failure, regression, production sampling。 |
| Scenario | AML alert investigation assistant。 |
| Artifacts | eval pack, control evidence, incident taxonomy, monitoring review。 |
| Adoption signal | release decisions have defensible quality evidence and incidents feed back into eval。 |
6.5 Data Product Manager Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for AI data product governance。 |
| Core skills | source authority, data contract, consent, lineage, freshness, data quality, access。 |
| Scenario | Customer 360 context product for contact center and personalization。 |
| Artifacts | data contract, lineage map, consent/preference map, freshness dashboard。 |
| Adoption signal | fewer duplicate extracts, better retrieval quality, clearer ownership。 |
6.6 AML/KYC Operations Lead Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 3 for AI responsible operations, Level 4 for manager coaching and QA。 |
| Core skills | human oversight, override taxonomy, QA calibration, support model, escalation。 |
| Scenario | AI-assisted KYC and AML case review queue。 |
| Artifacts | reviewer SOP, coaching plan, QA calibration pack, incident route。 |
| Adoption signal | consistent overrides, stable quality, reduced unsupported escalations。 |
6.7 Contact Center Transformation Lead Package
| Component | Requirement |
|---|---|
| Target proficiency | Level 4 for adoption and operations redesign。 |
| Core skills | AI literacy, knowledge assistant adoption, frontline coaching, support tiers, feedback loop。 |
| Scenario | AI knowledge assistant for complaint and product servicing。 |
| Artifacts | adoption plan, training scenarios, support runbook, manager dashboard。 |
| Adoption signal | improved AHT or FCR without complaint quality deterioration。 |
7. Templates
7.1 Role Profile Card
# Role Profile: Senior AI Product Manager
Purpose:
Own value, adoption, release evidence and lifecycle decisions for AI-enabled business capabilities.
Critical tasks:
- Frame AI use cases against business outcomes and no-AI alternatives.
- Define adoption, quality, risk, cost and benefit metrics.
- Partner with BA, Architect, EvalOps and Risk to define release gates.
- Decide scale, pause or stop using evidence.
Required proficiency:
- AI product and value: Level 4
- EvalOps literacy: Level 3
- AI risk and governance: Level 3
- Adoption telemetry: Level 4
Evidence required:
- AI PRD
- Eval strategy
- Adoption dashboard
- Scale/stop decision memo
Recertification trigger:
- Major platform change
- New high-risk AI responsibility
- Relevant AI incident or policy change
7.2 Skill Evidence Contract
# Skill Evidence Contract: Scenario-Based AI Evaluation Design
Target roles:
AI PM, AI BA, EvalOps Lead, RiskOps Lead
Target proficiency:
Level 3 for practitioners, Level 4 for reviewers
Evidence artifacts:
- Scenario pack
- Golden journey list
- Critical failure list
- Scoring rubric
- Release decision rule
- Monitoring and feedback loop
Critical failures:
- No customer harm scenarios
- No human escalation criteria
- No policy version or source authority
- No production monitoring trigger
Reviewers:
EvalOps, RiskOps, Business Process Owner
Validity:
12 months or until major policy, model, prompt, data or workflow change
7.3 Learning Path Card
# Learning Path: AI BA Transformation
Entry:
Experienced BA or CBAP-level practitioner working on AI-enabled process change.
Target:
Level 4 in process evidence, role redesign and requirements-to-eval.
Modules:
- Responsible AI in financial retail
- AI-assisted requirements mining
- AS-IS / TO-BE workflow redesign
- Human oversight and exception handling
- Acceptance criteria and eval linkage
- UAT and business acceptance evidence
Labs:
- KYC onboarding exception workflow
- Contact center complaint triage
- AML alert investigation support
Exit evidence:
- Process redesign pack
- Scenario assessment result
- Acceptance criteria and eval linkage
- Reviewer feedback record
7.4 Assessment Rubric
| Dimension | 1 Weak | 3 Acceptable | 5 Strong |
|---|---|---|---|
| Business framing | AI solution named without clear problem | Problem and baseline defined | Outcome, baseline, constraints and no-AI alternative clear。 |
| Role design | Human responsibility vague | Main handoffs defined | Human, AI, manager, risk, support and customer recourse explicit。 |
| Evidence | Assertions without artifacts | Some artifact links | Evidence chain from source to decision to monitoring。 |
| Risk | Generic risk list | Main risk controls defined | Customer harm, privacy, fairness, operational, model and audit risks tied to controls。 |
| Adoption | Training mentioned | Rollout plan exists | Manager cadence, support model, resistance signals and telemetry defined。 |
| Architecture | Tool named | Integration described | Identity, source authority, logging, fallback, versioning and rollback addressed。 |
7.5 Skills Debt Register
| Field | Definition |
|---|---|
| debt_id | Stable id。 |
| capability | AI capability impacted。 |
| role gap | Which role lacks verified proficiency。 |
| portfolio impact | Which roadmap item, release gate or operation is blocked。 |
| risk level | low, medium, high, critical。 |
| treatment | learning path, hiring, expert review, vendor support, scope reduction。 |
| owner | business capability owner and academy owner。 |
| due date | target closure date。 |
| evidence of closure | assessment, artifact, production signal or staffing change。 |
8. Metrics Dashboard
8.1 Executive View
| Metric | Definition | Decision supported |
|---|---|---|
| Critical role readiness | % priority roles with current Level 3+ or Level 4 evidence | can the AI portfolio scale safely? |
| Skills debt exposure | count and age of high-risk gaps by capability | where to invest or slow down roadmap? |
| Reviewer capacity | number of calibrated reviewers by domain | will release gates become bottlenecks? |
| Adoption conversion | % trained cohort using AI in target workflows with quality threshold met | is learning changing behavior? |
| Incident learning closure | % incidents reflected in updated scenarios or controls | is the academy learning from production? |
8.2 Product and Ops View
| Metric | Definition |
|---|---|
| path-to-artifact conversion | learners who produce accepted portfolio evidence。 |
| scenario failure pattern | common critical failures by role and module。 |
| manager coaching completion | teams with active coaching cadence and adoption review。 |
| support issue taxonomy | learning or workflow gaps found through support tickets。 |
| COP artifact reuse | patterns, rubrics or checklists reused in projects。 |
8.3 Risk View
| Metric | Definition |
|---|---|
| high-risk authorization coverage | users with current evidence before high-risk AI access。 |
| control evidence completeness | release evidence signed by competent roles。 |
| skills-related incidents | incidents where capability gap contributed。 |
| recertification breach | roles with expired evidence still assigned to sensitive tasks。 |
| key-person dependency | critical capability covered by one or two people only。 |
9. Governance Gates
| Gate | Applies when | Required evidence |
|---|---|---|
| AI tool access gate | Employee uses AI in risk-sensitive workflow | Tier 2 literacy and role-specific responsible-use assessment。 |
| Builder gate | Employee designs AI use case, prompt, workflow or data product | Level 3 evidence in relevant path and assigned reviewer。 |
| Reviewer gate | Employee reviews release, eval or risk evidence | Level 4 evidence, calibration record and governance appointment。 |
| Scale gate | AI use case expands to new teams or customer segments | role readiness, adoption telemetry, support model and incident route。 |
| Recertification gate | major model, prompt, policy, tool, workflow or incident change | updated scenario pass or artifact review。 |
Gate rule:
If role readiness is not proven, either reduce scope, add supervision, delay scale or assign a qualified reviewer.
10. 90-Day Rollout Plan
Days 1-15: Scope and baseline
| Work | Output |
|---|---|
| Select 2-3 priority AI domains | KYC, AML, contact center。 |
| Inventory AI roadmap and bottleneck roles | capability demand brief。 |
| Collect existing artifacts and training assets | evidence baseline。 |
| Define governance sponsors | academy operating charter。 |
Days 16-30: Role and skill model
| Work | Output |
|---|---|
| Draft role taxonomy | role profile registry v1。 |
| Draft skill ontology | skill graph v1。 |
| Define proficiency levels | enterprise ladder。 |
| Select high-risk gates | authorization and reviewer rules。 |
Days 31-50: Paths and assessments
| Work | Output |
|---|---|
| Build AI BA, AI PM, Architect, EvalOps and Ops paths | role-based learning paths。 |
| Build KYC, AML and contact center scenarios | scenario packs and rubrics。 |
| Calibrate reviewers | reviewer guide and sample scoring。 |
| Set evidence registry fields | portfolio evidence schema。 |
Days 51-70: Pilot
| Work | Output |
|---|---|
| Run pilot cohort | assessment results and learner feedback。 |
| Review portfolio artifacts | accepted evidence records。 |
| Launch COP clinics | pattern library seed set。 |
| Connect adoption signals from one live use case | telemetry proof。 |
Days 71-90: Govern and scale
| Work | Output |
|---|---|
| Build executive dashboard | readiness, skills debt, adoption and risk view。 |
| Run workforce capability review | investment and treatment decisions。 |
| Update paths based on pilot | path v2。 |
| Prepare scale plan | next domains, reviewers, operating cadence。 |
11. Anti-Patterns and Corrections
| Anti-pattern | Symptom | Correction |
|---|---|---|
| Training catalog first | Many courses, no role readiness | Start from AI portfolio and role taxonomy。 |
| Generic AI literacy for all | Everyone learns the same content | Create literacy tiers and role-specific scenarios。 |
| Badge inflation | People collect certificates but cannot defend artifacts | Require evidence contracts and portfolio review。 |
| Assessment only tests definitions | High quiz scores, weak project decisions | Use messy financial retail scenarios。 |
| COP as social channel | Lots of messages, little reuse | Require reusable patterns, reviewed examples and case clinics。 |
| HR-only ownership | Learning runs separately from AI release and adoption | Create joint academy governance with business, architecture, risk and HR。 |
| No skills debt reporting | Roadmap delays blamed on "capacity" | Track gaps by capability, role and risk exposure。 |
| Adoption ignored | Training looks successful but users do not change workflow | Connect learning paths to production telemetry and manager routines。 |
| Tools taught without controls | Users become faster at unsafe work | Tie tool training to data boundaries, escalation, logging and prohibited use。 |
12. Interview Answers
Question 1: What makes an AI academy different from a training program?
Short answer
An AI academy is a capability product platform. It defines roles, skills, proficiency, evidence, practice, communities and adoption outcomes. A training program usually only delivers content.
Expanded answer
For senior AI roles, I would not measure success by course completion. I would map the AI portfolio to roles such as AI PM, AI BA, Solution Architect, EvalOps, RiskOps, Data Product Manager and Ops Lead. Each role gets target proficiency by capability. Each critical skill gets an evidence contract, such as an AI PRD, eval pack, architecture decision, risk memo or adoption dashboard. Then I would connect learning and assessment data to production outcomes like adoption, quality, overrides, incidents and benefit realization.
Question 2: How would you prove an AI PM is competent?
Short answer
I would ask for role-specific evidence: an AI product brief, value hypothesis, eval release gate, adoption telemetry design and a scale/stop decision memo.
Expanded answer
An AI PM needs to show more than model familiarity. In a KYC onboarding copilot scenario, I would expect them to define the business problem, baseline, target adoption, quality thresholds, critical failures, risk controls, human review, cost per case and launch decision rules. The strongest evidence is a portfolio artifact reviewed by business, EvalOps and risk, plus production signals showing that target users adopted the workflow without quality deterioration.
Question 3: How do you handle skills debt?
Short answer
I treat skills debt as enterprise AI risk: a gap between AI portfolio demand and verified workforce capability.
Expanded answer
I would create a skills debt register by capability and role. For example, if only one architect can review entitlement-aware RAG, that is a key-person and release bottleneck risk. Treatment might be to train Level 4 reviewers, reduce roadmap scope, add supervised review, hire externally or platformize the pattern. The debt should be reviewed quarterly with AI portfolio decisions, because capability gaps can delay releases or create control weaknesses.
Question 4: How should a CBAP-level BA evolve for AI?
Short answer
The BA evolves from requirements documentation to process and evidence architecture.
Expanded answer
A CBAP-level BA already has strong stakeholder, requirements and process skills. In AI, they need to add requirements-to-eval thinking, scenario-based assessment, human oversight design, role redesign, source authority and adoption evidence. A strong AI BA can take a contact center or KYC workflow, identify where AI changes decisions and responsibilities, define exception paths, create acceptance criteria linked to eval, and produce evidence that business, risk, architecture and operations can review.
13. Portfolio Exercise
Create a portfolio pack named:
AI Workforce Capability Academy for Financial Retail Product, Architecture, BA and Operations Teams
Required artifacts
| Artifact | Content |
|---|---|
| Executive capability thesis | Why academy is a workforce capability platform, not training catalog。 |
| Role taxonomy | AI PM, AI BA, Solution Architect, EvalOps, RiskOps, Data Product, AML/KYC Ops, Contact Center Lead。 |
| Skill ontology | skill domains, tasks, evidence, proficiency, risk if absent。 |
| Proficiency ladder | Level 1-5 with authorization meaning and evidence thresholds。 |
| Role-to-capability map | RACI for KYC, AML and contact center AI capabilities。 |
| Learning paths | at least four role-based paths with modules, labs, evidence and recertification triggers。 |
| Assessment design | three financial retail scenarios with rubrics and critical failures。 |
| Evidence registry | schema for portfolio artifacts, reviewers, validity and production links。 |
| Adoption telemetry | dashboard linking readiness to workflow adoption, quality, incidents and benefits。 |
| Governance model | RACI, forums, gates, skills debt register and quarterly review。 |
Defense prompts
Answer these in the portfolio:
| Prompt | Strong answer includes |
|---|---|
| Why not use one AI literacy course for everyone? | role risk, task differences, evidence thresholds, literacy tiers。 |
| How do you know someone can safely own a use case? | scenario assessment, artifact review, supervised practice, production outcome。 |
| How does academy data influence AI portfolio decisions? | readiness heatmap, skills debt, reviewer capacity, scale/stop gate。 |
| What is the BA's unique contribution? | process evidence, human oversight, exception paths, acceptance criteria, stakeholder alignment。 |
| What is the architect's unique contribution? | control plane, source authority, access, logging, fallback, lifecycle governance。 |
14. Final Checklist
Use this checklist before declaring the academy ready.
| Check | Pass condition |
|---|---|
| Role taxonomy | Priority AI roles have clear accountabilities and decision authority。 |
| Skill ontology | Skills are defined as tasks, evidence and risk, not course titles。 |
| Proficiency | Levels map to autonomy, complexity, accountability and authorization。 |
| Learning paths | Paths are role-specific and end in artifacts or scenario assessment。 |
| Assessments | Scenarios include financial retail constraints and critical failures。 |
| Evidence registry | Artifacts have reviewers, validity and production links。 |
| Adoption telemetry | Learning data connects to workflow usage, quality, incidents and benefits。 |
| Governance | Business, HR, architecture, risk, EvalOps and operations share accountability。 |
| Skills debt | Critical gaps are visible, owned and reviewed with the AI portfolio。 |
| COP | Communities produce reusable patterns and improve standards。 |
Final operating statement:
The academy is working when the organization can prove which roles are ready,
which AI capabilities they can safely own,
which evidence supports that claim,
and which workforce risks still need investment.