AI 扩展计划 / Playbooks

AI Reference Implementation / Pattern Library / Reuse Assurance Playbook

版本: v1.0

675 行AI_REFERENCE_IMPLEMENTATION_PATTERN_LIBRARY_REUSE_ASSURANCE_PLAYBOOK.md

AI Reference Implementation / Pattern Library / Reuse Assurance Playbook

版本: v1.0 日期: 2026-06-30 适用对象: Senior AI PM, AI Architect, Platform PM, Enterprise Architect, CBAP-level BA, Security, Privacy, Model Risk, Compliance, Internal Audit, Financial Retail Product and Operations Leaders

1. Purpose / Audience / Core Principle

Purpose

本 playbook 给团队一套执行方法, 用于把重复 AI solution patterns 转化为 reference implementations 和 reusable pattern library, 并把质量、安全、eval、证据、控制、偏差、生命周期和复用价值一起管理。

适用场景:

多个团队重复做 RAG、policy copilot、evidence extraction、narrative draft、tool gateway 或 human review。
平台团队希望提供可消费的 AI implementation skeleton, 但不希望只交付一堆 starter repos。
风险、合规、安全和审计希望控制证据可复用, 但又不想让本地用例逃避责任。
Senior PM / BA 希望把业务流程模式、控制点和证据包沉淀成作品集级架构资产。

Core Principle

Reuse the pattern only when you can also reuse or regenerate the assurance evidence that makes the pattern safe, useful and governable in the new context.

中文表达:

AI 模式复用不是复制代码, 而是复用经过验证的架构意图、控制证据、eval baseline、threat model、observability 和生命周期纪律。

2. Source Anchors

Source	Official link	Playbook 用法
NIST AI RMF	https://www.nist.gov/itl/ai-risk-management-framework	用 Govern / Map / Measure / Manage 组织 pattern risk、measurement、monitoring and management
ISO/IEC 42001	https://www.iso.org/standard/81230.html	用 AI management system 的 documented information、operation、performance evaluation、management review 和 improvement
ISO/IEC/IEEE 42010	https://www.iso.org/standard/74393.html	用 stakeholder concerns、viewpoints、architecture rationale 和 architecture description 管理 pattern architecture
CNCF Platforms White Paper	https://tag-app-delivery.cncf.io/whitepapers/platforms/	用 platform-as-product、自助消费和 paved path 思想定义平台接口
DORA	https://dora.dev/	用 lead time、change failure、restore 和 delivery flow 衡量 reference implementation 成效
OpenTelemetry	https://opentelemetry.io/docs/	用 traces、metrics、logs 设计 AI pattern observability and evidence trace

使用纪律:

本 playbook 不替代企业 AI governance、model risk、security、privacy、legal、compliance 或 audit 流程。
不把 pattern library 做成全量 service catalog。这里只关注 reference implementations、pattern assurance 和 reusable evidence。
不把 golden paths 当作免审通道。Golden path 可以降低摩擦, 但 reference implementation 必须保留适用边界和偏差治理。

3. Executive Summary

当企业 AI 项目从 3 个变成 30 个时, 最先失控的不是模型, 而是重复实现和重复治理:

same RAG problem
same prompt boundary
same tool permission issue
same eval questions
same threat model
same evidence request
same release gate debate

本 playbook 的执行路径:

1. Identify repeated AI solution pattern
2. Write pattern card and applicability boundary
3. Build reference implementation anatomy
4. Package eval baseline, threat model and control evidence
5. Define reuse qualification and evidence inheritance rules
6. Manage variants and approved deviations
7. Instrument adoption telemetry and reuse ROI
8. Version, support, deprecate and migrate pattern assets

12 个核心资产:

Asset	Output
Pattern card	problem, context, solution, applicability, non-applicability
Reference implementation	runnable skeleton with prompt/RAG/tool/eval/telemetry hooks
Architecture description	stakeholder concerns, views, decisions and rationale
Reuse qualification checklist	determines whether a use case can inherit the pattern
Evidence pack	inherited evidence, local evidence and release decision support
Control mapping	controls linked to implementation checks and evidence
Eval baseline	golden, high-risk, regression, no-answer and red-team suites
Threat model	reusable risks plus local extension points
Variant model	allowed configuration and approved structural variants
Deviation process	risk-owned exception with expiry and compensating controls
Adoption telemetry	qualified reuse, evidence inheritance, lead time, defect and support metrics
Lifecycle plan	owner, version, support state, deprecation and migration

4. Operating Model

4.1 Roles

Role	Accountabilities
Pattern Owner	owns pattern card, applicability, maturity, roadmap and deprecation
Reference Implementation Owner	owns runnable skeleton, package, tests and integration guide
Design Authority	approves pattern admission, mandatory controls, variants and deviations
Platform PM	manages developer experience, consumption flow, support and ROI
AI Architect	owns architecture description, decisions, quality attributes and control hooks
BA / Process Owner	owns workflow fit, business rules, artifacts, exception paths and local qualification
EvalOps Owner	owns eval baseline, thresholds, regression memory and reviewer calibration
Security / Privacy	owns threat model, data boundary, access, logging and abuse tests
Risk / Compliance / Model Risk	challenges control evidence, residual risk and release conditions
Internal Audit	reviews evidence lineage, deviation expiry and operating effectiveness
Use Case Team	consumes the pattern and remains accountable for local evidence

4.2 RACI

Activity	Pattern Owner	Platform PM	AI Architect	BA	Security/Privacy	Risk/Model Risk	Use Case Team
Identify candidate pattern	A/R	R	R	R	C	C	C
Approve pattern admission	C	C	A/R	C	C	C	I
Build reference implementation	C	A/R	R	C	C	C	R
Define eval baseline	C	C	C	C	C	A/R	R
Define threat model	C	C	R	C	A/R	C	C
Run reuse qualification	C	C	R	A/R	C	C	R
Approve deviation	C	C	R	C	C	A/R	R
Monitor reuse ROI	R	A/R	C	C	C	C	C
Deprecate pattern	A/R	R	R	C	C	C	I

5. Execution Roadmap

Phase 1: Pattern Discovery

Step	Action	Evidence
1	Collect repeated AI use cases across portfolio	use case inventory
2	Cluster by problem shape, not org chart	pattern candidate list
3	Identify repeated controls and failure modes	defect and review log
4	Estimate reuse potential and risk	pattern investment brief
5	Select first patterns for reference implementation	design authority decision

Selection criteria:

Criterion	Strong candidate
Repetition	at least three current or near-term use cases
Risk	meaningful customer, regulatory, privacy or operational risk
Assurance effort	repeated eval, threat model, evidence or release review burden
Platform leverage	common connectors, telemetry, gateway or evaluation assets
Business value	faster delivery or better quality materially improves portfolio

Phase 2: Pattern Card

Create a pattern card before building code.

Field	Required content
Pattern name	short stable name, such as `customer-facing-rag`
Business problem	repeated problem in business language
Context	users, workflow, data, risk tier and operating constraints
Forces	speed, accuracy, control, explainability, cost, adoption and audit tension
Solution structure	components and responsibilities
Applicability	where the pattern fits
Non-applicability	where use is prohibited or requires separate design
Mandatory controls	eval, security, privacy, human review, telemetry and evidence
Variants	approved configurable differences
Maturity level	observed, documented, template, reference, assured, managed
Owner	pattern and implementation owners

Phase 3: Reference Implementation Build

Minimum package:

Package area	Contents
Runtime skeleton	service/app skeleton, config schema, sample workflow integration
Prompt pack	system boundary, task prompt, refusal, escalation and source requirement
RAG pack	source registry, ACL filter, chunking and citation contract
Tool gateway	tool registry, auth context, side-effect declaration, approval and audit
Eval harness	test runner, sample suites, rubric, thresholds, report format
Observability	OpenTelemetry trace fields, metrics, logs, dashboard spec
Evidence binder	standard folder/index structure for release and control evidence
Integration guide	how to adopt, configure, test and produce local evidence

Phase 4: Assurance Package

Assurance asset	Minimum standard
Eval baseline	golden, high-risk, regression, no-answer and adversarial samples
Threat model	prompt injection, data exfiltration, unauthorized retrieval, tool abuse
Control mapping	each mandatory control maps to implementation check and evidence
Privacy classification	prompt, retrieval, logs, traces and evidence retention classes
Security tests	ACL, injection, secret/log scan, tool permission and abuse checks
Human review model	reviewer role, queue, sampling, override reason and escalation
Release gate	pass/fail evidence, exception path and rollback condition

Phase 5: Pilot Adoption

Step	Action
1	Select one low-to-medium complexity use case inside pattern scope
2	Run reuse qualification with BA, architect, risk and platform
3	Configure skeleton and add local data/source/workflow bindings
4	Add local eval samples and local threat model extensions
5	Produce inherited plus local evidence pack
6	Release under controlled pilot
7	Measure lead time, defects, deviations, adoption and support burden

Phase 6: Scale and Lifecycle

Action	Output
Promote pattern maturity after successful reuse	maturity update
Add incidents and defects to regression suite	shared learning
Publish version notes and migration guide	consumer update
Review deviations quarterly	expiry and closure
Deprecate weak patterns	no-new-use decision
Retire unsupported assets	migration complete evidence

6. Pattern Taxonomy

Use this taxonomy as a starting map, not a fixed enterprise standard.

Pattern family	Typical use cases	Mandatory assurance focus
Customer-facing RAG	service chatbot, fee policy Q&A, dispute status explanation	source authority, citation, handoff, complaint and vulnerable customer escalation
Internal policy copilot	branch policy search, contact-center SOP assistant	role-based retrieval, policy version, answer boundary
Evidence extraction	KYC document extraction, dispute packet extraction, complaint fact extraction	source span, schema validation, confidence band, human validation
Investigation workbench	AML, fraud, dispute, complaint, collections	evidence graph, analyst accountability, high-risk escalation
Narrative drafting	regulatory narrative, case note, customer letter draft	grounded draft, maker-checker, approval, source trace
Tool gateway	account lookup, case update, workflow task creation	least privilege, approval, idempotency, audit, rollback
Human review workflow	QA sampling, exception review, escalation queue	reviewer capacity, override taxonomy, defect feedback
Eval and regression	shared RAG eval, extraction eval, agent action eval	golden set, high-risk slice, regression memory, threshold
Observability and evidence	release trace, cost, latency, quality, adoption	trace schema, metric contract, evidence lineage

7. Reference Implementation Anatomy Checklist

Area	Completion standard
Pattern identity	pattern id, owner, maturity, support level and approved scope
Architecture views	context, component, data flow, control, deployment and runtime views
Runnable skeleton	can run in a controlled environment with sample data
Configuration model	typed config for model route, source profile, tools, thresholds and telemetry
Prompt/RAG/tool skeleton	includes boundaries, ACL, citations, approvals and audit events
Eval baseline	included in package and runnable by adopting team
Threat model	common threats plus local extension guide
Control mapping	mandatory controls mapped to evidence objects
Telemetry	required traces, metrics, logs and privacy classes
Evidence binder	clear folder/index with inherited and local evidence
Deviation process	how to request, approve, expire and monitor deviations
Lifecycle metadata	version, compatibility, deprecation and migration rules

8. Reuse Qualification Checklist

Run this before a use case adopts a reference implementation.

Question	Green	Amber	Red
Workflow fit	Same workflow shape and decision boundary	Similar workflow with local exceptions	Different decision rights or final decision automation
User population	Same role type and training assumptions	New role with similar controls	External customer or untrained role not covered
Risk tier	Inside approved pattern tier	Slightly higher risk with added controls	Materially higher risk than pattern scope
Data class	Same or lower sensitivity	New sensitive fields with privacy review	Restricted data not allowed by skeleton
Source authority	Approved owner, freshness and ACL exist	Source cleanup required	No source owner or freshness path
Tool action	Read-only or draft-only as baseline	New low-risk write action with controls	Irreversible customer-impacting action
Eval coverage	Baseline covers major cases	Local samples required	Baseline misses core risk cases
Human oversight	Review queue and override reasons ready	Capacity constrained but manageable	No credible oversight path
Telemetry	Required events can be emitted	Partial telemetry with remediation date	Key evidence fields impossible
Support model	Team can operate pattern version	Extra support required	Unsupported fork planned

Decision rule:

Green majority with no red: proceed with standard adoption.
Any amber: document local controls and evidence.
Any red: stop adoption or request formal deviation with design authority and risk owner.

9. Evidence Pack

9.1 Evidence Binder Structure

Section	Contents
01-pattern-card	pattern definition, applicability, non-applicability, owner
02-architecture	views, ADRs, component responsibilities, data and control flow
03-reference-implementation	package version, configuration, test results, integration guide
04-reuse-qualification	fit checklist, decision, local assumptions
05-eval	inherited suites, local samples, run results, reviewer notes
06-threat-model	common threats, local extensions, mitigations
07-security-privacy	ACL tests, DLP/logging review, data class and retention
08-control-mapping	control claims, evidence objects, owners
09-telemetry	trace schema, metrics, dashboard and event completeness
10-deviations	deviation records, compensating controls, expiry
11-release	gate decision, approvals, rollback and monitoring plan
12-lifecycle	version, support state, migration and deprecation notes

9.2 Inherited vs Local Evidence Matrix

Evidence object	Inherit	Localize
Pattern rationale	Yes	Add business outcome and workflow
Architecture skeleton	Yes	Add actual systems and data stores
Prompt boundary	Partial	Add use-case terminology and forbidden advice
RAG source controls	Partial	Add actual source owners and freshness
Tool gateway controls	Partial	Add local tool permissions and action risk
Eval rubric	Yes	Add local samples and thresholds if risk differs
Threat model	Yes	Add channel, data and abuse cases
Privacy class	Partial	Confirm real data class and retention
Release gate	Yes	Add local approvals and residual risk
Observability	Yes	Add outcome metrics and dashboard thresholds

10. Control Mapping Worksheet

Control question	Implementation hook	Evidence
Who owns the pattern and approves changes?	pattern registry and owner field	registry record
Where is the pattern applicable?	pattern card and reuse qualification	approved checklist
How are architecture concerns documented?	architecture views and ADRs	architecture description
How is output quality measured?	eval harness and reviewer rubric	eval report
How is unsafe or unsupported output controlled?	refusal rules, escalation, critical failure gate	safety eval and gate memo
How is prompt injection tested?	adversarial eval suite	red-team report
How is unauthorized retrieval prevented?	ACL filter and entitlement tests	access test result
How are tool actions controlled?	tool contract, approval, idempotency, audit	tool gateway test
How is sensitive logging minimized?	telemetry privacy class and DLP scan	log sample review
How is human oversight proven?	review queue, override reason, QA sample	review metrics
How is runtime drift monitored?	OTel metrics and dashboard	runtime report
How are deviations managed?	deviation workflow with expiry	deviation record
How is continual improvement shown?	regression updates and version notes	release notes and backlog

11. Variant and Deviation Management

11.1 Variant Types

Variant	Allowed without deviation when
Source variant	source registry follows same owner, freshness, ACL and citation contract
Workflow label variant	workflow stages map to required telemetry fields
Model route variant	route is approved for same data class and eval parity passes
Language variant	eval and reviewer coverage exists for that language
Risk-tier variant	stays within approved pattern scope

11.2 Deviation Triggers

Formal deviation is required when:

Internal-only pattern becomes customer-facing.
Read-only tool becomes write or execute action.
New data class enters prompt, retrieval or logs.
Required eval suite is skipped or threshold is relaxed.
Required trace fields cannot be emitted.
Human review is reduced below baseline.
Source freshness or ownership cannot be proven.

11.3 Deviation Record

Field	Content
Deviation ID	stable id tied to pattern and use case
Pattern version	adopted reference implementation version
Requirement changed	baseline rule being changed
Business reason	concrete reason standard path does not fit
Risk impact	customer, regulatory, privacy, security and operational impact
Compensating control	replacement or risk-reduction measure
Evidence	eval, test, review, monitoring and approval evidence
Owner	residual risk owner and implementation owner
Expiry	date or trigger for renewal or closure
Decision	approved, rejected, or approved with restrictions

12. Lifecycle and Versioning

12.1 Version Policy

Change	Version impact
Documentation clarification	patch
New sample or non-breaking eval addition	patch
New optional configuration	minor
New supported variant	minor
Changed prompt boundary or telemetry schema	minor or major, based on compatibility
Changed control requirement	major
Tool action risk change	major
Deprecated source or unsupported model route	major with migration

12.2 Lifecycle States

State	Entry criteria	Consumer action
Candidate	pattern identified and owner assigned	discovery only
Beta	skeleton runs and initial evidence exists	controlled pilot
Approved	design authority approves evidence and scope	standard adoption
Restricted	issue or limitation narrows use	adopt only with approval
Deprecated	replacement exists or risk changed	no new adoption, migrate
Retired	support ended and consumers migrated	remove from production path

12.3 Quarterly Review Agenda

Agenda item	Decision
Reuse telemetry	continue investing, redesign or deprecate
Defects and incidents	add regression, patch skeleton or restrict scope
Deviations	close, renew with stronger control, or reject
Eval drift	update sample sets, thresholds and reviewer rubric
Security/privacy changes	update threat model and logging controls
Platform support	improve docs, packaging, APIs or dashboard
Consumer migration	set migration dates and support plan

13. Adoption Telemetry and Reuse ROI

13.1 Telemetry Contract

Metric	Definition
pattern_adoption_started	use case began qualification for a pattern
pattern_adoption_approved	reuse qualification approved
implementation_configured	skeleton configured for local workflow
inherited_evidence_linked	inherited evidence attached to local evidence pack
local_eval_completed	local eval run completed
deviation_opened	deviation requested
deviation_closed	deviation closed or expired
production_release_completed	pattern-based release entered controlled production
pattern_defect_reported	defect linked to pattern version
regression_added	defect or incident converted into shared regression case

13.2 ROI Metrics

Category	Metrics
Speed	intake-to-pilot lead time, integration days, release evidence cycle time
Quality	eval pass rate, critical defect rate, post-release defect escape
Risk	open deviations, expired deviations, control evidence completeness
Reuse	qualified reuse count, evidence inheritance rate, variant count
Cost	support tickets, maintenance cost, avoided duplicate build effort
Delivery health	DORA-style lead time, change failure and restore signals for pattern consumers
Lifecycle	deprecated consumer count, migration completion, unsupported fork count

13.3 Dashboard Sections

Section	Widgets
Pattern inventory	maturity, owner, version, support state
Adoption funnel	discovered, qualified, configured, pilot, production
Evidence health	missing evidence, stale evidence, inherited/local split
Deviations	open, expired, by pattern and risk tier
Quality	eval pass, regression failures, critical defects
Security/privacy	injection test status, ACL failure, logging issue
Runtime	cost, latency, error rate, trace completeness
ROI	lead time avoided, defect reduction, support burden

14. Platform / Team Interface

14.1 Consumption Flow

Use case intake
  -> pattern match
  -> reuse qualification
  -> skeleton configuration
  -> local evidence generation
  -> deviation review if needed
  -> controlled release
  -> telemetry and lifecycle tracking

14.2 Interface Contract

Interface	Platform provides	Use case team provides
Pattern registry	approved patterns, versions, owners, scopes	selected pattern and business case
Implementation package	code skeleton, config schema, test harness	local configuration and integration
Eval harness	reusable suites and runner	local samples and reviewer sign-off
Threat model	common risks and mitigations	local data/channel/tool extensions
Evidence binder	structure and inherited evidence	local evidence and approvals
Telemetry	required schema and dashboard base	emitted events and outcome joins
Support	documentation, office hours, issue triage	adoption feedback and defect reports
Lifecycle	version notes, migration guides, deprecation	migration execution and exception closure

15. Financial Retail Execution Examples

15.1 Customer-Facing RAG

Work item	Execution
Pattern fit	Authenticated customer service Q&A with approved sources
Reference assets	source registry, citation verifier, refusal and handoff prompt, trace schema
Eval baseline	fees, dispute status, account servicing, ambiguous customer requests
Local evidence	source owners, channel policy, complaint escalation, QA sampling
Deviation trigger	answer becomes final adverse decision or regulated advice

15.2 Internal Policy Copilot

Work item	Execution
Pattern fit	Employee policy lookup inside branch or contact-center workflow
Reference assets	role-based retrieval, SOP version trace, policy answer rubric
Eval baseline	policy conflict, outdated source, escalation boundary
Local evidence	business unit policy sources and training assumptions
Deviation trigger	output is inserted into customer communication without review

15.3 AML Investigation Workbench

Work item	Execution
Pattern fit	Analyst-owned investigation support and narrative preparation
Reference assets	evidence timeline, case summary, narrative draft, QA sampling
Eval baseline	missed evidence, unsupported escalation, high-risk alert
Local evidence	alert types, reviewer capacity, SAR-related data boundary
Deviation trigger	AI proposes final closure or suspicious activity conclusion

15.4 KYC Evidence Extraction

Work item	Execution
Pattern fit	Extract structured evidence from onboarding documents
Reference assets	document classifier, extraction schema, source span, confidence band
Eval baseline	beneficial ownership, missing document, expiry, high-risk country
Local evidence	product entity types, manual validation workflow
Deviation trigger	extraction drives straight-through approval without human validation

15.5 Dispute Evidence Assistant

Work item	Execution
Pattern fit	Build evidence timeline and draft dispute packet
Reference assets	event timeline, rule retrieval, draft packet, maker-checker
Eval baseline	reason code, provisional credit, recurring charge, merchant evidence
Local evidence	network rules, customer letter approval, operations QA
Deviation trigger	automatic chargeback submission or customer denial letter

15.6 Regulatory Reporting Narrative Draft

Work item	Execution
Pattern fit	Draft narrative from approved metrics and lineage
Reference assets	metric contract, source trace, maker-checker, attestation boundary
Eval baseline	variance explanation, restatement, late adjustment, unsupported cause
Local evidence	report owner, data lineage, reviewer sign-off
Deviation trigger	external filing text generated without authorized review

16. Anti-Patterns

Anti-pattern	Consequence	Replacement
Code repository labeled as reference implementation	Teams inherit implementation but not controls	Package skeleton with eval, threat model, controls and evidence
Pattern card without runnable asset	Teams still rebuild differently	Add minimal reference implementation and test harness
Assurance after adoption	Controls are retrofitted and weak	Build assurance package before pilot adoption
Reuse without qualification	Pattern applied outside its assumptions	Run fit checklist and red-stop on high-risk mismatch
Silent forks	Platform cannot support or audit consumers	Require versioned adoption record
Deviation by conversation	Risk acceptance is untraceable	Deviation record with owner, evidence and expiry
Eval copied without local samples	Local failures hidden	Inherit rubric, add local high-risk cases
Threat model copied blindly	New channel or tool risk missed	Local threat model extension
Evidence pack as static archive	Evidence becomes stale	Link evidence to versions, releases and quarterly review
Reuse value measured by asset count	Library looks large but does not improve outcomes	Measure qualified reuse, lead time, defects and control health

17. Interview Answers

Q1: How would you turn repeated AI use cases into reusable reference implementations?

I would start by clustering repeated use cases by problem shape, such as customer-facing RAG, internal policy copilot, evidence extraction, investigation workbench or narrative drafting. For each high-value pattern I would create a pattern card, runnable reference implementation, eval baseline, threat model, control mapping, telemetry contract and evidence binder. Then I would define reuse qualification, variant rules, deviation process and lifecycle ownership. The goal is to reuse assurance, not just code.

Q2: What makes a reference implementation trustworthy in a regulated financial environment?

It needs explicit applicability boundaries, architecture rationale, quality and safety evals, threat model, privacy and security controls, human review design, observability, release evidence and lifecycle management. It also needs a clear rule for what evidence can be inherited and what must be regenerated locally. Without that, the reference implementation may speed delivery while hiding risk.

Q3: How do you avoid over-standardizing AI patterns?

I separate mandatory controls from configurable variants. The pattern should standardize the things that reduce repeated risk: telemetry, eval harness, source authority, tool gateway controls, evidence structure and deviation process. It should allow local workflow, source, language, case type and business outcome differences where they do not break assumptions. When assumptions change, use deviation or a new pattern.

Q4: How would you explain reusable control evidence to audit?

Reusable control evidence means the common implementation and control tests have already been documented, versioned and reviewed. A local use case can inherit those specific evidence objects, such as prompt injection test design or telemetry schema, but it still must prove local source authority, data class, workflow fit, eval samples, approvals and residual risk. The evidence binder shows inherited versus local evidence and the pattern version.

Q5: How do you measure whether a pattern library is working?

I would measure qualified reuse, lead time reduction, evidence inheritance rate, deviation rate, post-release defect escape, support burden, regression reuse, deprecation compliance and delivery health. A good library reduces repeated work and repeated risk. A bad library has many documents, few qualified adopters, many silent forks and stale evidence.

Q6: What is the risk of using a golden path as a substitute for architecture review?

A golden path helps teams move quickly through a recommended delivery route, but it does not automatically prove that a use case fits the risk, data, workflow and control assumptions. For low-risk cases, golden path plus standard evidence may be enough. For high-risk financial retail AI, architecture review still needs reuse qualification, local evidence and deviation review where assumptions change.

18. Portfolio Exercise

Assignment

Create a reference implementation and pattern assurance pack for two financial retail AI patterns:

Option	Suggested scope
Customer-facing RAG	policy answer, citation, handoff and complaint escalation
Internal policy copilot	employee SOP search and workflow guidance
AML investigation workbench	evidence timeline, case summary and narrative draft
KYC evidence extraction	document extraction and validation workflow
Dispute evidence assistant	evidence packet and rule-grounded draft
Regulatory reporting narrative draft	data-lineage-grounded narrative and maker-checker

Deliverables

Deliverable	Completion standard
Pattern cards	problem, context, forces, applicability and non-applicability
Reference implementation anatomy	skeleton, prompt/RAG/tool/eval/telemetry/evidence components
Architecture description	views and architecture decisions using stakeholder concerns
Reuse qualification checklist	green/amber/red fit assessment
Evidence pack	inherited vs local evidence matrix
Control mapping	governance, security, privacy, eval, human review and monitoring
Eval baseline	golden, high-risk, regression, no-answer and adversarial suites
Threat model	reusable threats plus local extension
Deviation example	one approved deviation with compensating control and expiry
Lifecycle plan	version policy, support state and deprecation trigger
Adoption telemetry	reuse funnel, evidence health, defect and ROI dashboard
Executive memo	recommend which pattern to scale first and why

Scoring Rubric

Dimension	Strong answer
Architecture rigor	Clear distinction between pattern, template, golden path and reference implementation
Assurance rigor	Eval, threat model, control mapping and evidence are first-class
Reuse judgment	Qualification prevents blind copying
Financial retail realism	Examples reflect customer impact, AML/KYC/dispute/reporting controls
Platform thinking	Clear interface between platform and use case teams
Lifecycle discipline	Versioning, deviations, deprecation and migration are explicit
Portfolio value	ROI connects speed, quality, risk and support effort

19. Quality Bar

A reference implementation pattern is ready for senior review only if:

It has a named owner, maturity level, version and support state.
It has a pattern card with applicability and non-applicability.
It includes a runnable skeleton, not only documents.
It packages prompt/RAG/tool gateway assumptions explicitly.
It has eval baseline, threat model, control mapping and evidence binder.
It defines what evidence can be inherited and what must be local.
It has reuse qualification and red-stop conditions.
It supports variant management and approved deviations with expiry.
It emits telemetry for adoption, evidence health, quality, cost, latency and defects.
It has lifecycle rules for versioning, deprecation and migration.

Final principle:

Do not scale AI by copying demos. Scale AI by turning repeated patterns into governed reference implementations with reusable evidence and explicit local accountability.