Versioned assets
Maintain registries for:
- reasoning, speech, vision, embedding and reranking models;
- system prompts and structured output contracts;
- workflows and tool definitions;
- documents, chapter summaries and indices;
- policy rules and calculators;
- evaluation sets and scoring code;
- mobile, API and infrastructure releases.
Model record
Each model entry should include provider, exact identifier or checkpoint, licence, release date, input/output modes, context limits, serving configuration, benchmark results, subgroup results, cost, latency, data-handling terms, approval status and rollback target.
Promotion path
Candidate
→ offline benchmark
→ expert review
→ safety and adversarial suite
→ shadow mode
→ limited canary
→ controlled expansion
→ approved production
Promotion is based on demonstrated system-level improvement, not novelty or vendor ranking.
Prompt and workflow changes
Treat prompts as controlled software:
- review diffs;
- run regression and safety suites;
- test structured output validity;
- shadow material changes;
- record approver and rationale;
- support instant rollback.
Knowledge operations
Document updates trigger parsing, chapter/section generation, metadata checks, indexing, retrieval tests and publication approval. Withdrawal must immediately suppress affected chunks and identify impacted prior cases.
Monitoring and drift
Track:
- speech error by terminology and subgroup;
- image and diagnosis performance by crop and district;
- retrieval miss patterns;
- source-age and authority distribution;
- model calibration;
- policy-rule coverage;
- farmer and reviewer correction patterns;
- and provider behaviour or cost changes.
Feedback loop
Feedback becomes an improvement candidate only after validation. Separate:
- user dissatisfaction;
- transcription correction;
- factual correction;
- new evidence;
- reviewer disagreement;
- policy change;
- and confirmed outcome.
Do not fine-tune or alter retrieval automatically from raw thumbs-up/down data.
Rollback
Rollback plans must cover model routing, prompt, workflow, source publication, index, policy rule and application version. Preserve compatible schemas or provide migrations.