Trust boundaries
The system accepts untrusted farmer text, audio, images, documents, web pages and tool responses. None of these may directly control system instructions, authorization rules or privileged tools.
Threat register
| Threat | Example | Primary controls |
|---|---|---|
| Prompt injection in a document | A PDF says to ignore policy and recommend a product | Treat retrieved text as data; isolate instructions; allowlisted tools; output validation. |
| Web-source poisoning | SEO page imitates an authority | Domain and publication verification, authority scoring, cross-source checks. |
| Adversarial image | Text or pattern manipulates the vision model | Image sanitization, relevance checks, alternate model or human review. |
| Synthetic/replayed audio | Fabricated farmer statement or identity | Session binding, replay detection where justified, critical-entity confirmation. |
| Citation spoofing | Model invents DOI, page or rule | Resolve identifiers, retain source snapshots, verify quoted passage. |
| Knowledge-base poisoning | Unauthorized document becomes approved | Signed ingestion workflow, approvals, immutable versions, withdrawal. |
| Tool argument injection | Model constructs unsafe URLs or database queries | Typed tool schemas, parameter validation, network allowlists, read-only roles. |
| SSRF/data exfiltration | Research tool accesses internal network | Isolated egress proxy, destination policy, no private-address access. |
| Cross-case leakage | One farmer receives another case’s data | Tenant/case scoping, row-level controls, context construction tests. |
| Resource exhaustion | Oversized video or recursive research loop | File caps, budgets, timeouts, depth and token limits, circuit breakers. |
| Model extraction or prompt leakage | User asks for hidden rules or secrets | No secrets in prompts; response filters; authorization boundaries. |
| Reviewer account abuse | Officer exports sensitive maps | MFA, least privilege, audit, anomaly detection, export controls. |
Research sandbox
Live research should run in a constrained service with:
- read-only internet access;
- blocked private network ranges;
- URL and file-size controls;
- content-type validation;
- malware scanning;
- execution disabled for downloaded content;
- per-case time, query and token budgets;
- source snapshots and hashes;
- and an allow/deny policy for domains when needed.
Instruction hierarchy
- System safety and authorization policy.
- Department-approved workflow and policy rules.
- Typed task and tool contracts.
- Farmer request and case facts.
- Retrieved documents and web content.
Retrieved content cannot promote itself to a higher level.
Red-team scenarios
Test malicious PDFs, poisoned HTML, embedded image instructions, fake government domains, fabricated DOIs, contradictory sources, hidden Unicode, oversized media, repeated tool retries, data-extraction prompts and cross-case references. Every discovered failure becomes a regression test.
Live media threats
| Threat | Control |
|---|---|
| Room hijack or token replay | Short-lived scoped tokens, replay protection, participant notification |
| Malicious/oversized stream | Codec, bitrate, dimension and decode budgets; isolation |
| Replay/synthetic crop video | Interaction challenges, timestamp/scene checks and human review where material |
| Hidden visual prompt injection | Treat all frame text as untrusted data |
| Cross-case frame leakage | Case-scoped storage keys, authorization and provenance tests |