Decision
Build Kissan Ki Pehchan as a live Urdu audio-video agricultural consultation and reasoning platform. A farmer opens a WebRTC call, speaks naturally, shows the crop in real time and follows camera guidance. The system assembles local field context, retrieves approved knowledge, conducts controlled external research when justified, performs differential analysis, verifies exact regulatory claims and returns a spoken Urdu answer or human escalation.
Principal flow
camera · microphone · Urdu transcript · network adaptation
room · audio/video tracks · TURN · telemetry · officer join
STT/TTS · frame scoring · guided capture · case state
hypotheses · questions · retrieval · context tools · web research
Farmer outcomes
- A natural live video call rather than a sequence of uploads.
- Pakistani Urdu and common Urdu-English code-switching.
- Real-time camera instructions: move closer, hold still, show the underside, show the entire plant or walk to another area.
- Only questions and captures that materially reduce uncertainty.
- Likely diagnosis, important alternatives, evidence and limitations.
- Safe immediate measures and locally authorized treatment where available.
- Seamless degradation to reduced video, audio plus stills, push-to-talk or queued case.
- Visible escalation status and optional live officer participation.
Department outcomes
- Control approved sources, treatment rules, session policies and recording permissions.
- Inspect transcript, selected frames, evidence, tool calls, model/prompt versions and policy results.
- Join uncertain or high-risk calls without restarting the farmer journey.
- Withdraw documents/rules without retraining.
- Benchmark speech, live-video guidance, frame selection, retrieval, research, diagnosis and comprehension.
- Route between cloud and sovereign providers.
- Detect human-reviewed spatial patterns.
Pilot decision
The pilot validates the live-consultation and reasoning thesis, not province-scale infrastructure. Start with one primary crop, one secondary crop, one or two districts, 10–20 common conditions, approved documents, a limited treatment-rule set and a controlled farmer/officer cohort. LiveKit is part of the pilot baseline; multi-region media clusters are not.
Go/no-go questions
- Can farmers establish and complete a live call on target rural networks and devices?
- Does adaptive video guidance produce diagnostically useful views?
- Does Urdu STT preserve crop, chemical, quantity and location entities?
- Do adaptive questions obtain useful missing evidence?
- Does hierarchical retrieval find decisive passages and context?
- Does external research improve unusual cases without laundering authority?
- Are exact treatment claims fully verified?
- Can officers join or review within funded SLAs?
- Is cost per completed live advisory sustainable?