Google has started testing its AMIE diagnostic AI system in live video consultations, moving the research prototype beyond the text-only chat interface it has used since its debut in early 2024. According to AI News, the shift puts AMIE in front of a video feed instead of a transcript window, a change that sounds incremental but reworks the core problem the system was built to solve.
AMIE, short for Articulate Medical Intelligence Explorer, is a Google Research project designed to conduct patient history-taking and diagnostic reasoning through conversation. In its first published evaluation, AMIE was tested against primary care physicians in simulated consultations with trained actor-patients, using an OSCE-style rubric that scores history-taking, diagnostic accuracy, and communication quality. A later iteration added multimodal input, letting patients share photos — a rash, a skin lesion, a lab report — mid-conversation so AMIE could reason over both text and images. Video consultations are the next logical step in that progression, and also the hardest one.
From transcript to webcam: what actually changes
A photo shared mid-chat is a static artifact: the model gets one clean frame, on its own schedule, with no time pressure. A video consultation removes all three of those comforts. The system has to:
- Interpret a continuous visual stream rather than a single labeled image
- Track cues that only show up in motion — gait, tremor, breathing pattern, facial affect — not just static appearance
- Hold a real-time conversational turn structure, where it can't pause indefinitely to reason before responding, the way it can with a static text prompt
- Fuse audio, video, and dialogue history into one diagnostic judgment instead of reasoning over text with an occasional image attached
That last point is the real engineering shift. Multimodal-with-attachments and multimodal-live are different problems: the first is retrieval-augmented reasoning over a fixed input, the second is closer to streaming perception, where the model has to decide, turn by turn, what visual information is diagnostically relevant and what to ignore.
Why this is a harder test than the text benchmark
AMIE's text-based results were notable precisely because the format played to a language model's strengths: unlimited time per turn, no ambiguity about what counted as the input, and a scoring rubric built for transcripts. Video consultations strip out most of that slack. Latency now matters clinically, not just for user experience — a diagnostic assistant that takes ten seconds to react to a visible tremor isn't behaving like a clinician. In our estimation, Google is likely reusing infrastructure from its broader live-multimodal efforts rather than building a bespoke pipeline for AMIE, though the story doesn't specify this.
What it means for builders working on clinical or high-stakes agents
Google has been explicit that AMIE remains a research prototype, not a deployed clinical tool, and that framing matters for anyone building similar systems:
- Evaluation has to match the modality. A benchmark built for text transcripts won't catch failures that only appear in live video — missed visual cues, mistimed turns, or dropped context across a long stream.
- Live multimodal agents need a different safety net. Once a system reasons over a continuous feed instead of discrete inputs, you lose the natural checkpoint of a fixed, inspectable input. Logging and replaying what a live session actually perceived becomes its own infrastructure problem.
- Domain-specific grounding still trumps general capability. A generalist video-reasoning model won't automatically know that a hand tremor is diagnostically relevant and a fidgeting hand isn't — that judgment is what the medical training data and rubric-based evaluation are meant to instill.
For teams building agents in regulated or high-stakes domains — healthcare, but also legal intake, financial advice, or safety-critical support — AMIE's staged rollout (text, then image attachments, then live video) is a reasonable template: prove the reasoning works on the easiest input format before adding the harder ones.
AiiN's takeaway
The interesting part of this story isn't that Google is testing a video-capable medical AI — it's that AMIE's team is treating video as a distinct capability to validate, rather than assuming success on text and images transfers automatically. That's the right instinct, and it's one a lot of teams shipping multimodal agents skip. Text and image benchmarks are cheap to run and easy to publish; real-time video evaluation is neither, which is probably why this is still described as testing rather than a product launch. Anyone building a live multimodal agent — medical or not — should read this as a reminder that the model can now see video and the model can now reason correctly over video in real time are two very different claims, and only the second one is worth shipping.