Neuroscientist working at @googlehealth. Trying to tweet about medical AI, but will likely tweet about sustainable transport @janfreyberg@fediscience.org
London
Joined October 2011
- Tweets1.7K
- Following816
- Followers821
- Likes5.1K
Jan Freyberg retweeted
A clinical consultation begins long before words are spoken — from a patient’s gait to their facial cues.
Today, our team at @Google advances AMIE towards real-time video consultations, reaching expert-level performance for the first time.
Jan Freyberg retweeted
Yulin AWS simulator includes a simulated DynamoDB for tests and local development. Tables are held in memory, and every operation is authorized by simulated IAM.
yulinsim.dev/services/dynamo…
Jan Freyberg retweeted
Today, Google Research & @GoogleDeepMind introduce g-AMIE, an extension of our diagnostic AI system based on #Gemini 2.0 Flash. It uses a guardrail that prohibits medical advice sharing & instead provides a summary for a physician to review. Learn more: goo.gle/45shHyT
ALT Asynchronous oversight framework. 1. g-AMIE as well as NP/PA and PCP control groups perform history taking within guardrails, abstaining from individualized medical advice. 2. g-AMIE and control groups generate differential diagnoses (DDx) and management plans. 3. Overseeing physician revises DDx & management plan to ensure patient safety and accountability. 4. Overseeing PCP shares a revised message with the patient. “g-PCP” and “g-NP/PA” refer to providers operating under the same guardrail constraints as g-AMIE.
Jan Freyberg retweeted
🎉 Congratulations to Dr @RobertDudas5 and colleagues @CPFT_NHS on publishing a new book to guide person-centred #MentalHealth care! 📘
Robert has kindly donated 2⃣ signed copies of #ValuesInPsychiatry to @CpftLibrary for everyone to borrow and learn:
➡️ cpft.nhs.uk/research-news/cp…
Jan Freyberg retweeted
Introducing our latest milestone on conversational diagnostic AI g-AMIE
arxiv.org/abs/2507.15743
To harness AI's abundant potential in diagnosis and treatment, we need ways for licensed professionals to oversee and own responsibility for safety-critical decisions.
In clinical practice we handle analogous challenges with a variety of approaches to teamwork, supervision & oversight. But solutions for AI systems are lacking. Inspired by this, we introduce g-AMIE, a framework for Asynchronous Oversight with guardrails that decouple an effective consultation from authorised communication of an individualised diagnosis and management plan.
New work on making AMIE safer and better! Oversight by physicians provides guardrails for AMIE and allows clinicians to review, edit, and then release diagnoses using a purpose-built UI.
Jan Freyberg retweeted
We are excited to share our latest pre-print enabling effective human oversight for AMIE, our research diagnostic dialogue AI. We introduce a new asynchronous oversight paradigm, decoupling history-taking by AMIE from sharing a human-approved diagnosis – a thread 🧵:
Jan Freyberg retweeted
Reliable ground truth is a key obstacle in developing AI systems. In a case study on medical diagnosis, we developed a framework for evaluation with ground truth uncertainty stemming from rater disagreement. This work was finally accepted at Medical Image Analysis – a thread 🧵:
This was released last week so I'm a little late to the party, but here is a paper from my team: multimodal conversational diagnostic AI. We extend previous work on AMIE and enable it to request, reason about, and report on images.
research.google/blog/amie-ga…
More interesting: AMIE was rated better on a whole host of criteria, by both the patient actors, and by specialists. Patient actors had a better experience, felt that they AMIE was better addressed their concerns about things shown in images, and interpreted images correctly more
The full paper is here: gstatic.com/amie/multimodal_…
This was a huge and fun collaboration, with all the coauthors that Alan mentions here :) xitter.cf/alan_karthi/status/191…
As always, there are important limitations we detail in the paper and the results should be interpreted with appropriate caution. Further work is required to safely validate the potential capabilities in prospective settings which we are excited to share more about in due course. Work with an outstanding team including @RyutaroTanno, @KhaledSaab11, @vivnat, @AdamRodmanMD, @timstro, @taotu831, @hardyshakerman, @JanFreyberg, @_cjpark, @yasharmaa, @apalepu13, @arkitus, @weballergy, @valentinlievin, @ckbjimmy, @davidstutz92, @dgtbarrett, @yongcheng16 @SaraM66905 @AvinatanH48021 @joelle_barral @ymatias @pushmeet
Jan Freyberg retweeted
✨New study from our team @GoogleDeepMind @GoogleAI - AMIE goes Multimodal✨
Our research conversational diagnostic AI now fluently considers visual photos/tests. In randomized OSCE study AMIE outperformed PCPs in simulated consultations in which patients uploaded photos of skin concerns, ECG tracings or lab tests. Medical dialogue can hinge critically on multimodal tests like these, so AI systems need to expertly reason about this complex information during a diagnostic conversation. 👀More here: goo.gle/42D0QcB (1/n)
Jan Freyberg retweeted
Building on Articulate Medical Intelligence Explorer — AMIE, our research diagnostic conversational AI agent — today on the blog we share a first of its kind demonstration of a multimodal conversational diagnostic AI agent, multimodal AMIE. Learn more →goo.gle/42D0QcB
ALT We introduce multimodal AMIE: our diagnostic conversational AI that can intelligently request, interpret and reason about visual medical information during a clinical diagnostic conversation. We integrate multimodal perception and reasoning into AMIE through the combination of natively multimodal Gemini models and our state-aware reasoning framework.
Jan Freyberg retweeted
Introducing TxGemma, a family of open models specifically tailored for health settings, building on top of Gemma and Gemini.
developers.googleblog.com/en…
Jan Freyberg retweeted
Excited to introduce our new research paper on AMIE, a conversational diagnostic AI! arxiv.org/pdf/2401.05654.pdf. Fine-tuned on real transcripts and simulated dialogues, AMIE surpasses PCPs in both conversational quality and diagnostic accuracy in a randomized virtual OSCE study.
Happy to introduce AMIE (Articulate Medical Intelligence Explorer) our research LLM for diagnostic conversations. AMIE surpassed Primary Care Drs in conversational quality & diagnostic accuracy in a "virtual OSCE"-style randomized study. Preprint ➡️ arxiv.org/abs/2401.05654 (1/7)
Jan Freyberg retweeted
Pleasure to chat w/ @marilenharo @Nature news alongside @vivnat on behalf of many in a great team (@taotu831 @HardyShakerman @apalepu13 @KhaledSaab11 @RyutaroTanno @JanFreyberg @AziziShekoofeh @thekaransinghal & more).
@vivnat & I share more in the blog: blog.research.google/2024/01…
An experimental chatbot developed by Google researchers matched, or even surpassed, human doctors’ performance at conversing with simulated patients and listing possible diagnoses on the basis of the patients’ medical history. My latest story for @Nature: nature.com/articles/d41586-0…
